2009/235 dladm Possible Values List

[Mike Shapiro]

On Tue, Apr 14, 2009 at 01:18:47PM -0400, James Carlson wrote:
> Garrett D'Amore writes:
> > Unless some specific real world tests (e.g. TCP throughput or UDP stress 
> > tests) show otherwise, I'm disinclined to believe that there is any 
> > reason an administrator would need to know about the underlying hardware 
> > DMA limitations, or that the "optimum" value is anything other than the 
> > largest supported value.
> 
> Even then, I suspect that this may be more of a man page or whitepaper
> issue.
> 
> On a given L2 network, all of the links must be configured with
> *exactly* the same MTU; neither larger nor smaller.  There's no good
> way (at least today) to tell whether all of the systems are actually
> configured that way.  You just have to hope so, because the penalty
> for getting it wrong is a set of very obscure failure modes.  That's
> part of the reason why the IEEE hasn't endorsed jumbograms -- they're
> fragile.
> 
> The implication of that network-wide configuration issue is that the
> administrator must choose an MTU value based not just on local
> hardware considerations, but on the capabilities and configuration of
> all of the other nodes on his L2 network.
> 
> That's hard, and it's certainly not as simple as a single "preferred"
> value (unless that preferred value happens to be exactly 1500, the
> standards-conforming and safe default).  In fact, I'd expect that
> there are devices that have a set of multiple "sweet spot" MTU values
> where buffers and other detailed components all happen to line up
> perfectly.
> 
> Getting all of that information into a user interface seems hopeless
> to me, so I'm in favor of having the dladm command line display what
> the hardware can do, and leaving high-level network design and tuning
> to other documentation and/or tools.

I agree with the above, but you're describing the rest of the jumbo issue
beyond what I'm getting at.  There are two completely different questions:

(a) What is the jumbo size I ought to set on this L2 network
    given what is configured elsewhere on the network?

(b) What is the jumbo size I ought to use with this card
    for optimal large MTU performance on the card?

You're talking about (a), I'm talking about (b).  Indeed an overall
deployment may need to consider both, but that isn't a reason to
decree this is all hopeless and therefore stick to the nothing we do now.

This is a significant issue for server and storage administrators,
as is generally all issues of making Solaris networking perform better,
and it's not effective to continue solving every problem with documentation.

As I said, the optimal large transfer size is a function of the hardware
and the driver -- there is no earthly reason that can't become an r/o
driver property of some sort so as the h/w, firmware, or driver evolve,
our users aren't sitting around looking at some out-of-date wiki
or have some out-of-date config script, which is what they all have today.

-Mike

-- 
Mike Shapiro, Sun Microsystems Open Storage / Fishworks. blogs.sun.com/mws/