replication of stuff in /usr/gnu

[Garrett D'Amore]

Stephen Hahn wrote:
> * Garrett D'Amore <garrett at damore.org> [2007-07-06 10:59]:
>   
>> Stephen Hahn wrote:
>>     
>>> * Garrett D'Amore <garrett at damore.org> [2007-07-03 15:12]:
>>>  (Although you find --help and --version pointless, many *ix users now
>>>  expect commands to process these options.)
>>>       
>> I would expect that to be true for commands that actually have some 
>> non-trivial purpose.  E.g. for GNU grep this argument holds water.  But 
>> for "true" to have --version and --help?  It seems a bit of a stretch.
>>
>> If we expect *all* commands to support this, then we should just 
>> integrate that into our own command set.  But I think that this is not a 
>> realistic approach.
>>     
>  
>   (It seems this week that all threads lead to CLIP.)  The CLIP spec
>   specifically asks for help and version options in 6.1 and 6.2.  That
>   is, all long option commands should have --version and --help, and
>   that short option ("getopt-compliant") commands should support -V and
>   -?.
>
>   So, we've already elected that approach.
>   

Even for commands that take *no* options?  That's a bit of a surprise!

>   
>   
>>>  The current choice to ship GNU command variants was made to economize
>>>  utility developer time, in that it's a questionable use of time to
>>>  enhance our commands to be supersets of their Solaris-historic,
>>>  XPG4/XPG6, and GNU behaviours, 
>>>       
>> Uh, I disagree with that statement.  I've specifically made changes to 
>> code to eliminate redundancy between /usr/bin and /usr/xpg4, when the 
>> /usr/xpg4 version was clearly a superset of /usr/bin.
>>     
>  
>   I'll stand by my statement:  it was a questionable use of time to do
>   that enhancement.  Perhaps your other priorities allowed you to
>   resolve that question easily; in general, I would be suspicious.  
>   


In my case, it was something easy to do as an external developer.  It 
didn't fall within normal job duties, so yes, it was easily resolved.

>   
>>> rather than to ship the upstream
>>>  versions in a separate location.  During the discussion that led up to
>>>  2007/048, it was noted that deviations from the standard GNU coreutils
>>>  installation pattern would also lead to dissatisfied populations; as a
>>>  result, we elected to minimize the differences in our installation (to
>>>  omitting the su(1M) variant and following the "alternative
>>>  environment" pattern for /usr/xpg[46]/bin).  People assembling
>>>  distributions have the SUNWgnu-coreutils package boundary to base
>>>  their decision on, or can repackage the SFW proto area to their
>>>  liking.
>>>
>>>  I suppose my question is "what engineering quality are you trying to
>>>  optimize?", since your example involves 12 pathnames and ~200k of disk
>>>       
>> Having multiple versions of commands, beyond the waste of disk and 
>> pathnames, causes other issues:
>>
>> 1) increased build times.
>>     
>
>   PATH is a list, not a set:  the first command found is the one
>   to be executed.  Or are you talking about the cost of additional
>   directory entries from any new, not-build-involved command?
>   

I'm talking about the time to build multiple versions of the same 
program.  I guess it doesn't matter, since these are in sfw and not in 
the ON consolidation.


>   
>> 2) increased variability in the environment... making debug and 
>> diagnosis of bugs, and maintenance incrementally more complex
>>     
>
>   PATH is a well-known environment variable.  The default environment
>   has not been changed.
>   

But users will have changes.  Else why bother to deliver the commands in 
the first place.

>   
>> 3) in the case of the GNU variants, I submit that the GNU variants of 
>> these trivial utilities actually have a negative impact on 
>> performance... the GNU utilities are clearly *larger* than the Sun 
>> versions, and I can only imagine that this has an increased negative 
>> impact on cache, etc.
>>     
>
>   But, to run the GNU variant, you must actually change your path to
>   invoke it--presumably meaning that you were willing to trade absolute
>   performance for a known variant (used more widely than the historic
>   Solaris version in most cases).
>   

*BUT*, I may have good reasons to have GNU variants in my path first... 
because I prefer the GNU versions (for one reason or another... in my 
*particular* case I don't, but someone else might) because they offer 
different functionality.

If this is the case, should I also pay a performance penalty for these 
other commands which have no difference?  (In particular I'm thinking 
about /bin/true and similar commands which may be called from shell 
scripts.)

Note on shared systems (such as Sun Ray servers), the cache impact 
affects *multiple* users, the performance impact is not limited to just 
the user using the GNU program.

I agree that the impact is *small*, but why pay *any* such cost if there 
is no benefit in doing so?

>   
>> The cavalier attitude that "its only 200k" is symptomatic of a larger 
>> problem, which is that certain developers have stopped caring about 
>> performance, size, etc... the assumption is that Moore's Law overcomes 
>> sloppy engineering.
>>     
>
>   Actually, I don't feel that any of my reasoning in this space is
>   cavalier or sloppy...
>   

The 200k argument certainly was, IMO, cavalier.  Just because you have 
infinite disk space (and other system resources) to burn doesn't mean 
everyone else does.

>   
>   
>> I reject that attitude; this is how we got into a 
>> situation that we are in today:  I used to be able to boot and use 
>> productively a 33MHz 486 with 32MB of RAM.   Today, I need at least 
>> 512MB and 1GHz cpu to have the same responsiveness that I used to enjoy 
>> back on that 486 15 years ago.  And fundamentally, I'm not doing much 
>> different... running an Xserver, a few xterms, and a few vi sessions.  
>> (Yes, there's some new eye candy on the desktop, new hardware support, 
>> etc.  But a lot of what is occupying my system resources ... memory, 
>> cache, disk, and cpu cycles ... is code that qualifies as "bloat".)
>>     
>  
>   ... and I was around for all of this change, too.  I'm sorry you feel
>   that this set of utilities is somehow connected to a lack of progress
>   in system performance; including the large set of GNU utilities, in
>   this fashion, is considered progress in terms of system usability.
>   Retarding that progress for a hypothetical loss of performance doesn't
>   feel like the correct trade-off.
>   

I'm saying that this is just one more straw on the camel's back.

For a large number of system utilities, I agree that the GNU versions 
offer an EOU enhancement.  But I'm also saying that there is a 
significant set of GNU utilities for which this is *not* true.

Understand, please, that GNU coreutils is intended for use on systems 
that do not otherwise have these utilities.  But for systems like 
Solaris, which have a native version of commands like /bin/true, 
providing a 2nd version of the same command, which offers no difference 
in functionality, seems largely wasteful to me.

>> But then again, I've had in the past year to make Linux work on an 8MB 
>> system, and had to develop a thin-client application that fit within 
>> 512K of flash.  And more recently, I've been working on IP forwarding 
>> performance, where each extra branch in IP costs about .1% to .2% hit in 
>> the number of packets per second that the system can forward.
>>
>> Waste is still bad.  Moore's Law notwithstanding.  I will still tend to 
>> hunt down (and destroy, as much as possible) bloat where I find it, 
>> particularly where that bloat serves no useful purpose.
>>     
>  
>   I suspect strongly there are more rewarding veins of waste to mine.
>   

Probably true.  But yours is also low-hanging.  And more to the point, I 
hope to prevent *further* growth here.

I would really, really like to see justification for *any* new utility 
added to the system.  Where there is different functionality that users 
or layered software will notice, then I agree the EOU probably 
overrides.  But where there is no difference, then I'd argue against 
wasting the system resources.

    -- Garrett