Re: [Patches] Any remaining uses of option groups?

[patches-bounces@xxxxxxxxxx]

On 2013-07-08 at 10:00, Mark Hatle wrote:
>On 6/13/13 3:15 PM, Joseph S. Myers wrote:
>> My previous message removing a bitrotten and ill-conceived option group
>> <http://www.eglibc.org/archives/patches/msg01257.html> did not elicit any
>> responses expressing interest in option groups at all.  So, I now ask:
>> does anyone have a current use case for the option groups feature?  That
>> is, is there still any significant use of EGLIBC in building new embedded
>> GNU/Linux systems that are so space-constrained that the few MB taken up
>> by all the EGLIBC shared libraries put together is so significant that the
>> limited proportion of that space that can be saved through option groups
>> actually matters?
>
>We (Wind River) and we (Yocto Project) are still using this.  It's one of the
>ways we generate very small filesystems.  The number of users for the option
>groups is quite small, but it is a place where it's used.
>
>The majority of the option groups that I know are commonly enabled in a small
>configuration are:
>
>OPTION_EGLIBC_ADVANCED_INET6
>OPTION_EGLIBC_INET
>OPTION_EGLIBC_LIBM
>OPTION_EGLIBC_CRYPT
>OPTION_EGLIBC_LIBM_BIG
>OPTION_EGLIBC_UTMP
>OPTION_POSIX_REGEXP
>OPTION_EGLIBC_GETLOGIN
>OPTION_POSIX_REGEXP
>OPTION_EGLIBC_NIS
>
>Most others are disabled.
>
>See:
>http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta-yocto/conf/distro/poky-tiny.conf 
>
>(look for DISTRO_FEATURES*)

I should chime in here as poky-tiny's author. There was significant interest
from industry for small images. My 2011 presentation at the Embedded Linux
Conference Europe covers some of the motivation (slide 4/47):

http://elinux.org/images/2/2b/Elce11_hart.pdf

Which mentions things like SoC's where memory is very expensive, mass-market
where pennies matter, performance, maintenance, etc. as reasons why it still
makes sense to reduce the image size as much as possible. Some real world
examples follow on slide 5/47, including digital camers, medical devices,network
boot ramfs, etc.

The rest of the presentation outlines how to go about reducing image size purely
through configuration changes (no sourcecode changes). The libc option groups
are an integral part of that strategy. Without those, we have the following
options for meeting the demand:

1) Static linking
   This is generally unsupported and can result in larger images for all but the
   simplest of images (single binary for example)

2) library level granularity
   We could just not install certain libraries, but I believe this would be
   challenging given the internal dependencies and likely not granular enough to
   be generally useful.

3) object level granularity
   http://www.linuxjournal.com/article/5457
   This is something I did in a previous life as an intern at IBM. It's possible
   to do something like this for oe/yocto, but it would be a lot of overhead and
   the benefit would be lost to non-oe/yocto users.

4) promote uclibc for small images
   I hear a lot of pushback whenever this is mentioned. Some of this is FUD.
   Some of this is probably valid compatibility, performance concerns, or
   quality concerns. It's difficult to compete with a project like [e]glibc that
   has such a huge user-base reporting issues over so long a period.

I believe there is value in maintaining the option groups. If there are problems
with maintaining the implementation in its current form, perhaps we could
discuss ways to address those, rather than dropping the functionality entirely?

Thanks,

Darren Hart



>
>and
>
>http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-core/eglibc/eglibc-options.inc
>
>The above is where the translation from the DISTRO_FEATURES turns into the
>option groups.
>
>> I suspect that option groups are an idea whose time is past: that the
>> space constraints that were relevant when option groups were added six and
>> a half years ago are not the space constraints that are relevant today,
>> that today's embedded GNU/Linux systems have vastly more software than was
>> usual then, and that any residual use for the feature is outweighed by the
>> extra complexity it introduces, the combinatorial explosion of different
>> feature combinations, mostly untested and likely broken, and the extra
>> difficulty it adds to merges from glibc.  Thus, I think it may well be
>> time to remove this feature, and with it the vast bulk of the divergence
>> from glibc.
>
>I think they are still useful, but I agree.  The usage of them is diminishing
>over time.  I am starting to see a new class of '32-bit microcontrollers' that
>are using Linux (they're not really microcontrollers, they're traditional CPUs
>that are being sold into the microcontroller market...  This is pretty much the
>last remaining place where this work is useful.
>
>> The same applies to the changes to make --disable-versioning work:
>> although those don't add complexity in the same way, --disable-versioning
>> is also an option to make a small space gain at the expense of breaking
>> ABI compatibility with the default configuration.  So it may well be the
>> case that removing --disable-versioning from glibc (and so from EGLIBC)
>> would be better than merging the --disable-versioning fixes to glibc.
>
>I suspect this is far less used then even the option groups.
>
>> My suggestion regarding both of the above is that space-saving
>> configuration options are only now appropriate where they change neither
>> API nor ABI (so it's still reasonable to be able to eliminate some of the
>> IFUNC variants of a function, for example, as that doesn't affect the
>> public user interface).  Other things can already be removed by the user
>> at the whole-file level without needing any special configuration support
>> (e.g. locales or gconv modules).
>
>I agree this is the major usage today.  The option groups is really a limited
>user base.  I'd easily estimate it at less then 1% of the current embedded users
>of Wind River and the Yocto Project.
>
>> An underlying principle here: I have long considered there to be a
>> distinction between EGLIBC the project to enhance glibc for embedded
>> systems, and EGLIBC the separate source tree.  The goal of enhancing
>> embedded system support (meaning embedded systems being built now, rather
>> than those of 2006) can now be fulfilled in the context of the glibc
>> source tree.  The separate source tree was a workaround for former
>> dysfunction in glibc development.  That reason for its existence no longer
>> exists, and the remaining use of the separate source tree is now (a) to
>> hold residual changes relative to glibc, as listed at
>> <http://www.eglibc.org/archives/patches/msg01261.html>, until those
>> features can be merged into glibc in some form, and (b) a few bits of
>> compatibility code to help existing EGLIBC users where e.g. a makefile
>> variable name changed in the course of merging a feature into glibc.
>>
>> If you care about some some or all of the residual areas of difference, I
>> strongly encourage you to join in the process of merging patches to glibc
>> so the areas you care about cease to be differences between the two trees
>> at all and the separate tree can be made obsolete for you.  Active glibc
>> development means that differences between the two trees are quite likely
>> to become bitrotten in EGLIBC over time without someone actively watching
>> them and testing in those areas.
>>
>
>The two patches I care about right now (that I'm aware of) are the option groups
>and the 'ldconfig'/'ldd' to not require bash.  For the former, I don't have much
>insight into how invasive they are.  For the later, the patches were submitted
>-- and I know there is a pending action to get these submitted to the regular
>glibc group.  I believe the only thing preventing it right now is some type of
>copyright assignment.
>
>--Mark
>_______________________________________________
>Patches mailing list
>Patches@xxxxxxxxxx
>http://eglibc.org/cgi-bin/mailman/listinfo/patches




_______________________________________________
Patches mailing list
Patches@xxxxxxxxxx
http://eglibc.org/cgi-bin/mailman/listinfo/patches