To-Do list for glibc 2.3

The IA-64 ABI group introduced direct support for thread-local storage in the ABI and ELF. This has been extended to other architectures. To support the changes in the ELF format changes to the dynamic linker are needed. A description of the currently known ABIs can be found in this PDF.

Implement the new locale model described in this document. This requires fast handling of thread-local data. This item is only for the implementation of the new model on the x86 architecture. Each architecture needs its own owner and some of them require substantial amount work.

Some locales have problems to be described perfectly since the wide variety of charsets to be used limits the characters which can be used to the least common denominator. This can be changed by using transliteration internally which means unknown characters in the charset used are substituted according to the rules present in the LC_CTYPE category data.

The ISO 14651 table used in glibc 2.2 are very old and do not reflect the current state-of-the-art. New tables are available but using them requires changes to the localedef program. New syntaxes got introduced and probably some semantic changed as well.

The getifaddrs function should be implemented. See this document for more information.

The current test suite is not testing several aspects of the library at all:

• Functionality only root can use isn't tested. There should be a special check-root make goal.
• Sporadic testing of the namespace cleanliness of headers and the library can be performed by some scripts which are available (from drepper). These scripts should be integrated in the normal test suite.
• This test might not be necessary much longer (see the optimization list). It should never be the case that one function is called by two exported names. E.g., at no time should open and __open be used together. This would create one unnecessary PLT entry in the DSO.

Update: I've implemented a xcheck goal which allows running tests which cannot generally be run. A more comprehensive test suite should probably be be build in a different framework.

Filter DSOs are useful to implement APIs/ABIs but the current implementation isn't achieving the goal correctly. The filtered object is simply added in the lookup path which is wrong. Instead a separate group of objects (the filtered object plus the dependencies) must be created which is consulted if lookup in the filter object is happening.

The current malloc implementation handles more usage patterns badly than it is necessary (all malloc implementations handle some patterns badly). The rewrite will address the most important once which are reducing fragmentation and improving general performance.

Convert libm-tests to use exact parameters so that we have no rounding in the arguments. We could even consider using Hexadecimal Constants for both inputs and outputs so that the used values are correct.

Symbols, which are exported to satisfy dependencies between two DSOs of the libc package, should be exported in a special version named GLIBC_PRIVATE.

The pre-linking support to improve application startup time should be integrated in ld.so.

Use the 128-bit long double format now that the math functions are complete. This requires an ABI change.

Use the 128-bit long double format now that the math functions are complete. This requires an ABI change.

Investigate whether any changes are necessary for the use of the headers in C++ code.

Update: The C header side is done. Changes to libstdc++ and the compiler's headers are necessary.

Implement an interface to ld.so which is similar in to the auditing interface Solaris' ld.so has. This will allow writing an ltrace implementation which does not depend on interposition and is much more reliable.

Hurd can support an AIO implementation which doesn't rely on threads.

On Hurd the libc can do a better job in implementing posix_spawn due to more access to the job handling.

For certain files (not too large, and real files) we can use mmap to implement the stream operations. These significantly simplifies the buffer operations since the mmaped buffer can be made available to the user as is. No need to handle underflow. This of course can also significantly improve performance due to the lack of copying.

A call to setlocale opens currently 11 locale files. The needed file operations are expensive. The files for one locale should be consolidated.

One drawback is, though, that this will probably eliminate the size optimization currently implemented which reduces entire locale files to hard links of existing files. The space savings can be large. Therefore this scheme should be optional.

The __libc_subinit feature is used to perform initializations of the libc. There are not many users of this feature left since most things cannot be handled this way. Therefore the last users should be converted to be called directly and the __libc_subinit mechanism should be retired.

Currently not much thought is spent on whether the function which is used in the implementation of another function is exported or not. Currently only namespace cleanliness is considered. The use of exported functions means, though, that the calls are more expensive since runtime linking is used. This can be avoided by using internal names. There are only very few exceptions (namely the malloc functions).

Some people will argue that the consequence will be that overloading of functions won't work anymore. But this isn't a problem since it never worked reliable. The implementation can change at any time and other functions used internally. The only save way to overload/interpose functions is to do this at the entry point of the library.

One interesting way to implement this optimization is to inline system calls. See the next point on the list.

There already exists some limited use of the INLINE_SYSCALL macro to avoid function calls when the called function would just perform a system call. This can be extended. Functions like open can be overloaded using a macro of the same name which expands to a system call inline.

This has the advantage that the function call overhead is gone (on some architectures like x86 this saves even more since register contents is preserved). Additionally there can be a INLINE_SYSCALL_NORERROR variant of the macro which does not cause the error value to be written into errno if this is not needed by the caller.

Update: This feature will not be implemented in glibc 2.3. Maybe never. The problem is the increase in code size resulting from these changes. Unless benchmarks show a benefit these changes will not be made.

It might have advantages to move all global variables used in ld.so into one struct. This way to compiler might be able to generate fewer relocations.

The order in which object files (i.e., their sections) are added to DSOs is currently determined by the order of the files in the libc_pic.a archive which in turn is mostly determined by the order in which source directories are entered during the build and the order of the names in the Makefile. This is not optimal since it doesn't place sections which are closely related (in a callgraph) together. If also places functions which are hardly ever used together with those often used. The result is bad utilization of the instruction cache.

I have started work on a tool to use various forms of input to rearrange the file order but it needs more tweaking.

Update: This feature will not be in 2.3. Initial tests have shown the need for much more research.

Paolo Bonzini has a patch to speed up regex. This needs to be integrated and tested. The patch is available on request (from drepper).

Update: A first patch similar to the one Paolo send is in. This should provide a 10% speedup with a 5% increase in code size.

The code to communicate with the NSCD currently opens a new connection for every lookup. While this normally is not much a problem (since the OSes optimize this) it can become a bottleneck and resource hog if many requests are made, especially in multi-threaded applications.

There are several possibilities to overcome the limitation:

• The connection to the NSCD can be kept up and reused. Multiple outstanding requests should be kept in a wait queue and results dispatched to the waiting thread. The results should be able to come in out-of-order.
• The cache data the NSCD keeps can be made available to the clients. The sharing can happen through POSIX shared memory, file descriptor transfer via Unix socket (and using mmap thereafter), or whatever. The NSCD does not use pointers in the cache data, only offsets, to facilitate using the data in another address space. Algorithms to ensure proper use of the cache data are not easy but also nothing really new. Things like a dying NSCD must also be accounted for. See the PROJECTS file in the libc sources for some more comments.

Sun's NSCD version implements a feature where the NSCD keeps N entries for each database current. I.e., if an entries lifespan is over and it is one of the N entries to be kept the NSCD updates the information instead of removing the entry.

How to decide about which N entries to keep has to be examined. Factors should be number of uses (of course), influenced by aging. Just imagine a computer used by several people. The IDs of the current user should be preferred even if the last user spent more time.

glibc uses a few functions from GMP. The code was taken from GMP a long time ago. GMP changed a lot meanwhile and it might be worthwhile updating the code.