x86: Add SSE2 optimized chacha20

[glibc.git] / manual / maint.texi

@node Maintenance, Platform, Installation, Top
@c %MENU% How to enhance and port the GNU C Library
@appendix Library Maintenance

@menu
* Source Layout::         How to add new functions or header files
                             to the GNU C Library.
* Symbol handling::       How to handle symbols in the GNU C Library.
* Porting::               How to port the GNU C Library to
                             a new machine or operating system.
@end menu

@node Source Layout
@appendixsec Adding New Functions

The process of building the library is driven by the makefiles, which
make heavy use of special features of GNU @code{make}.  The makefiles
are very complex, and you probably don't want to try to understand them.
But what they do is fairly straightforward, and only requires that you
define a few variables in the right places.

The library sources are divided into subdirectories, grouped by topic.

The @file{string} subdirectory has all the string-manipulation
functions, @file{math} has all the mathematical functions, etc.

Each subdirectory contains a simple makefile, called @file{Makefile},
which defines a few @code{make} variables and then includes the global
makefile @file{Rules} with a line like:

@smallexample
include ../Rules
@end smallexample

@noindent
The basic variables that a subdirectory makefile defines are:

@table @code
@item subdir
The name of the subdirectory, for example @file{stdio}.
This variable @strong{must} be defined.

@item headers
The names of the header files in this section of the library,
such as @file{stdio.h}.

@item routines
@itemx aux
The names of the modules (source files) in this section of the library.
These should be simple names, such as @samp{strlen} (rather than
complete file names, such as @file{strlen.c}).  Use @code{routines} for
modules that define functions in the library, and @code{aux} for
auxiliary modules containing things like data definitions.  But the
values of @code{routines} and @code{aux} are just concatenated, so there
really is no practical difference.

@item tests
The names of test programs for this section of the library.  These
should be simple names, such as @samp{tester} (rather than complete file
names, such as @file{tester.c}).  @w{@samp{make tests}} will build and
run all the test programs.  If a test program needs input, put the test
data in a file called @file{@var{test-program}.input}; it will be given to
the test program on its standard input.  If a test program wants to be
run with arguments, put the arguments (all on a single line) in a file
called @file{@var{test-program}.args}.  Test programs should exit with
zero status when the test passes, and nonzero status when the test
indicates a bug in the library or error in building.

@item others
The names of ``other'' programs associated with this section of the
library.  These are programs which are not tests per se, but are other
small programs included with the library.  They are built by
@w{@samp{make others}}.

@item install-lib
@itemx install-data
@itemx install
Files to be installed by @w{@samp{make install}}.  Files listed in
@samp{install-lib} are installed in the directory specified by
@samp{libdir} in @file{configparms} or @file{Makeconfig}
(@pxref{Installation}).  Files listed in @code{install-data} are
installed in the directory specified by @samp{datadir} in
@file{configparms} or @file{Makeconfig}.  Files listed in @code{install}
are installed in the directory specified by @samp{bindir} in
@file{configparms} or @file{Makeconfig}.

@item distribute
Other files from this subdirectory which should be put into a
distribution tar file.  You need not list here the makefile itself or
the source and header files listed in the other standard variables.
Only define @code{distribute} if there are files used in an unusual way
that should go into the distribution.

@item generated
Files which are generated by @file{Makefile} in this subdirectory.
These files will be removed by @w{@samp{make clean}}, and they will
never go into a distribution.

@item extra-objs
Extra object files which are built by @file{Makefile} in this
subdirectory.  This should be a list of file names like @file{foo.o};
the files will actually be found in whatever directory object files are
being built in.  These files will be removed by @w{@samp{make clean}}.
This variable is used for secondary object files needed to build
@code{others} or @code{tests}.
@end table

@menu
* Platform: Adding Platform-specific.             Adding platform-specific
                                         features.
@end menu

@node Adding Platform-specific
@appendixsubsec Platform-specific types, macros and functions

It's sometimes necessary to provide nonstandard, platform-specific
features to developers.  The C library is traditionally the
lowest library layer, so it makes sense for it to provide these
low-level features.  However, including these features in the C
library may be a disadvantage if another package provides them
as well as there will be two conflicting versions of them.  Also,
the features won't be available to projects that do not use
@theglibc{} but use other GNU tools, like GCC.

The current guidelines are:
@itemize @bullet
@item
If the header file provides features that only make sense on a particular
machine architecture and have nothing to do with an operating system, then
the features should ultimately be provided as GCC built-in functions.  Until
then, @theglibc{} may provide them in the header file.  When the GCC built-in
functions become available, those provided in the header file should be made
conditionally available prior to the GCC version in which the built-in
function was made available.

@item
If the header file provides features that are specific to an operating system,
both GCC and @theglibc{} could provide it, but @theglibc{} is preferred
as it already has a lot of information about the operating system.

@item
If the header file provides features that are specific to an operating system
but used by @theglibc{}, then @theglibc{} should provide them.
@end itemize

The general solution for providing low-level features is to export them as
follows:

@itemize @bullet
@item
A nonstandard, low-level header file that defines macros and inline
functions should be called @file{sys/platform/@var{name}.h}.

@item
Each header file's name should include the platform name, to avoid
users thinking there is anything in common between the different
header files for different platforms.  For example, a
@file{sys/platform/@var{arch}.h} name such as
@file{sys/platform/ppc.h} is better than @file{sys/platform.h}.

@item
A platform-specific header file provided by @theglibc{} should coordinate
with GCC such that compiler built-in versions of the functions and macros are
preferred if available.  This means that user programs will only ever need to
include @file{sys/platform/@var{arch}.h}, keeping the same names of types,
macros, and functions for convenience and portability.

@item
Each included symbol must have the prefix @code{__@var{arch}_}, such as
@code{__ppc_get_timebase}.
@end itemize


The easiest way to provide a header file is to add it to the
@code{sysdep_headers} variable.  For example, the combination of
Linux-specific header files on PowerPC could be provided like this:

@smallexample
sysdep_headers += sys/platform/ppc.h
@end smallexample

Then ensure that you have added a @file{sys/platform/ppc.h}
header file in the machine-specific directory, e.g.,
@file{sysdeps/powerpc/sys/platform/ppc.h}.


@node Symbol handling
@appendixsec Symbol handling in the GNU C Library

@menu
* 64-bit time symbol handling :: How to handle 64-bit time related
                                    symbols in the GNU C Library.
@end menu

@node 64-bit time symbol handling
@appendixsubsec 64-bit time symbol handling in the GNU C Library

With respect to time handling, @glibcadj{} configurations fall in two
classes depending on the value of @code{__TIMESIZE}:

@table @code

@item @code{__TIMESIZE == 32}

These @dfn{dual-time} configurations have both 32-bit and 64-bit time
support.  32-bit time support provides type @code{time_t} and cannot
handle dates beyond @dfn{Y2038}.  64-bit time support provides type
@code{__time64_t} and can handle dates beyond @dfn{Y2038}.

In these configurations, time-related types have two declarations,
a 64-bit one, and a 32-bit one; and time-related functions generally
have two definitions: a 64-bit one, and a 32-bit one which is a wrapper
around the former.  Therefore, for every @code{time_t}-related symbol,
there is a corresponding @code{__time64_t}-related symbol, the name of
which is usually the 32-bit symbol's name with @code{__} (a double
underscore) prepended and @code{64} appended.  For instance, the
64-bit-time counterpart of @code{clock_gettime} is
@code{__clock_gettime64}.

@item @code{__TIMESIZE == 64}

These @dfn{single-time} configurations only have a 64-bit @code{time_t}
and related functions, which can handle dates beyond 2038-01-19
03:14:07 (aka @dfn{Y2038}).

In these configurations, time-related types only have a 64-bit
declaration; and time-related functions only have one 64-bit definition.
However, for every @code{time_t}-related symbol, there is a
corresponding @code{__time64_t}-related macro, the name of which is
derived as in the dual-time configuration case, and which expands to
the symbol's name.  For instance, the macro @code{__clock_gettime64}
expands to @code{clock_gettime}.

These macros are purely internal to @theglibc{} and exist only so that
a single definition of the 64-bit time functions can be used on both
single-time and dual-time configurations, and so that glibc code can
freely call the 64-bit functions internally in all configurations.

@end table

@c The following paragraph should be removed once external interfaces
@c get support for both time sizes.

Note: at this point, 64-bit time support in dual-time configurations is
work-in-progress, so for these configurations, the public API only makes
the 32-bit time support available.  In a later change, the public API
will allow user code to choose the time size for a given compilation
unit.

64-bit variants of time-related types or functions are defined for all
configurations and use 64-bit-time symbol names (for dual-time
configurations) or macros (for single-time configurations).

32-bit variants of time-related types or functions are defined only for
dual-time configurations.

Here is an example with @code{localtime}:

Function @code{localtime} is declared in @file{time/time.h} as
@smallexample
extern struct tm *localtime (const time_t *__timer) __THROW;
libc_hidden_proto (localtime)
@end smallexample

For single-time configurations, @code{__localtime64} is a macro which
evaluates to @code{localtime}; for dual-time configurations,
@code{__localtime64} is a function similar to @code{localtime} except
it uses Y2038-proof types:
@smallexample
#if __TIMESIZE == 64
# define __localtime64 localtime
#else
extern struct tm *__localtime64 (const __time64_t *__timer) __THROW;
libc_hidden_proto (__localtime64)
#endif
@end smallexample

(note: type @code{time_t} is replaced with @code{__time64_t} because
@code{time_t} is not Y2038-proof, but @code{struct tm} is not
replaced because it is already Y2038-proof.)

The 64-bit-time implementation of @code{localtime} is written as follows
and is compiled for both dual-time and single-time configuration classes.

@smallexample
struct tm *
__localtime64 (const __time64_t *t)
@{
  return __tz_convert (*t, 1, &_tmbuf);
@}
libc_hidden_def (__localtime64)
@end smallexample

The 32-bit-time implementation is a wrapper and is only compiled for
dual-time configurations:

@smallexample
#if __TIMESIZE != 64

struct tm *
localtime (const time_t *t)
@{
  __time64_t t64 = *t;
  return __localtime64 (&t64);
@}
libc_hidden_def (localtime)

#endif
@end smallexample

@node Porting
@appendixsec Porting @theglibc{}

@Theglibc{} is written to be easily portable to a variety of
machines and operating systems.  Machine- and operating system-dependent
functions are well separated to make it easy to add implementations for
new machines or operating systems.  This section describes the layout of
the library source tree and explains the mechanisms used to select
machine-dependent code to use.

All the machine-dependent and operating system-dependent files in the
library are in the subdirectory @file{sysdeps} under the top-level
library source directory.  This directory contains a hierarchy of
subdirectories (@pxref{Hierarchy Conventions}).

Each subdirectory of @file{sysdeps} contains source files for a
particular machine or operating system, or for a class of machine or
operating system (for example, systems by a particular vendor, or all
machines that use IEEE 754 floating-point format).  A configuration
specifies an ordered list of these subdirectories.  Each subdirectory
implicitly appends its parent directory to the list.  For example,
specifying the list @file{unix/bsd/vax} is equivalent to specifying the
list @file{unix/bsd/vax unix/bsd unix}.  A subdirectory can also specify
that it implies other subdirectories which are not directly above it in
the directory hierarchy.  If the file @file{Implies} exists in a
subdirectory, it lists other subdirectories of @file{sysdeps} which are
appended to the list, appearing after the subdirectory containing the
@file{Implies} file.  Lines in an @file{Implies} file that begin with a
@samp{#} character are ignored as comments.  For example,
@file{unix/bsd/Implies} contains:
@smallexample
# BSD has Internet-related things.
unix/inet
@end smallexample
@noindent
and @file{unix/Implies} contains:
@need 300
@smallexample
posix
@end smallexample

@noindent
So the final list is @file{unix/bsd/vax unix/bsd unix/inet unix posix}.

@file{sysdeps} has a ``special'' subdirectory called @file{generic}.  It
is always implicitly appended to the list of subdirectories, so you
needn't put it in an @file{Implies} file, and you should not create any
subdirectories under it intended to be new specific categories.
@file{generic} serves two purposes.  First, the makefiles do not bother
to look for a system-dependent version of a file that's not in
@file{generic}.  This means that any system-dependent source file must
have an analogue in @file{generic}, even if the routines defined by that
file are not implemented on other platforms.  Second, the @file{generic}
version of a system-dependent file is used if the makefiles do not find
a version specific to the system you're compiling for.

If it is possible to implement the routines in a @file{generic} file in
machine-independent C, using only other machine-independent functions in
the C library, then you should do so.  Otherwise, make them stubs.  A
@dfn{stub} function is a function which cannot be implemented on a
particular machine or operating system.  Stub functions always return an
error, and set @code{errno} to @code{ENOSYS} (Function not implemented).
@xref{Error Reporting}.  If you define a stub function, you must place
the statement @code{stub_warning(@var{function})}, where @var{function}
is the name of your function, after its definition.  This causes the
function to be listed in the installed @code{<gnu/stubs.h>}, and
makes GNU ld warn when the function is used.

Some rare functions are only useful on specific systems and aren't
defined at all on others; these do not appear anywhere in the
system-independent source code or makefiles (including the
@file{generic} directory), only in the system-dependent @file{Makefile}
in the specific system's subdirectory.

If you come across a file that is in one of the main source directories
(@file{string}, @file{stdio}, etc.), and you want to write a machine- or
operating system-dependent version of it, move the file into
@file{sysdeps/generic} and write your new implementation in the
appropriate system-specific subdirectory.  Note that if a file is to be
system-dependent, it @strong{must not} appear in one of the main source
directories.

There are a few special files that may exist in each subdirectory of
@file{sysdeps}:

@comment Blank lines after items make the table look better.
@table @file
@item Makefile

A makefile for this machine or operating system, or class of machine or
operating system.  This file is included by the library makefile
@file{Makerules}, which is used by the top-level makefile and the
subdirectory makefiles.  It can change the variables set in the
including makefile or add new rules.  It can use GNU @code{make}
conditional directives based on the variable @samp{subdir} (see above) to
select different sets of variables and rules for different sections of
the library.  It can also set the @code{make} variable
@samp{sysdep-routines}, to specify extra modules to be included in the
library.  You should use @samp{sysdep-routines} rather than adding
modules to @samp{routines} because the latter is used in determining
what to distribute for each subdirectory of the main source tree.

Each makefile in a subdirectory in the ordered list of subdirectories to
be searched is included in order.  Since several system-dependent
makefiles may be included, each should append to @samp{sysdep-routines}
rather than simply setting it:

@smallexample
sysdep-routines := $(sysdep-routines) foo bar
@end smallexample

@need 1000
@item Subdirs

This file contains the names of new whole subdirectories under the
top-level library source tree that should be included for this system.
These subdirectories are treated just like the system-independent
subdirectories in the library source tree, such as @file{stdio} and
@file{math}.

Use this when there are completely new sets of functions and header
files that should go into the library for the system this subdirectory
of @file{sysdeps} implements.  For example,
@file{sysdeps/unix/inet/Subdirs} contains @file{inet}; the @file{inet}
directory contains various network-oriented operations which only make
sense to put in the library on systems that support the Internet.

@item configure

This file is a shell script fragment to be run at configuration time.
The top-level @file{configure} script uses the shell @code{.} command to
read the @file{configure} file in each system-dependent directory
chosen, in order.  The @file{configure} files are often generated from
@file{configure.ac} files using Autoconf.

A system-dependent @file{configure} script will usually add things to
the shell variables @samp{DEFS} and @samp{config_vars}; see the
top-level @file{configure} script for details.  The script can check for
@w{@samp{--with-@var{package}}} options that were passed to the
top-level @file{configure}.  For an option
@w{@samp{--with-@var{package}=@var{value}}} @file{configure} sets the
shell variable @w{@samp{with_@var{package}}} (with any dashes in
@var{package} converted to underscores) to @var{value}; if the option is
just @w{@samp{--with-@var{package}}} (no argument), then it sets
@w{@samp{with_@var{package}}} to @samp{yes}.

@item configure.ac

This file is an Autoconf input fragment to be processed into the file
@file{configure} in this subdirectory.  @xref{Introduction,,,
autoconf.info, Autoconf: Generating Automatic Configuration Scripts},
for a description of Autoconf.  You should write either @file{configure}
or @file{configure.ac}, but not both.  The first line of
@file{configure.ac} should invoke the @code{m4} macro
@samp{GLIBC_PROVIDES}.  This macro does several @code{AC_PROVIDE} calls
for Autoconf macros which are used by the top-level @file{configure}
script; without this, those macros might be invoked again unnecessarily
by Autoconf.
@end table

That is the general system for how system-dependencies are isolated.
@iftex
The next section explains how to decide what directories in
@file{sysdeps} to use.  @ref{Porting to Unix}, has some tips on porting
the library to Unix variants.
@end iftex

@menu
* Hierarchy Conventions::       The layout of the @file{sysdeps} hierarchy.
* Porting to Unix::             Porting the library to an average
                                   Unix-like system.
@end menu

@node Hierarchy Conventions
@appendixsubsec Layout of the @file{sysdeps} Directory Hierarchy

A GNU configuration name has three parts: the CPU type, the
manufacturer's name, and the operating system.  @file{configure} uses
these to pick the list of system-dependent directories to look for.  If
the @samp{--nfp} option is @emph{not} passed to @file{configure}, the
directory @file{@var{machine}/fpu} is also used.  The operating system
often has a @dfn{base operating system}; for example, if the operating
system is @samp{Linux}, the base operating system is @samp{unix/sysv}.
The algorithm used to pick the list of directories is simple:
@file{configure} makes a list of the base operating system,
manufacturer, CPU type, and operating system, in that order.  It then
concatenates all these together with slashes in between, to produce a
directory name; for example, the configuration @w{@samp{i686-linux-gnu}}
results in @file{unix/sysv/linux/i386/i686}.  @file{configure} then
tries removing each element of the list in turn, so
@file{unix/sysv/linux} and @file{unix/sysv} are also tried, among others.
Since the precise version number of the operating system is often not
important, and it would be very inconvenient, for example, to have
identical @file{irix6.2} and @file{irix6.3} directories,
@file{configure} tries successively less specific operating system names
by removing trailing suffixes starting with a period.

As an example, here is the complete list of directories that would be
tried for the configuration @w{@samp{i686-linux-gnu}}:

@smallexample
sysdeps/i386/elf
sysdeps/unix/sysv/linux/i386
sysdeps/unix/sysv/linux
sysdeps/gnu
sysdeps/unix/common
sysdeps/unix/mman
sysdeps/unix/inet
sysdeps/unix/sysv/i386/i686
sysdeps/unix/sysv/i386
sysdeps/unix/sysv
sysdeps/unix/i386
sysdeps/unix
sysdeps/posix
sysdeps/i386/i686
sysdeps/i386/i486
sysdeps/libm-i387/i686
sysdeps/i386/fpu
sysdeps/libm-i387
sysdeps/i386
sysdeps/wordsize-32
sysdeps/ieee754
sysdeps/libm-ieee754
sysdeps/generic
@end smallexample

Different machine architectures are conventionally subdirectories at the
top level of the @file{sysdeps} directory tree.  For example,
@w{@file{sysdeps/sparc}} and @w{@file{sysdeps/m68k}}.  These contain
files specific to those machine architectures, but not specific to any
particular operating system.  There might be subdirectories for
specializations of those architectures, such as
@w{@file{sysdeps/m68k/68020}}.  Code which is specific to the
floating-point coprocessor used with a particular machine should go in
@w{@file{sysdeps/@var{machine}/fpu}}.

There are a few directories at the top level of the @file{sysdeps}
hierarchy that are not for particular machine architectures.

@table @file
@item generic
As described above (@pxref{Porting}), this is the subdirectory
that every configuration implicitly uses after all others.

@item ieee754
This directory is for code using the IEEE 754 floating-point format,
where the C type @code{float} is IEEE 754 single-precision format, and
@code{double} is IEEE 754 double-precision format.  Usually this
directory is referred to in the @file{Implies} file in a machine
architecture-specific directory, such as @file{m68k/Implies}.

@item libm-ieee754
This directory contains an implementation of a mathematical library
usable on platforms which use @w{IEEE 754} conformant floating-point
arithmetic.

@item libm-i387
This is a special case.  Ideally the code should be in
@file{sysdeps/i386/fpu} but for various reasons it is kept aside.

@item posix
This directory contains implementations of things in the library in
terms of @sc{POSIX.1} functions.  This includes some of the @sc{POSIX.1}
functions themselves.  Of course, @sc{POSIX.1} cannot be completely
implemented in terms of itself, so a configuration using just
@file{posix} cannot be complete.

@item unix
This is the directory for Unix-like things.  @xref{Porting to Unix}.
@file{unix} implies @file{posix}.  There are some special-purpose
subdirectories of @file{unix}:

@table @file
@item unix/common
This directory is for things common to both BSD and System V release 4.
Both @file{unix/bsd} and @file{unix/sysv/sysv4} imply @file{unix/common}.

@item unix/inet
This directory is for @code{socket} and related functions on Unix systems.
@file{unix/inet/Subdirs} enables the @file{inet} top-level subdirectory.
@file{unix/common} implies @file{unix/inet}.
@end table

@item mach
This is the directory for things based on the Mach microkernel from CMU
(including @gnuhurdsystems{}).  Other basic operating systems
(VMS, for example) would have their own directories at the top level of
the @file{sysdeps} hierarchy, parallel to @file{unix} and @file{mach}.
@end table

@node Porting to Unix
@appendixsubsec Porting @theglibc{} to Unix Systems

Most Unix systems are fundamentally very similar.  There are variations
between different machines, and variations in what facilities are
provided by the kernel.  But the interface to the operating system
facilities is, for the most part, pretty uniform and simple.

The code for Unix systems is in the directory @file{unix}, at the top
level of the @file{sysdeps} hierarchy.  This directory contains
subdirectories (and subdirectory trees) for various Unix variants.

The functions which are system calls in most Unix systems are
implemented in assembly code, which is generated automatically from
specifications in files named @file{syscalls.list}.  There are several
such files, one in @file{sysdeps/unix} and others in its subdirectories.
Some special system calls are implemented in files that are named with a
suffix of @samp{.S}; for example, @file{_exit.S}.  Files ending in
@samp{.S} are run through the C preprocessor before being fed to the
assembler.

These files all use a set of macros that should be defined in
@file{sysdep.h}.  The @file{sysdep.h} file in @file{sysdeps/unix}
partially defines them; a @file{sysdep.h} file in another directory must
finish defining them for the particular machine and operating system
variant.  See @file{sysdeps/unix/sysdep.h} and the machine-specific
@file{sysdep.h} implementations to see what these macros are and what
they should do.

The system-specific makefile for the @file{unix} directory
(@file{sysdeps/unix/Makefile}) gives rules to generate several files
from the Unix system you are building the library on (which is assumed
to be the target system you are building the library @emph{for}).  All
the generated files are put in the directory where the object files are
kept; they should not affect the source tree itself.  The files
generated are @file{ioctls.h}, @file{errnos.h}, @file{sys/param.h}, and
@file{errlist.c} (for the @file{stdio} section of the library).

@ignore
@c This section might be a good idea if it is finished,
@c but there's no point including it as it stands. --rms
@c @appendixsec Compatibility with Traditional C

@c ??? This section is really short now.  Want to keep it? --roland

@c It's not anymore true.  glibc 2.1 cannot be used with K&R compilers.
@c --drepper

Although @theglibc{} implements the @w{ISO C} library facilities, you
@emph{can} use @theglibc{} with traditional, ``pre-ISO'' C
compilers.  However, you need to be careful because the content and
organization of the @glibcadj{} header files differs from that of
traditional C implementations.  This means you may need to make changes
to your program in order to get it to compile.
@end ignore