docs/overview.txt
author "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Sat Oct 10 11:24:31 2009 +0200 (2009-10-10)
changeset 1574 07684767f873
parent 1554 3b8df55987ae
child 1575 eb92c99b17cd
permissions -rw-r--r--
docs: configure no longer accepts --with-contrib

It's been a while that configure does not recognise --with-contrib.
Applying contibutions is to be done manually, using patch.
     1 File.........: overview.txt
     2 Content......: Overview of how crosstool-NG works.
     3 Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>
     4 License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
     5 
     6 ____________________
     7                    /
     8 Table Of Content  /
     9 _________________/
    10 
    11 
    12 Introduction
    13 History
    14 Installing crosstool-NG
    15   Install method
    16   The hacker's way
    17   Preparing for packaging
    18   Shell completion
    19   Contributed code
    20 Configuring crosstool-NG
    21   Interesting config options
    22   Re-building an existing toolchain
    23 Running crosstool-NG
    24   Stopping and restarting a build
    25   Testing all toolchains at once
    26   Overriding the number of // jobs
    27   Note on // jobs
    28   Tools wrapper
    29 Using the toolchain
    30   The 'populate' script
    31 Toolchain types
    32   Seemingly-native toolchains
    33 Internals
    34   Makefile front-end
    35   Kconfig parser
    36   Architecture-specific
    37   Adding a new version of a component
    38   Build scripts
    39 
    40 ________________
    41                /
    42 Introduction  /
    43 _____________/
    44 
    45 crosstool-NG aims at building toolchains. Toolchains are an essential component
    46 in a software development project. It will compile, assemble and link the code
    47 that is being developed. Some pieces of the toolchain will eventually end up
    48 in the resulting binary/ies: static libraries are but an example.
    49 
    50 So, a toolchain is a very sensitive piece of software, as any bug in one of the
    51 components, or a poorly configured component, can lead to execution problems,
    52 ranging from poor performance, to applications ending unexpectedly, to
    53 mis-behaving software (which more than often is hard to detect), to hardware
    54 damage, or even to human risks (which is more than regrettable).
    55 
    56 Toolchains are made of different piece of software, each being quite complex
    57 and requiring specially crafted options to build and work seamlessly. This
    58 is usually not that easy, even in the not-so-trivial case of native toolchains.
    59 The work reaches a higher degree of complexity when it comes to cross-
    60 compilation, where it can become quite a nightmare...
    61 
    62 Some cross-toolchains exist on the internet, and can be used for general
    63 development, but they have a number of limitations:
    64   - they can be general purpose, in that they are configured for the majority:
    65     no optimisation for your specific target,
    66   - they can be prepared for a specific target and thus are not easy to use,
    67     nor optimised for, or even supporting your target,
    68   - they often are using aging components (compiler, C library, etc...) not
    69     supporting special features of your shiny new processor;
    70 On the other side, these toolchain offer some advantages:
    71   - they are ready to use and quite easy to install and setup,
    72   - they are proven if used by a wide community.
    73 
    74 But once you want to get all the juice out of your specific hardware, you will
    75 want to build your own toolchain. This is where crosstool-NG comes into play.
    76 
    77 There are also a number of tools that build toolchains for specific needs,
    78 which are not really scalable. Examples are:
    79   - buildroot (buildroot.uclibc.org) whose main purpose is to build root file
    80     systems, hence the name. But once you have your toolchain with buildroot,
    81     part of it is installed in the root-to-be, so if you want to build a whole
    82     new root, you either have to save the existing one as a template and
    83     restore it later, or restart again from scratch. This is not convenient,
    84   - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very
    85     similar to buildroot,
    86   - other projects (openembedded.org for example), which are again used to
    87     build root file systems.
    88 
    89 crosstool-NG is really targeted at building toolchains, and only toolchains.
    90 It is then up to you to use it the way you want.
    91 
    92 ___________
    93           /
    94 History  /
    95 ________/
    96 
    97 crosstool was first 'conceived' by Dan Kegel, who offered it to the community
    98 as a set of scripts, a repository of patches, and some pre-configured, general
    99 purpose setup files to be used to configure crosstool. This is available at
   100 http://www.kegel.com/crosstool, and the subversion repository is hosted on
   101 google at http://code.google.com/p/crosstool/.
   102 
   103 I once managed to add support for uClibc-based toolchains, but it did not make
   104 into mainline, mostly because I didn't have time to port the patch forward to
   105 the new versions, due in part to the big effort it was taking.
   106 
   107 So I decided to clean up crosstool in the state it was, re-order the things
   108 in place, add appropriate support for what I needed, that is uClibc support
   109 and a menu-driven configuration, named the new implementation crosstool-NG,
   110 (standing for crosstool Next Generation, as many other comunity projects do,
   111 and as a wink at the TV series "Star Trek: The Next Generation" ;-) ) and
   112 made it available to the community, in case it was of interest to any one.
   113 
   114 ___________________________
   115                           /
   116 Installing crosstool-NG  /
   117 ________________________/
   118 
   119 There are two ways you can use crosstool-NG:
   120  - build and install it, then get rid of the sources like you'd do for most
   121    programs,
   122  - or only build it and run from the source directory.
   123 
   124 The former should be used if you got crosstool-NG from a packaged tarball, see
   125 "Install method", below, while the latter is most useful for developpers that
   126 checked the code out from SVN, and want to submit patches, see "The Hacker's
   127 way", below.
   128 
   129 Install method |
   130 ---------------+
   131 
   132 If you go for the install, then you just follow the classical, but yet easy
   133 ./configure way:
   134   ./configure --prefix=/some/place
   135   make
   136   make install
   137   export PATH="${PATH}:/some/place/bin"
   138 
   139 You can then get rid of crosstool-NG source. Next create a directory to serve
   140 as a working place, cd in there and run:
   141   ct-ng help
   142 
   143 See below for complete usage.
   144 
   145 The Hacker's way |
   146 -----------------+
   147 
   148 If you go the hacker's way, then the usage is a bit different, although very
   149 simple:
   150   ./configure --local
   151   make
   152 
   153 Now, *do not* remove crosstool-NG sources. They are needed to run crosstool-NG!
   154 Stay in the directory holding the sources, and run:
   155   ./ct-ng help
   156 
   157 See below for complete usage.
   158 
   159 Now, provided you used a clone of the repository, you can send me your changes.
   160 See the file README, at the top of crosstool-NG source, for how to submit
   161 changees.
   162 
   163 Preparing for packaging |
   164 ------------------------+
   165 
   166 If you plan on packaging crosstool-NG, you surely don't want to install it
   167 in your root file system. The install procedure of crosstool-NG honors the
   168 DESTDIR variable:
   169 
   170   ./configure --prefix=/usr
   171   make
   172   make DESTDIR=/packaging/place install
   173 
   174 Shell completion |
   175 -----------------+
   176 
   177 crosstool-NG comes with a shell script fragment that defines bash-compatible
   178 completion. That shell fragment is currently not installed automatically, but
   179 this is planned.
   180 
   181 To install the shell script fragment, you have two options:
   182  - install system-wide, most probably by copying ct-ng.comp into
   183    /etc/bash_completion.d/
   184  - install for a single user, by copying ct-ng.comp into ${HOME}/ and
   185    sourcing this file from your ${HOME}/.bashrc
   186 
   187 Contributed code |
   188 -----------------+
   189 
   190 Some people contibuted code that couldn't get merged for various reasons. This
   191 code is available as lzma-compressed patches, in the contrib/ sub-directory.
   192 These patches are to be applied to the source of crosstool-NG, prior to
   193 installing, using something like the following:
   194   lzcat contrib/foobar.patch.lzma |patch -p1
   195 
   196 There is no guarantee that a particuliar contribution applies to the current
   197 version of crosstool-ng, or that it will work at all. Use contributions at
   198 your own risk.
   199 
   200 ____________________________
   201                            /
   202 Configuring crosstool-NG  /
   203 _________________________/
   204 
   205 crosstool-NG is configured with a configurator presenting a menu-stuctured set
   206 of options. These options let you specify the way you want your toolchain
   207 built, where you want it installed, what architecture and specific processor it
   208 will support, the version of the components you want to use, etc... The
   209 value for those options are then stored in a configuration file.
   210 
   211 The configurator works the same way you configure your Linux kernel. It is
   212 assumed you now how to handle this.
   213 
   214 To enter the menu, type:
   215   ct-ng menuconfig
   216 
   217 Almost every config item has a help entry. Read them carefully.
   218 
   219 String and number options can refer to environment variables. In such a case,
   220 you must use the shell syntax: ${VAR}. You shall neither single- nor double-
   221 quote the string/number options.
   222 
   223 There are three environment variables that are computed by crosstool-NG, and
   224 that you can use:
   225 
   226 CT_TARGET:
   227   It represents the target tuple you are building for. You can use it for
   228   example in the installation/prefix directory, such as:
   229     /opt/x-tools/${CT_TARGET}
   230 
   231 CT_TOP_DIR:
   232   The top directory where crosstool-NG is running. You shouldn't need it in
   233   most cases. There is one case where you may need it: if you have local
   234   patches and you store them in your running directory, you can refer to them
   235   by using CT_TOP_DIR, such as:
   236     ${CT_TOP_DIR}/patches.myproject
   237 
   238 CT_VERSION:
   239   The version of crosstool-NG you are using. Not much use for you, but it's
   240   there if you need it.
   241 
   242 Interesting config options |
   243 ---------------------------+
   244 
   245 CT_LOCAL_TARBALLS_DIR:
   246   If you already have some tarballs in a direcotry, enter it here. That will
   247   speed up the retrieving phase, where crosstool-NG would otherwise download
   248   those tarballs.
   249 
   250 CT_PREFIX_DIR:
   251   This is where the toolchain will be installed in (and for now, where it
   252   will run from). Common use is to add the target tuple in the directory
   253   path, such as (see above):
   254     /opt/x-tools/${CT_TARGET}
   255 
   256 CT_TARGET_VENDOR:
   257   An identifier for your toolchain, will take place in the vendor part of the
   258   target tuple. It shall *not* contain spaces or dashes. Usually, keep it
   259   to a one-word string, or use underscores to separate words if you need.
   260   Avoid dots, commas, and special characters.
   261 
   262 CT_TARGET_ALIAS:
   263   An alias for the toolchian. It will be used as a prefix to the toolchain
   264   tools. For example, you will have ${CT_TARGET_ALIAS}-gcc
   265 
   266 Also, if you think you don't see enough versions, you can try to enable one of
   267 those:
   268 
   269 CT_OBSOLETE:
   270   Show obsolete versions or tools. Most of the time, you don't want to base
   271   your toolchain on too old a version (of gcc, for example). But at times, it
   272   can come handy to use such an old version for regression tests. Those old
   273   versions are hidden behind CT_OBSOLETE. Those versions (or features) are so
   274   marked because maintaining support for those in crosstool-NG would be too
   275   costly, time-wise, and time is dear.
   276 
   277 CT_EXPERIMENTAL:
   278   Show experimental versions or tools. Again, you might not want to base your
   279   toolchain on too recent tools (eg. gcc) for production. But if you need a
   280   feature present only in a recent version, or a new tool, you can find them
   281   hidden behind CT_EXPERIMENTAL. Those versions (or features) did not (yet)
   282   receive thorough testing in crosstool-NG, and/or are not mature enough to
   283   be blindly trusted.
   284 
   285 Re-building an existing toolchain |
   286 ----------------------------------+
   287 
   288 If you have an existing toolchain, you can re-use the options used to build it
   289 to create a new toolchain. That needs a very little bit of effort on your side
   290 but is quite easy. The options to build a toolchain are saved with the
   291 toolchain, and you can retrieve this configuration by running:
   292   ${CT_TARGET}-config
   293 
   294 This will dump the configuration to stdout, so to rebuild a toolchain with this
   295 configuration, the following is all you need to do:
   296   ${CT_TARGET}-config >.config
   297   ct-ng oldconfig
   298 
   299 Then, you can review and change the configuration by running:
   300   ct-ng menuconfig
   301 
   302 ________________________
   303                        /
   304 Running crosstool-NG  /
   305 _____________________/
   306 
   307 To build the toolchain, simply type:
   308   ct-ng build
   309 
   310 This will use the above configuration to retrieve, extract and patch the
   311 components, build, install and eventually test your newly built toolchain.
   312 
   313 You are then free to add the toolchain /bin directory in your PATH to use
   314 it at will.
   315 
   316 In any case, you can get some terse help. Just type:
   317   ct-ng help
   318 or:
   319   man 1 ct-ng
   320 
   321 Stopping and restarting a build |
   322 --------------------------------+
   323 
   324 If you want to stop the build after a step you are debugging, you can pass the
   325 variable STOP to make:
   326   ct-ng STOP=some_step
   327 
   328 Conversely, if you want to restart a build at a specific step you are
   329 debugging, you can pass the RESTART variable to make:
   330   ct-ng RESTART=some_step
   331 
   332 Alternatively, you can call make with the name of a step to just do that step:
   333   ct-ng libc_headers
   334 is equivalent to:
   335   ct-ng RESTART=libc_headers STOP=libc_headers
   336 
   337 The shortcuts +step_name and step_name+ allow to respectively stop or restart
   338 at that step. Thus:
   339   ct-ng +libc_headers        and:    ct-ng libc_headers+
   340 are equivalent to:
   341   ct-ng STOP=libc_headers    and:    ct-ng RESTART=libc_headers
   342 
   343 To obtain the list of acceptable steps, please call:
   344   ct-ng list-steps
   345 
   346 Note that in order to restart a build, you'll have to say 'Y' to the config
   347 option CT_DEBUG_CT_SAVE_STEPS, and that the previous build effectively went
   348 that far.
   349 
   350 Building all toolchains at once |
   351 --------------------------------+
   352 
   353 You can build all samples; simply call:
   354   ct-ng build-all
   355 
   356 Overriding the number of // jobs |
   357 ---------------------------------+
   358 
   359 If you want to override the number of jobs to run in // (the -j option to
   360 make), you can either re-enter the menuconfig, or simply add it on the command
   361 line, as such:
   362   ct-ng build.4
   363 
   364 which tells crosstool-NG to override the number of // jobs to 4.
   365 
   366 You can see the actions that support overriding the number of // jobs in
   367 the help menu. Those are the ones with [.#] after them (eg. build[.#] or
   368 build-all[.#], and so on...).
   369 
   370 Note on // jobs |
   371 ----------------+
   372 
   373 The crosstool-NG script 'ct-ng' is a Makefile-script. It does *not* execute
   374 in parallel (there is not much to gain). When speaking of // jobs, we are
   375 refering to the number of // jobs when making the *components*. That is, we
   376 speak of the number of // jobs used to build gcc, glibc, and so on...
   377 
   378 Tools wrapper |
   379 --------------+
   380 
   381 Starting with gcc-4.3 come two new dependencies: GMP and MPFR. With gcc-4.4,
   382 come three new ones: GMP, PPL and CLooG/ppl. These are libraries that enable
   383 advanced features to gcc. Additionally, some of the libraries can be used by
   384 binutils and gdb. Unfortunately, not all systems on which crosstool-NG runs
   385 have all of those libraries. And for those that do, the versions of those
   386 libraries may be older than the version required by gcc.
   387 
   388 This is why crosstool-NG builds its own set of libraries as part of the
   389 toolchain.
   390 
   391 The libraries are built as shared libraries, because building them as static
   392 libraries has some short-comings. This poses no problem at build time, as
   393 crosstool-NG correctly points gcc (and binutils and gdb) to the correct
   394 place where our own version of the libraries are installed. But it poses
   395 a problem when gcc et al. are run: the place where the libraries are is most
   396 probably not known to the host dynamic linker. Still worse, if the host system
   397 has its own versions, then ld.so would load the wrong library!
   398 
   399 So we have to force the dynamic linker to load the correct version. We do this
   400 by using the LD_LIBRARY_PATH variable, that informs the dynamic linker where
   401 to look for shared libraries prior to searching its standard places. But we
   402 can't impose that burden on all the system (because it'd be a nightmare to
   403 configure, and because two toolchains on the same system may use different
   404 versions of the libraries); so we have to do it on a per-toolchain basis.
   405 
   406 So we rename all binaries of the toolchain (by adding a dot '.' as their first
   407 character), and add a small program, the so-called "tools wrapper", that
   408 correctly sets LD_LIBRARY_PATH prior to running the real tool.
   409 
   410 First, the wrapper was written as a POSIX-compliant shell script. That shell
   411 script is very simple, if not trivial, and works great. The only drawback is
   412 that it does not work on host systems that lack a shell, for example the
   413 MingW32 environment. To solve the issue, the wrapper has been re-written in C,
   414 and compiled at build time. This C wrapper is much more complex than the shell
   415 script, and although it sems to be working, it's been only lightly tested.
   416 Some of the expected short-comings with this C wrapper are;
   417  - multi-byte file names may not be handled correctly
   418  - it's really big for what it does
   419 
   420 So, the default wrapper installed with your toolchain is the shell script.
   421 If you know that your system is missing a shell, then you shall use the C
   422 wrapper (and report back whether it works, or does not work, for you).
   423 
   424 
   425 _______________________
   426                       /
   427 Using the toolchain  /
   428 ____________________/
   429 
   430 Using the toolchain is as simple as adding the toolchain's bin directory in
   431 your PATH, such as:
   432   export PATH="${PATH}:/your/toolchain/path/bin"
   433 
   434 and then using the target tuple to tell the build systems to use your
   435 toolchain:
   436   ./configure --target=your-target-tuple
   437 or
   438   make CC=your-target-tuple-gcc
   439 or
   440   make CROSS_COMPILE=your-target-tuple-
   441 and so on...
   442 
   443 It is strongly advised not to use the toolchain sys-root directory as an
   444 install directory for your programs/packages. If you do so, you will not be
   445 able to use your toolchain for another project. It is even strongly advised
   446 that your toolchain is chmod-ed to read-only once successfully build, so that
   447 you don't go polluting your toolchain with your programs/packages' files.
   448 
   449 Thus, when you build a program/package, install it in a separate directory,
   450 eg. /your/root. This directory is the /image/ of what would be in the root file
   451 system of your target, and will contain all that your programs/packages have
   452 installed.
   453 
   454 The 'populate' script |
   455 ----------------------+
   456 
   457 When your root directory is ready, it is still missing some important bits: the
   458 toolchain's libraries. To populate your root directory with those libs, just
   459 run:
   460   your-target-tuple-populate -s /your/root -d /your/root-populated
   461 
   462 This will copy /your/root into /your/root-populated, and put the needed and only
   463 the needed libraries there. Thus you don't polute /your/root with any cruft that
   464 would no longer be needed should you have to remove stuff. /your/root always
   465 contains only those things you install in it.
   466 
   467 You can then use /your/root-populated to build up your file system image, a
   468 tarball, or to NFS-mount it from your target, or whatever you need.
   469 
   470 The populate script accepts the following options:
   471 
   472  -s src_dir
   473     Use 'src_dir' as the un-populated root directory.
   474 
   475  -d dst_dir
   476     Put the populated root directory in 'dst_dir'.
   477 
   478  -l lib1 [...]
   479     Always add specified libraries.
   480 
   481  -L file
   482     Always add libraries listed in 'file'.
   483 
   484  -f
   485     Remove 'dst_dir' if it previously existed; continue even if any library
   486     specified with -l or -L is missing.
   487 
   488  -v
   489     Be verbose, and tell what's going on (you can see exactly where libs are
   490     coming from).
   491 
   492  -h
   493     Print the help.
   494 
   495 See 'your-target-tuple-populate -h' for more information on the options.
   496 
   497 Here is how populate works:
   498 
   499   1) performs some sanity checks:
   500      - src_dir and dst_dir are specified
   501      - src_dir exists
   502      - unless forced, dst_dir does not exist
   503      - src_dir != dst_dir
   504 
   505   2) copy src_dir to dst_dir
   506 
   507   3) add forced libraries to dst_dir
   508      - build the list from -l and -L options
   509      - get forced libraries from the sysroot (see below for heuristics)
   510        - abort on the first missing library, unless -f is specified
   511 
   512   4) add all missing libraries to dst_dir
   513      - scan dst_dir for every ELF files that are 'executable' or
   514        'shared object'
   515      - list the "NEEDED Shared library" fields
   516        - check if the library is already in dst_dir/lib or dst_dir/usr/lib
   517        - if not, get the library from the sysroot
   518          - if it's in sysroot/lib, copy it to dst_dir/lib
   519          - if it's in sysroot/usr/lib, copy it to dst_dir/usr/lib
   520          - in both cases, use the SONAME of the library to create the file
   521            in dst_dir
   522          - if it was not found in the sysroot, this is an error.
   523 
   524 ___________________
   525                   /
   526 Toolchain types  /
   527 ________________/
   528 
   529 There are four kinds of toolchains you could encounter.
   530 
   531 First off, you must understand the following: when it comes to compilers there
   532 are up to four machines involved:
   533   1) the machine configuring the toolchain components: the config machine
   534   2) the machine building the toolchain components:    the build machine
   535   3) the machine running the toolchain:                the host machine
   536   4) the machine the toolchain is generating code for: the target machine
   537 
   538 We can most of the time assume that the config machine and the build machine
   539 are the same. Most of the time, this will be true. The only time it isn't
   540 is if you're using distributed compilation (such as distcc). Let's forget
   541 this for the sake of simplicity.
   542 
   543 So we're left with three machines:
   544  - build
   545  - host
   546  - target
   547 
   548 Any toolchain will involve those three machines. You can be as pretty sure of
   549 this as "2 and 2 are 4". Here is how they come into play:
   550 
   551 1) build == host == target
   552     This is a plain native toolchain, targetting the exact same machine as the
   553     one it is built on, and running again on this exact same machine. You have
   554     to build such a toolchain when you want to use an updated component, such
   555     as a newer gcc for example.
   556     crosstool-NG calls it "native".
   557 
   558 2) build == host != target
   559     This is a classic cross-toolchain, which is expected to be run on the same
   560     machine it is compiled on, and generate code to run on a second machine,
   561     the target.
   562     crosstool-NG calls it "cross".
   563 
   564 3) build != host == target
   565     Such a toolchain is also a native toolchain, as it targets the same machine
   566     as it runs on. But it is build on another machine. You want such a
   567     toolchain when porting to a new architecture, or if the build machine is
   568     much faster than the host machine.
   569     crosstool-NG calls it "cross-native".
   570 
   571 4) build != host != target
   572     This one is called a canadian-toolchain (*), and is tricky. The three
   573     machines in play are different. You might want such a toolchain if you
   574     have a fast build machine, but the users will use it on another machine,
   575     and will produce code to run on a third machine.
   576     crosstool-NG calls it "canadian".
   577 
   578 crosstool-NG can build all these kinds of toolchains (or is aiming at it,
   579 anyway!)
   580 
   581 (*) The term Canadian Cross came about because at the time that these issues
   582     were all being hashed out, Canada had three national political parties.
   583     http://en.wikipedia.org/wiki/Cross_compiler
   584 
   585 Seemingly-native toolchains |
   586 ----------------------------+
   587 
   588 Seemingly-native toolchains are toolchains that target the same architecture
   589 as the one it is built on, and on which it will run, but the machine tuple
   590 may be different (eg i686 vs. i386, or x86_64-unknown-linux-gnu vs.
   591 x86_64-pc-linux-gnu). This also applies if the target architecture is of the
   592 same kind (eg. x86 vs. x86_64, or ppc vs. ppc64).
   593 
   594 Such toolchain is tricky to build, as the configure scripts may incorrectly
   595 assume that files (headers and libs) from the build (or host) machine can be
   596 used by the cross-compiler it is going to build. The problem seems to arise
   597 only with glibc (and eglibc?) starting with version 2.7.
   598 
   599 
   600 _____________
   601             /
   602 Internals  /
   603 __________/
   604 
   605 Internally, crosstool-NG is script-based. To ease usage, the frontend is
   606 Makefile-based.
   607 
   608 Makefile front-end |
   609 -------------------+
   610 
   611 The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
   612 script with an action will act exactly as if the Makefile was in the current
   613 working directory and make was called with the action as rule. Thus:
   614   ct-ng menuconfig
   615 
   616 is equivalent to having the Makefile in CWD, and calling:
   617   make menuconfig
   618 
   619 Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
   620 traditional command.
   621 
   622 ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
   623 at configuration time with ./configure.
   624 
   625 ct-ng also searches for config files, sub-tools, samples, scripts and patches in
   626 that library directory.
   627 
   628 Because of a stupid make behavior/bug I was unable to track down, implicit make
   629 rules are disabled: installing with --local would triger those rules, and mconf
   630 was unbuildable.
   631 
   632 Kconfig parser |
   633 ---------------+
   634 
   635 The kconfig language is a hacked version, vampirised from the Linux kernel
   636 (http://www.kernel.org/), and (heavily) adapted to my needs.
   637 
   638 The list of the most notable changes (at least the ones I remember) follows:
   639 - the CONFIG_ prefix has been replaced with CT_
   640 - a leading | in prompts is skipped, and subsequent leading spaces are not
   641   trimmed
   642 - otherwise leading spaces are silently trimmed
   643 
   644 The kconfig parsers (conf and mconf) are not installed pre-built, but as
   645 source files. Thus you can have the directory where crosstool-NG is installed,
   646 exported (via NFS or whatever) and have clients with different architectures
   647 use the same crosstool-NG installation, and most notably, the same set of
   648 patches.
   649 
   650 Architecture-specific |
   651 ----------------------+
   652 
   653 Note: this chapter is not really well written, and might thus be a little bit
   654 complex to understand. To get a better grasp of what an architecture is, the
   655 reader is kindly encouraged to look at the "arch/" sub-directory, and to the
   656 existing architectures to see how things are laid out.
   657 
   658 An architecture is defined by:
   659 
   660  - a human-readable name, in lower case letters, with numbers as appropriate.
   661    The underscore is allowed; space and special characters are not.
   662      Eg.: arm, x86_64
   663  - a file in "config/arch/", named after the architecture's name, and suffixed
   664    with ".in".
   665      Eg.: config/arch/arm.in
   666  - a file in "scripts/build/arch/", named after the architecture's name, and
   667    suffixed with ".sh".
   668      Eg.: scripts/build/arch/arm.sh
   669 
   670 The architecture's ".in" file API:
   671  > the config option "ARCH_%arch%" (where %arch% is to be replaced with the
   672    actual architecture name).
   673    That config option must have *neither* a type, *nor* a prompt! Also, it can
   674    *not* depend on any other config option (EXPERIMENTAL is managed as above).
   675      Eg.:
   676        config ARCH_arm
   677    + mandatory:
   678        defines a (terse) help entry for this architecture:
   679        Eg.:
   680          config ARCH_arm
   681            help
   682              The ARM architecture.
   683    + optional:
   684        selects adequate associated config options.
   685        Note: 64-bit architectures *shall* select ARCH_64
   686        Eg.:
   687          config ARCH_arm
   688            select ARCH_SUPPORTS_BOTH_ENDIAN
   689            select ARCH_DEFAULT_LE
   690            help
   691              The ARM architecture.
   692        Eg.:
   693          config ARCH_x86_64
   694             select ARCH_64
   695             help
   696               The x86_64 architecture.
   697 
   698  > other target-specific options, at your discretion. Note however that to
   699    avoid name-clashing, such options shall be prefixed with "ARCH_%arch%",
   700    where %arch% is again replaced by the actual architecture name.
   701    (Note: due to historical reasons, and lack of time to clean up the code,
   702     I may have left some config options that do not completely conform to
   703     this, as the architecture name was written all upper case. However, the
   704     prefix is unique among architectures, and does not cause harm).
   705 
   706 The architecture's ".sh" file API:
   707  > the function "CT_DoArchTupleValues"
   708    + parameters: none
   709    + environment:
   710      - all variables from the ".config" file,
   711      - the two variables "target_endian_eb" and "target_endian_el" which are
   712        the endianness suffixes
   713    + return value: 0 upon success, !0 upon failure
   714    + provides:
   715      - mandatory
   716      - the environment variable CT_TARGET_ARCH
   717      - contains:
   718        the architecture part of the target tuple.
   719        Eg.: "armeb" for big endian ARM
   720             "i386" for an i386
   721    + provides:
   722      - optional
   723      - the environment variable CT_TARGET_SYS
   724      - contains:
   725        the sytem part of the target tuple.
   726        Eg.: "gnu" for glibc on most architectures
   727             "gnueabi" for glibc on an ARM EABI
   728      - defaults to:
   729        - for glibc-based toolchain: "gnu"
   730        - for uClibc-based toolchain: "uclibc"
   731    + provides:
   732      - optional
   733      - the environment variable CT_KERNEL_ARCH
   734      - contains:
   735        the architecture name as understandable by the Linux kernel build
   736        system.
   737        Eg.: "arm" for an ARM
   738             "powerpc" for a PowerPC
   739             "i386" for an x86
   740      - defaults to:
   741        ${CT_ARCH}
   742    + provides:
   743      - optional
   744      - the environment variables to configure the cross-gcc (defaults)
   745        - CT_ARCH_WITH_ARCH    : the gcc ./configure switch to select architecture level         ( "--with-arch=${CT_ARCH_ARCH}"   )
   746        - CT_ARCH_WITH_ABI     : the gcc ./configure switch to select ABI level                  ( "--with-abi=${CT_ARCH_ABI}"     )
   747        - CT_ARCH_WITH_CPU     : the gcc ./configure switch to select CPU instruction set        ( "--with-cpu=${CT_ARCH_CPU}"     )
   748        - CT_ARCH_WITH_TUNE    : the gcc ./configure switch to select scheduling                 ( "--with-tune=${CT_ARCH_TUNE}"   )
   749        - CT_ARCH_WITH_FPU     : the gcc ./configure switch to select FPU type                   ( "--with-fpu=${CT_ARCH_FPU}"     )
   750        - CT_ARCH_WITH_FLOAT   : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/  )
   751    + provides:
   752      - optional
   753      - the environment variables to pass to the cross-gcc to build target binaries (defaults)
   754        - CT_ARCH_ARCH_CFLAG   : the gcc switch to select architecture level                     ( "-march=${CT_ARCH_ARCH}"            )
   755        - CT_ARCH_ABI_CFLAG    : the gcc switch to select ABI level                              ( "-mabi=${CT_ARCH_ABI}"              )
   756        - CT_ARCH_CPU_CFLAG    : the gcc switch to select CPU instruction set                    ( "-mcpu=${CT_ARCH_CPU}"              )
   757        - CT_ARCH_TUNE_CFLAG   : the gcc switch to select scheduling                             ( "-mtune=${CT_ARCH_TUNE}"            )
   758        - CT_ARCH_FPU_CFLAG    : the gcc switch to select FPU type                               ( "-mfpu=${CT_ARCH_FPU}"              )
   759        - CT_ARCH_FLOAT_CFLAG  : the gcc switch to choose floating point arithmetics             ( "-msoft-float" or /empty/           )
   760        - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian                   ( "-mbig-endian" or "-mlittle-endian" )
   761      - default to:
   762        see above.
   763    + provides:
   764      - optional
   765      - the environement variables to configure the core and final compiler, specific to this architecture:
   766        - CT_ARCH_CC_CORE_EXTRA_CONFIG   : additional, architecture specific core gcc ./configure flags
   767        - CT_ARCH_CC_EXTRA_CONFIG        : additional, architecture specific final gcc ./configure flags
   768      - default to:
   769        - all empty
   770    + provides:
   771      - optional
   772      - the architecture-specific CFLAGS and LDFLAGS:
   773        - CT_ARCH_TARGET_CLFAGS
   774        - CT_ARCH_TARGET_LDFLAGS
   775      - default to:
   776        - all empty
   777 
   778 You can have a look at "config/arch/arm.in" and "scripts/build/arch/arm.sh" for
   779 a quite complete example of what an actual architecture description looks like.
   780 
   781 Kernel specific |
   782 ----------------+
   783 
   784 A kernel is defined by:
   785 
   786  - a human-readable name, in lower case letters, with numbers as appropriate.
   787    The underscore is allowed; space and special characters are not (although
   788    they are internally replaced with underscores.
   789      Eg.: linux, bare-metal
   790  - a file in "config/kernel/", named after the kernel name, and suffixed with
   791    ".in".
   792      Eg.: config/kernel/linux.in, config/kernel/bare-metal.in
   793  - a file in "scripts/build/kernel/", named after the kernel name, and suffixed
   794    with ".sh".
   795      Eg.: scripts/build/kernel/linux.sh, scripts/build/kernel/bare-metal.sh
   796 
   797 The kernel's ".in" file must contain:
   798  > an optional lines containing exactly "# EXPERIMENTAL", starting on the
   799    first column, and without any following space or other character.
   800    If this line is present, then this kernel is considered EXPERIMENTAL,
   801    and correct dependency on EXPERIMENTAL will be set.
   802 
   803  > the config option "KERNEL_%kernel_name%" (where %kernel_name% is to be
   804    replaced with the actual kernel name, with all special characters and
   805    spaces replaced by underscores).
   806    That config option must have *neither* a type, *nor* a prompt! Also, it can
   807    *not* depends on EXPERIMENTAL.
   808      Eg.: KERNEL_linux, KERNEL_bare_metal
   809    + mandatory:
   810        defines a (terse) help entry for this kernel.
   811        Eg.:
   812          config KERNEL_bare_metal
   813            help
   814              Build a compiler for use without any kernel.
   815    + optional:
   816        selects adequate associated config options.
   817        Eg.:
   818          config KERNEL_bare_metal
   819            select BARE_METAL
   820            help
   821              Build a compiler for use without any kernel.
   822 
   823  > other kernel specific options, at your discretion. Note however that, to
   824    avoid name-clashing, such options should be prefixed with
   825    "KERNEL_%kernel_name%", where %kernel_name% is again tp be replaced with
   826    the actual kernel name.
   827    (Note: due to historical reasons, and lack of time to clean up the code,
   828     I may have left some config options that do not completely conform to
   829     this, as the kernel name was written all upper case. However, the prefix
   830     is unique among kernels, and does not cause harm).
   831 
   832 The kernel's ".sh" file API:
   833  > is a bash script fragment
   834 
   835  > defines the function CT_DoKernelTupleValues
   836    + see the architecture's CT_DoArchTupleValues, except for:
   837    + set the environment variable CT_TARGET_KERNEL, the kernel part of the
   838      target tuple
   839    + return value: ignored
   840 
   841  > defines the function "do_kernel_get":
   842    + parameters: none
   843    + environment:
   844       - all variables from the ".config" file.
   845    + return value: 0 for success, !0 for failure.
   846    + behavior: download the kernel's sources, and store the tarball into
   847      "${CT_TARBALLS_DIR}". To this end, a functions is available, that
   848      abstracts downloading tarballs:
   849      - CT_DoGet <tarball_base_name> <URL1 [URL...]>
   850        Eg.: CT_DoGet linux-2.6.26.5 ftp://ftp.kernel.org/pub/linux/kernel/v2.6
   851      Note: retrieving sources from svn, cvs, git and the likes is not supported
   852      by CT_DoGet. You'll have to do this by hand, as it is done for eglibc in
   853      "scripts/build/libc/eglibc.sh"
   854 
   855  > defines the function "do_kernel_extract":
   856    + parameters: none
   857    + environment:
   858       - all variables from the ".config" file,
   859    + return value: 0 for success, !0 for failure.
   860    + behavior: extract the kernel's tarball into "${CT_SRC_DIR}", and apply
   861      required patches. To this end, a function is available, that abstracts
   862      extracting tarballs:
   863      - CT_ExtractAndPatch <tarball_base_name>
   864        Eg.: CT_ExtractAndPatch linux-2.6.26.5
   865 
   866  > defines the function "do_kernel_headers":
   867    + parameters: none
   868    + environment:
   869       - all variables from the ".config" file,
   870    + return value: 0 for success, !0 for failure.
   871    + behavior: install the kernel headers (if any) in "${CT_SYSROOT_DIR}/usr/include"
   872 
   873  > defines any kernel-specific helper functions
   874    These functions, if any, must be prefixed with "do_kernel_%CT_KERNEL%_",
   875    where '%CT_KERNEL%' is to be replaced with the actual kernel name, to avoid
   876    any name-clashing.
   877 
   878 You can have a look at "config/kernel/linux.in" and "scripts/build/kernel/linux.sh"
   879 as an example of what a complex kernel description looks like.
   880 
   881 Adding a new version of a component |
   882 ------------------------------------+
   883 
   884 When a new component, such as the Linux kernel, gcc or any other is released,
   885 adding the new version to crosstool-NG is quite easy. There is a script that
   886 will do all that for you:
   887   scripts/addToolVersion.sh
   888 
   889 Run it with no option to get some help.
   890 
   891 Build scripts |
   892 --------------+
   893 
   894 To Be Written later...