docs/overview.txt
author "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Sun Aug 30 00:27:12 2009 +0200 (2009-08-30)
changeset 1493 0dce3a3986a1
parent 1491 ae44a02d67fb
child 1513 0a405ac9d9ce
permissions -rw-r--r--
config: choose whether to use the shell or the C wrapper

Offer a config choice on whether to isntall the script wrapper, or
the compiled C wrapper. Update docs/overview.txt accordingly.
     1 File.........: overview.txt
     2 Content......: Overview of how crosstool-NG works.
     3 Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>
     4 License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
     5 
     6 ____________________
     7                    /
     8 Table Of Content  /
     9 _________________/
    10 
    11 
    12 Introduction
    13 History
    14 Installing crosstool-NG
    15   Install method
    16   The hacker's way
    17   Preparing for packaging
    18   Shell completion
    19   Contributed code
    20 Configuring crosstool-NG
    21   Interesting config options
    22   Re-building an existing toolchain
    23 Running crosstool-NG
    24   Stopping and restarting a build
    25   Testing all toolchains at once
    26   Overriding the number of // jobs
    27   Note on // jobs
    28   Tools wrapper
    29 Using the toolchain
    30 Toolchain types
    31 Internals
    32   Makefile front-end
    33   Kconfig parser
    34   Architecture-specific
    35   Adding a new version of a component
    36   Build scripts
    37 
    38 ________________
    39                /
    40 Introduction  /
    41 _____________/
    42 
    43 crosstool-NG aims at building toolchains. Toolchains are an essential component
    44 in a software development project. It will compile, assemble and link the code
    45 that is being developed. Some pieces of the toolchain will eventually end up
    46 in the resulting binary/ies: static libraries are but an example.
    47 
    48 So, a toolchain is a very sensitive piece of software, as any bug in one of the
    49 components, or a poorly configured component, can lead to execution problems,
    50 ranging from poor performance, to applications ending unexpectedly, to
    51 mis-behaving software (which more than often is hard to detect), to hardware
    52 damage, or even to human risks (which is more than regrettable).
    53 
    54 Toolchains are made of different piece of software, each being quite complex
    55 and requiring specially crafted options to build and work seamlessly. This
    56 is usually not that easy, even in the not-so-trivial case of native toolchains.
    57 The work reaches a higher degree of complexity when it comes to cross-
    58 compilation, where it can become quite a nightmare...
    59 
    60 Some cross-toolchains exist on the internet, and can be used for general
    61 development, but they have a number of limitations:
    62   - they can be general purpose, in that they are configured for the majority:
    63     no optimisation for your specific target,
    64   - they can be prepared for a specific target and thus are not easy to use,
    65     nor optimised for, or even supporting your target,
    66   - they often are using aging components (compiler, C library, etc...) not
    67     supporting special features of your shiny new processor;
    68 On the other side, these toolchain offer some advantages:
    69   - they are ready to use and quite easy to install and setup,
    70   - they are proven if used by a wide community.
    71 
    72 But once you want to get all the juice out of your specific hardware, you will
    73 want to build your own toolchain. This is where crosstool-NG comes into play.
    74 
    75 There are also a number of tools that build toolchains for specific needs,
    76 which are not really scalable. Examples are:
    77   - buildroot (buildroot.uclibc.org) whose main purpose is to build root file
    78     systems, hence the name. But once you have your toolchain with buildroot,
    79     part of it is installed in the root-to-be, so if you want to build a whole
    80     new root, you either have to save the existing one as a template and
    81     restore it later, or restart again from scratch. This is not convenient,
    82   - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very
    83     similar to buildroot,
    84   - other projects (openembedded.org for example), which are again used to
    85     build root file systems.
    86 
    87 crosstool-NG is really targeted at building toolchains, and only toolchains.
    88 It is then up to you to use it the way you want.
    89 
    90 ___________
    91           /
    92 History  /
    93 ________/
    94 
    95 crosstool was first 'conceived' by Dan Kegel, who offered it to the community
    96 as a set of scripts, a repository of patches, and some pre-configured, general
    97 purpose setup files to be used to configure crosstool. This is available at
    98 http://www.kegel.com/crosstool, and the subversion repository is hosted on
    99 google at http://code.google.com/p/crosstool/.
   100 
   101 I once managed to add support for uClibc-based toolchains, but it did not make
   102 into mainline, mostly because I didn't have time to port the patch forward to
   103 the new versions, due in part to the big effort it was taking.
   104 
   105 So I decided to clean up crosstool in the state it was, re-order the things
   106 in place, add appropriate support for what I needed, that is uClibc support
   107 and a menu-driven configuration, named the new implementation crosstool-NG,
   108 (standing for crosstool Next Generation, as many other comunity projects do,
   109 and as a wink at the TV series "Star Trek: The Next Generation" ;-) ) and
   110 made it available to the community, in case it was of interest to any one.
   111 
   112 ___________________________
   113                           /
   114 Installing crosstool-NG  /
   115 ________________________/
   116 
   117 There are two ways you can use crosstool-NG:
   118  - build and install it, then get rid of the sources like you'd do for most
   119    programs,
   120  - or only build it and run from the source directory.
   121 
   122 The former should be used if you got crosstool-NG from a packaged tarball, see
   123 "Install method", below, while the latter is most useful for developpers that
   124 checked the code out from SVN, and want to submit patches, see "The Hacker's
   125 way", below.
   126 
   127 Install method |
   128 ---------------+
   129 
   130 If you go for the install, then you just follow the classical, but yet easy
   131 ./configure way:
   132   ./configure --prefix=/some/place
   133   make
   134   make install
   135   export PATH="${PATH}:/some/place/bin"
   136 
   137 You can then get rid of crosstool-NG source. Next create a directory to serve
   138 as a working place, cd in there and run:
   139   ct-ng help
   140 
   141 See below for complete usage.
   142 
   143 The Hacker's way |
   144 -----------------+
   145 
   146 If you go the hacker's way, then the usage is a bit different, although very
   147 simple:
   148   ./configure --local
   149   make
   150 
   151 Now, *do not* remove crosstool-NG sources. They are needed to run crosstool-NG!
   152 Stay in the directory holding the sources, and run:
   153   ./ct-ng help
   154 
   155 See below for complete usage.
   156 
   157 Now, provided you checked-out the code, you can send me your interesting changes
   158 by running:
   159   svn diff
   160 
   161 and mailing me the result! :-P
   162 
   163 Preparing for packaging |
   164 ------------------------+
   165 
   166 If you plan on packaging crosstool-NG, you surely don't want to install it
   167 in your root file system. The install procedure of crosstool-NG honors the
   168 DESTDIR variable:
   169 
   170   ./configure --prefix=/usr
   171   make
   172   make DESTDIR=/packaging/place install
   173 
   174 Shell completion |
   175 -----------------+
   176 
   177 crosstool-NG comes with a shell script fragment that defines bash-compatible
   178 completion. That shell fragment is currently not installed automatically, but
   179 this is planned.
   180 
   181 To install the shell script fragment, you have two options:
   182  - install system-wide, most probably by copying ct-ng.comp into
   183    /etc/bash_completion.d/
   184  - install for a single user, by copying ct-ng.comp into ${HOME}/ and
   185    sourcing this file from your ${HOME}/.bashrc
   186 
   187 Contributed code |
   188 -----------------+
   189 
   190 Some people contibuted code that couldn't get merged for various reasons. This
   191 code is available as patches in the contrib/ sub-directory. These patches are
   192 to be applied to the source of crosstool-NG, prior to installing.
   193 
   194 An easy way to use contributed code is to pass the --with-contrib= option to
   195 ./configure. The possible values depend upon which contributions are packaged
   196 with your version, but you can get with it with passing one of those two
   197 special values:
   198   --with-contrib=list
   199     will list all available contributions
   200 
   201   --with-contrib=all
   202     will select all avalaible contributions
   203 
   204 There is no guarantee that a particuliar contribution applies to the current
   205 version of crosstool-ng, or that it will work at all. Use contributions at
   206 your own risk.
   207 
   208 ____________________________
   209                            /
   210 Configuring crosstool-NG  /
   211 _________________________/
   212 
   213 crosstool-NG is configured with a configurator presenting a menu-stuctured set
   214 of options. These options let you specify the way you want your toolchain
   215 built, where you want it installed, what architecture and specific processor it
   216 will support, the version of the components you want to use, etc... The
   217 value for those options are then stored in a configuration file.
   218 
   219 The configurator works the same way you configure your Linux kernel. It is
   220 assumed you now how to handle this.
   221 
   222 To enter the menu, type:
   223   ct-ng menuconfig
   224 
   225 Almost every config item has a help entry. Read them carefully.
   226 
   227 String and number options can refer to environment variables. In such a case,
   228 you must use the shell syntax: ${VAR}. You shall neither single- nor double-
   229 quote the string/number options.
   230 
   231 There are three environment variables that are computed by crosstool-NG, and
   232 that you can use:
   233 
   234 CT_TARGET:
   235   It represents the target tuple you are building for. You can use it for
   236   example in the installation/prefix directory, such as:
   237     /opt/x-tools/${CT_TARGET}
   238 
   239 CT_TOP_DIR:
   240   The top directory where crosstool-NG is running. You shouldn't need it in
   241   most cases. There is one case where you may need it: if you have local
   242   patches and you store them in your running directory, you can refer to them
   243   by using CT_TOP_DIR, such as:
   244     ${CT_TOP_DIR}/patches.myproject
   245 
   246 CT_VERSION:
   247   The version of crosstool-NG you are using. Not much use for you, but it's
   248   there if you need it.
   249 
   250 Interesting config options |
   251 ---------------------------+
   252 
   253 CT_LOCAL_TARBALLS_DIR:
   254   If you already have some tarballs in a direcotry, enter it here. That will
   255   speed up the retrieving phase, where crosstool-NG would otherwise download
   256   those tarballs.
   257 
   258 CT_PREFIX_DIR:
   259   This is where the toolchain will be installed in (and for now, where it
   260   will run from). Common use is to add the target tuple in the directory
   261   path, such as (see above):
   262     /opt/x-tools/${CT_TARGET}
   263 
   264 CT_TARGET_VENDOR:
   265   An identifier for your toolchain, will take place in the vendor part of the
   266   target tuple. It shall *not* contain spaces or dashes. Usually, keep it
   267   to a one-word string, or use underscores to separate words if you need.
   268   Avoid dots, commas, and special characters.
   269 
   270 CT_TARGET_ALIAS:
   271   An alias for the toolchian. It will be used as a prefix to the toolchain
   272   tools. For example, you will have ${CT_TARGET_ALIAS}-gcc
   273 
   274 Also, if you think you don't see enough versions, you can try to enable one of
   275 those:
   276 
   277 CT_OBSOLETE:
   278   Show obsolete versions or tools. Most of the time, you don't want to base
   279   your toolchain on too old a version (of gcc, for example). But at times, it
   280   can come handy to use such an old version for regression tests. Those old
   281   versions are hidden behind CT_OBSOLETE. Those versions (or features) are so
   282   marked because maintaining support for those in crosstool-NG would be too
   283   costly, time-wise, and time is dear.
   284 
   285 CT_EXPERIMENTAL:
   286   Show experimental versions or tools. Again, you might not want to base your
   287   toolchain on too recent tools (eg. gcc) for production. But if you need a
   288   feature present only in a recent version, or a new tool, you can find them
   289   hidden behind CT_EXPERIMENTAL. Those versions (or features) did not (yet)
   290   receive thorough testing in crosstool-NG, and/or are not mature enough to
   291   be blindly trusted.
   292 
   293 Re-building an existing toolchain |
   294 ----------------------------------+
   295 
   296 If you have an existing toolchain, you can re-use the options used to build it
   297 to create a new toolchain. That needs a very little bit of effort on your side
   298 but is quite easy. The options to build a toolchain are saved with the
   299 toolchain, and you can retrieve this configuration by running:
   300   ${CT_TARGET}-config
   301 
   302 This will dump the configuration to stdout, so to rebuild a toolchain with this
   303 configuration, the following is all you need to do:
   304   ${CT_TARGET}-config >.config
   305   ct-ng oldconfig
   306 
   307 Then, you can review and change the configuration by running:
   308   ct-ng menuconfig
   309 
   310 ________________________
   311                        /
   312 Running crosstool-NG  /
   313 _____________________/
   314 
   315 To build the toolchain, simply type:
   316   ct-ng build
   317 
   318 This will use the above configuration to retrieve, extract and patch the
   319 components, build, install and eventually test your newly built toolchain.
   320 
   321 You are then free to add the toolchain /bin directory in your PATH to use
   322 it at will.
   323 
   324 In any case, you can get some terse help. Just type:
   325   ct-ng help
   326 or:
   327   man 1 ct-ng
   328 
   329 Stopping and restarting a build |
   330 --------------------------------+
   331 
   332 If you want to stop the build after a step you are debugging, you can pass the
   333 variable STOP to make:
   334   ct-ng STOP=some_step
   335 
   336 Conversely, if you want to restart a build at a specific step you are
   337 debugging, you can pass the RESTART variable to make:
   338   ct-ng RESTART=some_step
   339 
   340 Alternatively, you can call make with the name of a step to just do that step:
   341   ct-ng libc_headers
   342 is equivalent to:
   343   ct-ng RESTART=libc_headers STOP=libc_headers
   344 
   345 The shortcuts +step_name and step_name+ allow to respectively stop or restart
   346 at that step. Thus:
   347   ct-ng +libc_headers        and:    ct-ng libc_headers+
   348 are equivalent to:
   349   ct-ng STOP=libc_headers    and:    ct-ng RESTART=libc_headers
   350 
   351 To obtain the list of acceptable steps, please call:
   352   ct-ng list-steps
   353 
   354 Note that in order to restart a build, you'll have to say 'Y' to the config
   355 option CT_DEBUG_CT_SAVE_STEPS, and that the previous build effectively went
   356 that far.
   357 
   358 Building all toolchains at once |
   359 --------------------------------+
   360 
   361 You can build all samples; simply call:
   362   ct-ng build-all
   363 
   364 Overriding the number of // jobs |
   365 ---------------------------------+
   366 
   367 If you want to override the number of jobs to run in // (the -j option to
   368 make), you can either re-enter the menuconfig, or simply add it on the command
   369 line, as such:
   370   ct-ng build.4
   371 
   372 which tells crosstool-NG to override the number of // jobs to 4.
   373 
   374 You can see the actions that support overriding the number of // jobs in
   375 the help menu. Those are the ones with [.#] after them (eg. build[.#] or
   376 build-all[.#], and so on...).
   377 
   378 Note on // jobs |
   379 ----------------+
   380 
   381 The crosstool-NG script 'ct-ng' is a Makefile-script. It does *not* execute
   382 in parallel (there is not much to gain). When speaking of // jobs, we are
   383 refering to the number of // jobs when making the *components*. That is, we
   384 speak of the number of // jobs used to build gcc, glibc, and so on...
   385 
   386 Tools wrapper |
   387 --------------+
   388 
   389 Starting with gcc-4.3 come two new dependencies: GMP and MPFR. With gcc-4.4,
   390 come three new ones: GMP, PPL and CLooG/ppl. These are libraries that enable
   391 advanced features to gcc. Additionally, some of the libraries can be used by
   392 binutils and gdb. Unfortunately, not all systems on which crosstool-NG runs
   393 have all of those libraries. And for those that do, the versions of those
   394 libraries may be older than the version required by gcc.
   395 
   396 This is why crosstool-NG builds its own set of libraries as part of the
   397 toolchain.
   398 
   399 The libraries are built as shared libraries, because building them as static
   400 libraries has some short-comings. This poses no problem at build time, as
   401 crosstool-NG correctly points gcc (and binutiols and gdb) to the correct
   402 place where our own version of the libraries are installed. But it poses
   403 a problem when gcc et al. are run: the place where the libraries are is most
   404 probably not known to the host dynamic linker. Still worse, if the host system
   405 has its own versions, then ld.so would load the wrong library!
   406 
   407 So we have to force the dynamic linker to load the correct version. We do this
   408 by using the LD_LIBRARY_PATH variable, that informs the dynamic linker where
   409 to look for shared libraries prior to searching its standard places. But we
   410 can't impose that burden on all the system (because it'd be a nightmare to
   411 configure, and because two tolchains on the same system may use different
   412 versions of the libraries); so we have to do it on a per-toolchain basis.
   413 
   414 So we rename all binaries of the toolchain (by adding a dot '.' as their first
   415 character), and add a small program, the so-called "tools wrapper", that
   416 correctly sets LD_LIBRARY_PATH prior to running the real tool.
   417 
   418 First, the wrapper was written as a POSIX-compliant shell script. That shell
   419 script is very simple, if not trivial, and works great. The only drawback is
   420 that it does not work on host systems that lack a shell, for example the
   421 MingW32 environment. To solve the issue, the wrapper has been re-written in C,
   422 and compiled at build time. This C wrapper is much more complex than the shell
   423 script, and although it sems to be working, it's been only lightly tested.
   424 Some of the expected short-comings with this C wrapper are;
   425  - multi-byte file names may not be handled correctly
   426  - it's really big for what it does
   427 
   428 So, the default wrapper installed with your toolchain is the shell script.
   429 If you know that your system is missing a shell, then you shall use the C
   430 wrapper (and report back whether it works, or does not work, for you).
   431 
   432 
   433 _______________________
   434                       /
   435 Using the toolchain  /
   436 ____________________/
   437 
   438 Using the toolchain is as simple as adding the toolchain's bin directory in
   439 your PATH, such as:
   440   export PATH="${PATH}:/your/toolchain/path/bin"
   441 
   442 and then using the target tuple to tell the build systems to use your
   443 toolchain:
   444   ./configure --target=your-target-tuple
   445 or
   446   make CC=your-target-tuple-gcc
   447 or
   448   make CROSS_COMPILE=your-target-tuple-
   449 and so on...
   450 
   451 It is strongly advised not to use the toolchain sys-root directory as an
   452 install directory for your programs/packages. If you do so, you will not be
   453 able to use your toolchain for another project. It is even strongly advised
   454 that your toolchain is chmod-ed to read-only once successfully build, so that
   455 you don't go polluting your toolchain with your programs/packages' files.
   456 
   457 Thus, when you build a program/package, install it in a separate directory,
   458 eg. /your/root. This directory is the /image/ of what would be in the root file
   459 system of your target, and will contain all that your programs/packages have
   460 installed.
   461 
   462 The 'populate' script |
   463 ----------------------+
   464 
   465 When your root directory is ready, it is still missing some important bits: the
   466 toolchain's libraries. To populate your root directory with those libs, just
   467 run:
   468   your-target-tuple-populate -s /your/root -d /your/root-populated
   469 
   470 This will copy /your/root into /your/root-populated, and put the needed and only
   471 the needed libraries there. Thus you don't polute /your/root with any cruft that
   472 would no longer be needed should you have to remove stuff. /your/root always
   473 contains only those things you install in it.
   474 
   475 You can then use /your/root-populated to build up your file system image, a
   476 tarball, or to NFS-mount it from your target, or whatever you need.
   477 
   478 The populate script accepts the following options:
   479 
   480  -s src_dir
   481     Use 'src_dir' as the un-populated root directory.
   482 
   483  -d dst_dir
   484     Put the populated root directory in 'dst_dir'.
   485 
   486  -l lib1 [...]
   487     Always add specified libraries.
   488 
   489  -L file
   490     Always add libraries listed in 'file'.
   491 
   492  -f
   493     Remove 'dst_dir' if it previously existed; continue even if any library
   494     specified with -l or -L is missing.
   495 
   496  -v
   497     Be verbose, and tell what's going on (you can see exactly where libs are
   498     coming from).
   499 
   500  -h
   501     Print the help.
   502 
   503 See 'your-target-tuple-populate -h' for more information on the options.
   504 
   505 Here is how populate works:
   506 
   507   1) performs some sanity checks:
   508      - src_dir and dst_dir are specified
   509      - src_dir exists
   510      - unless forced, dst_dir does not exist
   511      - src_dir != dst_dir
   512 
   513   2) copy src_dir to dst_dir
   514 
   515   3) add forced libraries to dst_dir
   516      - build the list from -l and -L options
   517      - get forced libraries from the sysroot (see below for heuristics)
   518        - abort on the first missing library, unless -f is specified
   519 
   520   4) add all missing libraries to dst_dir
   521      - scan dst_dir for every ELF files that are 'executable' or
   522        'shared object'
   523      - list the "NEEDED Shared library" fields
   524        - check if the library is already in dst_dir/lib or dst_dir/usr/lib
   525        - if not, get the library from the sysroot
   526          - if it's in sysroot/lib, copy it to dst_dir/lib
   527          - if it's in sysroot/usr/lib, copy it to dst_dir/usr/lib
   528          - in both cases, use the SONAME of the library to create the file
   529            in dst_dir
   530          - if it was not found in the sysroot, this is an error.
   531 
   532 ___________________
   533                   /
   534 Toolchain types  /
   535 ________________/
   536 
   537 There are four kinds of toolchains you could encounter.
   538 
   539 First off, you must understand the following: when it comes to compilers there
   540 are up to four machines involved:
   541   1) the machine configuring the toolchain components: the config machine
   542   2) the machine building the toolchain components:    the build machine
   543   3) the machine running the toolchain:                the host machine
   544   4) the machine the toolchain is generating code for: the target machine
   545 
   546 We can most of the time assume that the config machine and the build machine
   547 are the same. Most of the time, this will be true. The only time it isn't
   548 is if you're using distributed compilation (such as distcc). Let's forget
   549 this for the sake of simplicity.
   550 
   551 So we're left with three machines:
   552  - build
   553  - host
   554  - target
   555 
   556 Any toolchain will involve those three machines. You can be as pretty sure of
   557 this as "2 and 2 are 4". Here is how they come into play:
   558 
   559 1) build == host == target
   560     This is a plain native toolchain, targetting the exact same machine as the
   561     one it is built on, and running again on this exact same machine. You have
   562     to build such a toolchain when you want to use an updated component, such
   563     as a newer gcc for example.
   564     crosstool-NG calls it "native".
   565 
   566 2) build == host != target
   567     This is a classic cross-toolchain, which is expected to be run on the same
   568     machine it is compiled on, and generate code to run on a second machine,
   569     the target.
   570     crosstool-NG calls it "cross".
   571 
   572 3) build != host == target
   573     Such a toolchain is also a native toolchain, as it targets the same machine
   574     as it runs on. But it is build on another machine. You want such a
   575     toolchain when porting to a new architecture, or if the build machine is
   576     much faster than the host machine.
   577     crosstool-NG calls it "cross-native".
   578 
   579 4) build != host != target
   580     This one is called a canadian-toolchain (*), and is tricky. The three
   581     machines in play are different. You might want such a toolchain if you
   582     have a fast build machine, but the users will use it on another machine,
   583     and will produce code to run on a third machine.
   584     crosstool-NG calls it "canadian".
   585 
   586 crosstool-NG can build all these kinds of toolchains (or is aiming at it,
   587 anyway!)
   588 
   589 (*) The term Canadian Cross came about because at the time that these issues
   590     were all being hashed out, Canada had three national political parties.
   591     http://en.wikipedia.org/wiki/Cross_compiler
   592 
   593 _____________
   594             /
   595 Internals  /
   596 __________/
   597 
   598 Internally, crosstool-NG is script-based. To ease usage, the frontend is
   599 Makefile-based.
   600 
   601 Makefile front-end |
   602 -------------------+
   603 
   604 The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
   605 script with an action will act exactly as if the Makefile was in the current
   606 working directory and make was called with the action as rule. Thus:
   607   ct-ng menuconfig
   608 
   609 is equivalent to having the Makefile in CWD, and calling:
   610   make menuconfig
   611 
   612 Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
   613 traditional command.
   614 
   615 ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
   616 at configuration time with ./configure.
   617 
   618 ct-ng also searches for config files, sub-tools, samples, scripts and patches in
   619 that library directory.
   620 
   621 Because of a stupid make behavior/bug I was unable to track down, implicit make
   622 rules are disabled: installing with --local would triger those rules, and mconf
   623 was unbuildable.
   624 
   625 Kconfig parser |
   626 ---------------+
   627 
   628 The kconfig language is a hacked version, vampirised from the Linux kernel
   629 (http://www.kernel.org/), and (heavily) adapted to my needs.
   630 
   631 The list of the most notable changes (at least the ones I remember) follows:
   632 - the CONFIG_ prefix has been replaced with CT_
   633 - a leading | in prompts is skipped, and subsequent leading spaces are not
   634   trimmed
   635 - otherwise leading spaces are silently trimmed
   636 
   637 The kconfig parsers (conf and mconf) are not installed pre-built, but as
   638 source files. Thus you can have the directory where crosstool-NG is installed,
   639 exported (via NFS or whatever) and have clients with different architectures
   640 use the same crosstool-NG installation, and most notably, the same set of
   641 patches.
   642 
   643 Architecture-specific |
   644 ----------------------+
   645 
   646 Note: this chapter is not really well written, and might thus be a little bit
   647 complex to understand. To get a better grasp of what an architecture is, the
   648 reader is kindly encouraged to look at the "arch/" sub-directory, and to the
   649 existing architectures to see how things are laid out.
   650 
   651 An architecture is defined by:
   652 
   653  - a human-readable name, in lower case letters, with numbers as appropriate.
   654    The underscore is allowed; space and special characters are not.
   655      Eg.: arm, x86_64
   656  - a file in "config/arch/", named after the architecture's name, and suffixed
   657    with ".in".
   658      Eg.: config/arch/arm.in
   659  - a file in "scripts/build/arch/", named after the architecture's name, and
   660    suffixed with ".sh".
   661      Eg.: scripts/build/arch/arm.sh
   662 
   663 The architecture's ".in" file API:
   664  > the config option "ARCH_%arch%" (where %arch% is to be replaced with the
   665    actual architecture name).
   666    That config option must have *neither* a type, *nor* a prompt! Also, it can
   667    *not* depend on any other config option (EXPERIMENTAL is managed as above).
   668      Eg.:
   669        config ARCH_arm
   670    + mandatory:
   671        defines a (terse) help entry for this architecture:
   672        Eg.:
   673          config ARCH_arm
   674            help
   675              The ARM architecture.
   676    + optional:
   677        selects adequate associated config options.
   678        Note: 64-bit architectures *shall* select ARCH_64
   679        Eg.:
   680          config ARCH_arm
   681            select ARCH_SUPPORTS_BOTH_ENDIAN
   682            select ARCH_DEFAULT_LE
   683            help
   684              The ARM architecture.
   685        Eg.:
   686          config ARCH_x86_64
   687             select ARCH_64
   688             help
   689               The x86_64 architecture.
   690 
   691  > other target-specific options, at your discretion. Note however that to
   692    avoid name-clashing, such options shall be prefixed with "ARCH_%arch%",
   693    where %arch% is again replaced by the actual architecture name.
   694    (Note: due to historical reasons, and lack of time to clean up the code,
   695     I may have left some config options that do not completely conform to
   696     this, as the architecture name was written all upper case. However, the
   697     prefix is unique among architectures, and does not cause harm).
   698 
   699 The architecture's ".sh" file API:
   700  > the function "CT_DoArchTupleValues"
   701    + parameters: none
   702    + environment:
   703      - all variables from the ".config" file,
   704      - the two variables "target_endian_eb" and "target_endian_el" which are
   705        the endianness suffixes
   706    + return value: 0 upon success, !0 upon failure
   707    + provides:
   708      - mandatory
   709      - the environment variable CT_TARGET_ARCH
   710      - contains:
   711        the architecture part of the target tuple.
   712        Eg.: "armeb" for big endian ARM
   713             "i386" for an i386
   714    + provides:
   715      - optional
   716      - the environment variable CT_TARGET_SYS
   717      - contains:
   718        the sytem part of the target tuple.
   719        Eg.: "gnu" for glibc on most architectures
   720             "gnueabi" for glibc on an ARM EABI
   721      - defaults to:
   722        - for glibc-based toolchain: "gnu"
   723        - for uClibc-based toolchain: "uclibc"
   724    + provides:
   725      - optional
   726      - the environment variable CT_KERNEL_ARCH
   727      - contains:
   728        the architecture name as understandable by the Linux kernel build
   729        system.
   730        Eg.: "arm" for an ARM
   731             "powerpc" for a PowerPC
   732             "i386" for an x86
   733      - defaults to:
   734        ${CT_ARCH}
   735    + provides:
   736      - optional
   737      - the environment variables to configure the cross-gcc (defaults)
   738        - CT_ARCH_WITH_ARCH    : the gcc ./configure switch to select architecture level         ( "--with-arch=${CT_ARCH_ARCH}"   )
   739        - CT_ARCH_WITH_ABI     : the gcc ./configure switch to select ABI level                  ( "--with-abi=${CT_ARCH_ABI}"     )
   740        - CT_ARCH_WITH_CPU     : the gcc ./configure switch to select CPU instruction set        ( "--with-cpu=${CT_ARCH_CPU}"     )
   741        - CT_ARCH_WITH_TUNE    : the gcc ./configure switch to select scheduling                 ( "--with-tune=${CT_ARCH_TUNE}"   )
   742        - CT_ARCH_WITH_FPU     : the gcc ./configure switch to select FPU type                   ( "--with-fpu=${CT_ARCH_FPU}"     )
   743        - CT_ARCH_WITH_FLOAT   : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/  )
   744    + provides:
   745      - optional
   746      - the environment variables to pass to the cross-gcc to build target binaries (defaults)
   747        - CT_ARCH_ARCH_CFLAG   : the gcc switch to select architecture level                     ( "-march=${CT_ARCH_ARCH}"            )
   748        - CT_ARCH_ABI_CFLAG    : the gcc switch to select ABI level                              ( "-mabi=${CT_ARCH_ABI}"              )
   749        - CT_ARCH_CPU_CFLAG    : the gcc switch to select CPU instruction set                    ( "-mcpu=${CT_ARCH_CPU}"              )
   750        - CT_ARCH_TUNE_CFLAG   : the gcc switch to select scheduling                             ( "-mtune=${CT_ARCH_TUNE}"            )
   751        - CT_ARCH_FPU_CFLAG    : the gcc switch to select FPU type                               ( "-mfpu=${CT_ARCH_FPU}"              )
   752        - CT_ARCH_FLOAT_CFLAG  : the gcc switch to choose floating point arithmetics             ( "-msoft-float" or /empty/           )
   753        - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian                   ( "-mbig-endian" or "-mlittle-endian" )
   754      - default to:
   755        see above.
   756    + provides:
   757      - optional
   758      - the environement variables to configure the core and final compiler, specific to this architecture:
   759        - CT_ARCH_CC_CORE_EXTRA_CONFIG   : additional, architecture specific core gcc ./configure flags
   760        - CT_ARCH_CC_EXTRA_CONFIG        : additional, architecture specific final gcc ./configure flags
   761      - default to:
   762        - all empty
   763    + provides:
   764      - optional
   765      - the architecture-specific CFLAGS and LDFLAGS:
   766        - CT_ARCH_TARGET_CLFAGS
   767        - CT_ARCH_TARGET_LDFLAGS
   768      - default to:
   769        - all empty
   770 
   771 You can have a look at "config/arch/arm.in" and "scripts/build/arch/arm.sh" for
   772 a quite complete example of what an actual architecture description looks like.
   773 
   774 Kernel specific |
   775 ----------------+
   776 
   777 A kernel is defined by:
   778 
   779  - a human-readable name, in lower case letters, with numbers as appropriate.
   780    The underscore is allowed; space and special characters are not (although
   781    they are internally replaced with underscores.
   782      Eg.: linux, bare-metal
   783  - a file in "config/kernel/", named after the kernel name, and suffixed with
   784    ".in".
   785      Eg.: config/kernel/linux.in, config/kernel/bare-metal.in
   786  - a file in "scripts/build/kernel/", named after the kernel name, and suffixed
   787    with ".sh".
   788      Eg.: scripts/build/kernel/linux.sh, scripts/build/kernel/bare-metal.sh
   789 
   790 The kernel's ".in" file must contain:
   791  > an optional lines containing exactly "# EXPERIMENTAL", starting on the
   792    first column, and without any following space or other character.
   793    If this line is present, then this kernel is considered EXPERIMENTAL,
   794    and correct dependency on EXPERIMENTAL will be set.
   795 
   796  > the config option "KERNEL_%kernel_name%" (where %kernel_name% is to be
   797    replaced with the actual kernel name, with all special characters and
   798    spaces replaced by underscores).
   799    That config option must have *neither* a type, *nor* a prompt! Also, it can
   800    *not* depends on EXPERIMENTAL.
   801      Eg.: KERNEL_linux, KERNEL_bare_metal
   802    + mandatory:
   803        defines a (terse) help entry for this kernel.
   804        Eg.:
   805          config KERNEL_bare_metal
   806            help
   807              Build a compiler for use without any kernel.
   808    + optional:
   809        selects adequate associated config options.
   810        Eg.:
   811          config KERNEL_bare_metal
   812            select BARE_METAL
   813            help
   814              Build a compiler for use without any kernel.
   815 
   816  > other kernel specific options, at your discretion. Note however that, to
   817    avoid name-clashing, such options should be prefixed with
   818    "KERNEL_%kernel_name%", where %kernel_name% is again tp be replaced with
   819    the actual kernel name.
   820    (Note: due to historical reasons, and lack of time to clean up the code,
   821     I may have left some config options that do not completely conform to
   822     this, as the kernel name was written all upper case. However, the prefix
   823     is unique among kernels, and does not cause harm).
   824 
   825 The kernel's ".sh" file API:
   826  > is a bash script fragment
   827 
   828  > defines the function CT_DoKernelTupleValues
   829    + see the architecture's CT_DoArchTupleValues, except for:
   830    + set the environment variable CT_TARGET_KERNEL, the kernel part of the
   831      target tuple
   832    + return value: ignored
   833 
   834  > defines the function "do_kernel_get":
   835    + parameters: none
   836    + environment:
   837       - all variables from the ".config" file.
   838    + return value: 0 for success, !0 for failure.
   839    + behavior: download the kernel's sources, and store the tarball into
   840      "${CT_TARBALLS_DIR}". To this end, a functions is available, that
   841      abstracts downloading tarballs:
   842      - CT_DoGet <tarball_base_name> <URL1 [URL...]>
   843        Eg.: CT_DoGet linux-2.6.26.5 ftp://ftp.kernel.org/pub/linux/kernel/v2.6
   844      Note: retrieving sources from svn, cvs, git and the likes is not supported
   845      by CT_DoGet. You'll have to do this by hand, as it is done for eglibc in
   846      "scripts/build/libc/eglibc.sh"
   847 
   848  > defines the function "do_kernel_extract":
   849    + parameters: none
   850    + environment:
   851       - all variables from the ".config" file,
   852    + return value: 0 for success, !0 for failure.
   853    + behavior: extract the kernel's tarball into "${CT_SRC_DIR}", and apply
   854      required patches. To this end, a function is available, that abstracts
   855      extracting tarballs:
   856      - CT_ExtractAndPatch <tarball_base_name>
   857        Eg.: CT_ExtractAndPatch linux-2.6.26.5
   858 
   859  > defines the function "do_kernel_headers":
   860    + parameters: none
   861    + environment:
   862       - all variables from the ".config" file,
   863    + return value: 0 for success, !0 for failure.
   864    + behavior: install the kernel headers (if any) in "${CT_SYSROOT_DIR}/usr/include"
   865 
   866  > defines any kernel-specific helper functions
   867    These functions, if any, must be prefixed with "do_kernel_%CT_KERNEL%_",
   868    where '%CT_KERNEL%' is to be replaced with the actual kernel name, to avoid
   869    any name-clashing.
   870 
   871 You can have a look at "config/kernel/linux.in" and "scripts/build/kernel/linux.sh"
   872 as an example of what a complex kernel description looks like.
   873 
   874 Adding a new version of a component |
   875 ------------------------------------+
   876 
   877 When a new component, such as the Linux kernel, gcc or any other is released,
   878 adding the new version to crosstool-NG is quite easy. There is a script that
   879 will do all that for you:
   880   scripts/addToolVersion.sh
   881 
   882 Run it with no option to get some help.
   883 
   884 Build scripts |
   885 --------------+
   886 
   887 To Be Written later...