docs/overview.txt
author "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Fri May 25 19:30:42 2007 +0000 (2007-05-25)
changeset 135 b2695c2f1919
parent 92 1159b1384a78
child 136 22b5ef41df97
permissions -rw-r--r--
Add the possibility to stop after a specified step.
Update the Makefile help and the documentation accordingly.
     1 File.........: overview.txt
     2 Content......: Overview of how ct-ng works.
     3 Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>
     4 License......: see COPYING in the root of this package
     5 
     6 ________________
     7                /
     8 Introduction  /
     9 _____________/
    10 
    11 crosstool-NG aims at building toolchains. Toolchains are an essential component
    12 in a software development project. It will compile, assemble and link the code
    13 that is being developped. Some pieces of the toolchain will eventually end up
    14 in the resulting binary/ies: static libraries are but an example.
    15 
    16 So, a toolchain is a very sensitive piece of software, as any bug in one of the
    17 components, or a poorly configured component, can lead to execution problems,
    18 ranging from poor performance, to applications ending unexpectedly, to
    19 mis-behaving software (which more than often is hard to detect), to hardware
    20 damage, or even to human risks (which is more than regretable).
    21 
    22 Toolchains are made of different piece of software, each being quite complex
    23 and requiring specially crafted options to build and work seamlessly. This
    24 is usually not that easy, even in the not-so-trivial case of native toolchains.
    25 The work reaches a higher degree of complexity when it comes to cross-
    26 compilation, where it can become quite a nightmare...
    27 
    28 Some cross-toolchains exist on the internet, and can be used for general
    29 development, but they have a number of limitations:
    30   - they can be general purpose, in that they are configured for the majority:
    31     no optimisation for your specific target,
    32   - they can be prepared for a specific target and thus are not easy to use,
    33     nor optimised for, or even supporting your target,
    34   - they often are using ageing components (compiler, C library, etc...) not
    35     supporting special features of your shiny new processor;
    36 On the other side, these toolchain offer some advantages:
    37   - they are ready to use and quite easy to install and setup,
    38   - they are proven if used by a wide community.
    39 
    40 But once you want to get all the juice out of your specific hardware, you will
    41 want to build your own toolchain. This is where crosstool-ng comes into play.
    42 
    43 There are also a number of tools that builds toolchains for specific needs,
    44 which is not really scalable. Examples are:
    45   - buildroot (buildroot.uclibc.org) whose main puprpose is to build root file
    46     systems, hence the name. But once you have your toolchain with buildroot,
    47     part of it is installed in the root-to-be, so if you want to build a whole
    48     new root, you either have to save the existing one as a template and
    49     restore it later, or restart again from scratch. This is not convenient,
    50   - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very
    51     similar to buildroot,
    52   - other projects (openembeded.org for example), which is again used to
    53     build root file systems.
    54 
    55 crosstool-NG is really targetted at building toolchains, and only toolchains.
    56 It is then up to you to use it the way you want.
    57 
    58 ___________
    59           /
    60 History  /
    61 ________/
    62 
    63 crosstool was first 'conceived' by Dan Kegel, which offered it to the community,
    64 as a set of scripts, a repository of patches, and some pre-configured, general
    65 purpose setup files to be used to configure crosstool. This is available at
    66 www.kegel.com/crosstool, and the subversion repository is hosted on google at
    67 http://code.google.com/p/crosstool/.
    68 
    69 At the time of writing, crosstool only supports building with one C library,
    70 namely glibc, and one C compiler, gcc; it is cripled with historical support
    71 for legacy components, and is some kind of a mess to upgrade. Also, submited
    72 patches take a looong time before they are integrated mainline.
    73 
    74 I once managed to add support for uClibc-based toolchains, but it did not make
    75 into mainline, mostly because I don't have time to port the patch forward to
    76 the new versions, due in part to the big effort it was taking.
    77 
    78 So I decided to clean up crosstool in the state it was, re-order the things
    79 in place, and add appropriate support for what I needed, that is uClibc
    80 support.
    81 
    82 The only option left to me was rewrite crosstool from scratch. I decided to go
    83 this way, and name the new implementation ct-ng, standing for crosstool Next
    84 Generation, as many other comunity projects do, and as a wink at the TV series
    85 "Star Trek: The Next Generation". ;-)
    86 
    87 _____________
    88             /
    89 Operation  /
    90 __________/
    91 
    92 ct-ng is configured by a configurator presenting a menu-stuctured set of
    93 options. These options let you specify the way you want your toolchain built,
    94 where you want it installed, what architecture and specific processor it
    95 will support, the version of the components you want to use, etc... The
    96 value for those options are then stored in a configuration file.
    97 
    98 To enter the menu, type:
    99   make menuconfig
   100 
   101 To build the so-configured target, simply type:
   102   make
   103 
   104 This will use the above configuration to retrieve, extract and patch the
   105 components, build, install and eventually test your newly built toolchain.
   106 
   107 You are then free to add the toolchain /bin directory in your PATH to use
   108 it at will.
   109 
   110 In any case, you can get some terse help. Just type:
   111   make help
   112 
   113 
   114 Stoping and restarting a build |
   115 -------------------------------*
   116 
   117 If you want to stop the build after a step you are debugging, you can pass the
   118 variable STOP to make:
   119   make STOP=some_step
   120 
   121 Conversely, if you want to restart a build at a specific step you are
   122 debugging, you can pass the RESTART variable to make:
   123   make RESTART=some_step
   124 
   125 The list of steps is, in order of appearence in the build process:
   126   - libc_check_config
   127   - kernel_check_config
   128   - kernel_headers
   129   - binutils
   130   - libc_headers
   131   - cc_core
   132   - libfloat
   133   - libc
   134   - cc
   135   - libc_finish
   136   - debug
   137 
   138 ____________________________
   139                            /
   140 Configuring crosstool-NG  /
   141 _________________________/
   142 
   143 crosstool-NG is configured the same way you configure your Linux kernel: by
   144 using a curses-based menu. It is assumed you now how to handle this.
   145 
   146 Almost every config item has a help entry. Read it carefully.
   147 
   148 String and number options can refer to environment variables. In such a case,
   149 you  must use the shell syntax: ${VAR}. No such option is ever needed by make.
   150 You need to neither single- nor double-quote the string options.
   151 
   152 There are three environment variablea that are computed by crosstool-NG, and
   153 that you can use:
   154 
   155 CT_TARGET:
   156   It represents the target triplet you are building for. You can use it for
   157   example in the installation/prefix directory, such as:
   158     /opt/x-tools/${CT_TARGET}
   159 
   160 CT_TOP_DIR:
   161   The top directory where crosstool-NG sits. You shouldn't need it in most
   162   cases. There is one case where you may need it: if you have local patches
   163   and you store them in your copy of crosstool-NG, you can refer to them
   164   by using CT_TOP_DIR, such as:
   165     ${CT_TOP_DIR}/patches.myproject
   166 
   167 CT_VERSION:
   168   The version of crosstool-NG you are using. Not much help for you, but it's
   169   there if you need it.
   170 
   171 
   172 Interesting config options |
   173 ---------------------------*
   174 
   175 CT_LOCAL_TARBALLS_DIR:
   176   If you already have sone tarballs in a direcotry, enter it here. That will
   177   speed up the retrieving phase, where crosstool-ng would otherwise download
   178   those tarballs.
   179 
   180 CT_PREFIX_DIR:
   181   This is where the toolchain will be installed in (and for now, where it
   182   will run from).
   183 
   184 CT_LOG_FILE:
   185   The file where *all* log messages will go. Keep the default, in goes in
   186   ${CT_PREFIX_DIR}/${CT_TARGET}.log
   187 
   188 CT_TARGET_VENDOR:
   189   An identifier for your toolchain, will take place in the vendor part of the
   190   target triplet. It shall *not* contain spaces or dashes. Usually, keep it
   191   to a one-word string, or use underscores to separate words if you need.
   192   Avoid dots, commas, and special characters.
   193 
   194 CT_TARGET_ALIAS:
   195   An alias for the toolchian. It will be used as a prefix to the toolchain
   196   tools. For example, you will have ${CT_TARGET_ALIAS}-gcc
   197 
   198 ___________________
   199                   /
   200 Toolchain types  /
   201 ________________/
   202 
   203 There are four kinds of toolchains you could encounter.
   204 
   205 First off, you must understand the following: when it comes to compilers there
   206 are up to four machines involved:
   207   1) the machine configuring the toolchain components: the config machine
   208   2) the machine building the toolchain components:    the build machine
   209   3) the machine running the toolchain:                the host machine
   210   4) the machine the toolchain is building for:        the target machine
   211 
   212 We can most of the time assume that the config machine and the build machine
   213 are the same. Most of the time, this will be true. The only time it isn't
   214 is if you're using distributed compilation (such as distcc). Let's forget
   215 this for the sake of simplicity.
   216 
   217 So we're left with three machines:
   218  - build
   219  - host
   220  - target
   221 
   222 Any toolchain will involve those three machines. You can be as pretty sure of
   223 this as "2 and 2 are 4". Here is how they come into play:
   224 
   225 1) build == host == target
   226     This is a plain native toolchain, targetting the exact same machine as the
   227     one it is built on, and running again on this exact same machine. You have
   228     to build such a toolchain when you want to use an updated component, such
   229     as a newer gcc for example.
   230     ct-ng calls it "native".
   231 
   232 2) build == host != target
   233     This is a classic cross-toolchain, which is expected to be run on the same
   234     machine it is compiled on, and generate code to run on a second machine,
   235     the target.
   236     ct-ng calls it "cross".
   237 
   238 3) build != host == target
   239     Such a toolchain is also a native toolchain, as it targets the same machine
   240     as it runs on. But it is build on another machine. You want such a
   241     toolchain when porting to a new architecture, or if the build machine is
   242     much faster than the host machine.
   243     ct-ng calls it "cross-native".
   244 
   245 4) build != host != target
   246     This one is called a canadian-toolchain (*), and is tricky. The three
   247     machines in play are different. You might want such a toolchain if you
   248     have a fast build machine, but the users will use it on another machine,
   249     and will produce code to run on a third machine.
   250     ct-ng calls it "canadian".
   251 
   252 ct-ng can build all these kinds of toolchains (or is aiming at it, anyway!)
   253 
   254 (*) The term Canadian Cross came about because at the time that these issues
   255     were all being hashed out, Canada had three national political parties.
   256     http://en.wikipedia.org/wiki/Cross_compiler
   257 
   258 _____________
   259             /
   260 Internals  /
   261 __________/
   262 
   263 Internally, crosstool-NG is script-based. To ease usage, the frontend is
   264 Makefile-based.
   265 
   266 Makefile front-end |
   267 -------------------*
   268 
   269 The Makefile defines a set of rules to call each action. You can get the
   270 list, along with some terse description, by typing "make help" in your
   271 favourite command line.
   272 
   273 The Makefile sets the version variable from the version file in ${CT_TOP_DIR}
   274 which is then available to others in the CT_VERSION environment variable.
   275 
   276 The kconfig language is a hacked version, vampirised from the toybox project
   277 by Rob LANDLEY (http://www.landley.net/code/toybox/), adapted to my needs.
   278