Use CT_DoExecLog when building gcc.
/trunk/scripts/build/cc_gcc.sh | 55 27 28 0 ++++++++++++++++++++++++------------------------
1 file changed, 27 insertions(+), 28 deletions(-)
1 File.........: overview.txt
2 Content......: Overview of how crosstool-NG works.
3 Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>
4 License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
14 Installing crosstool-NG
18 Configuring crosstool-NG
19 Interesting config options
20 Re-building an existing toolchain
22 Stopping and restarting a build
23 Testing all toolchains at once
24 Overriding the number of // jobs
31 Adding a new version of a component
39 crosstool-NG aims at building toolchains. Toolchains are an essential component
40 in a software development project. It will compile, assemble and link the code
41 that is being developed. Some pieces of the toolchain will eventually end up
42 in the resulting binary/ies: static libraries are but an example.
44 So, a toolchain is a very sensitive piece of software, as any bug in one of the
45 components, or a poorly configured component, can lead to execution problems,
46 ranging from poor performance, to applications ending unexpectedly, to
47 mis-behaving software (which more than often is hard to detect), to hardware
48 damage, or even to human risks (which is more than regrettable).
50 Toolchains are made of different piece of software, each being quite complex
51 and requiring specially crafted options to build and work seamlessly. This
52 is usually not that easy, even in the not-so-trivial case of native toolchains.
53 The work reaches a higher degree of complexity when it comes to cross-
54 compilation, where it can become quite a nightmare...
56 Some cross-toolchains exist on the internet, and can be used for general
57 development, but they have a number of limitations:
58 - they can be general purpose, in that they are configured for the majority:
59 no optimisation for your specific target,
60 - they can be prepared for a specific target and thus are not easy to use,
61 nor optimised for, or even supporting your target,
62 - they often are using aging components (compiler, C library, etc...) not
63 supporting special features of your shiny new processor;
64 On the other side, these toolchain offer some advantages:
65 - they are ready to use and quite easy to install and setup,
66 - they are proven if used by a wide community.
68 But once you want to get all the juice out of your specific hardware, you will
69 want to build your own toolchain. This is where crosstool-NG comes into play.
71 There are also a number of tools that build toolchains for specific needs,
72 which are not really scalable. Examples are:
73 - buildroot (buildroot.uclibc.org) whose main purpose is to build root file
74 systems, hence the name. But once you have your toolchain with buildroot,
75 part of it is installed in the root-to-be, so if you want to build a whole
76 new root, you either have to save the existing one as a template and
77 restore it later, or restart again from scratch. This is not convenient,
78 - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very
80 - other projects (openembedded.org for example), which is again used to
81 build root file systems.
83 crosstool-NG is really targeted at building toolchains, and only toolchains.
84 It is then up to you to use it the way you want.
91 crosstool was first 'conceived' by Dan Kegel, who offered it to the community
92 as a set of scripts, a repository of patches, and some pre-configured, general
93 purpose setup files to be used to configure crosstool. This is available at
94 http://www.kegel.com/crosstool, and the subversion repository is hosted on
95 google at http://code.google.com/p/crosstool/.
97 I once managed to add support for uClibc-based toolchains, but it did not make
98 into mainline, mostly because I didn't have time to port the patch forward to
99 the new versions, due in part to the big effort it was taking.
101 So I decided to clean up crosstool in the state it was, re-order the things
102 in place, add appropriate support for what I needed, that is uClibc support
103 and a menu-driven configuration, named the new implementation crosstool-NG,
104 (standing for crosstool Next Generation, as many other comunity projects do,
105 and as a wink at the TV series "Star Trek: The Next Generation" ;-) ) and
106 made it available to the community, in case it was of interest to any one.
108 ___________________________
110 Installing crosstool-NG /
111 ________________________/
113 There are two ways you can use crosstool-NG:
114 - build and install it, then get rid of the sources like you'd do for most
116 - or only build it and run from the source directory.
118 The former should be used if you got crosstool-NG from a packaged tarball, see
119 "Install method", below, while the latter is most useful for developpers that
120 checked the code out from SVN, and want to submit patches, see "The Hacker's
126 If you go for the install, then you just follow the classical, but yet easy
128 ./configure --prefix=/some/place
131 export PATH="${PATH}:/some/place/bin"
133 You can then get rid of crosstool-NG source. Next create a directory to serve
134 as a working place, cd in there and run:
137 See below for complete usage.
142 If you go the hacker's way, then the usage is a bit different, although very
147 Now, *do not* remove crosstool-NG sources. They are needed to run crosstool-NG!
148 Stay in the directory holding the sources, and run:
151 See below for complete usage.
153 Now, provided you checked-out the code, you can send me your interesting changes
157 and mailing me the result! :-P
162 Some people contibuted code that couldn't get merged for various reasons. This
163 code is available as patches in the contrib/ sub-directory. These patches are
164 to be applied to the source of crosstool-NG, prior to installing.
166 An easy way to use contributed code is to pass the --with-contrib= option to
167 ./configure. The possible values depend upon which contributions are packaged
168 with your version, but you can get with it with passing one of those two
171 will list all available contributions
174 will select all avalaible contributions
176 There is no guarantee that a particuliar contribution applies to the current
177 version of crosstool-ng, or that it will work at all. Use contributions at
180 ____________________________
182 Configuring crosstool-NG /
183 _________________________/
185 crosstool-NG is configured with a configurator presenting a menu-stuctured set
186 of options. These options let you specify the way you want your toolchain
187 built, where you want it installed, what architecture and specific processor it
188 will support, the version of the components you want to use, etc... The
189 value for those options are then stored in a configuration file.
191 The configurator works the same way you configure your Linux kernel. It is
192 assumed you now how to handle this.
194 To enter the menu, type:
197 Almost every config item has a help entry. Read them carefully.
199 String and number options can refer to environment variables. In such a case,
200 you must use the shell syntax: ${VAR}. You shall neither single- nor double-
201 quote the string/number options.
203 There are three environment variables that are computed by crosstool-NG, and
207 It represents the target tuple you are building for. You can use it for
208 example in the installation/prefix directory, such as:
209 /opt/x-tools/${CT_TARGET}
212 The top directory where crosstool-NG is running. You shouldn't need it in
213 most cases. There is one case where you may need it: if you have local
214 patches and you store them in your running directory, you can refer to them
215 by using CT_TOP_DIR, such as:
216 ${CT_TOP_DIR}/patches.myproject
219 The version of crosstool-NG you are using. Not much use for you, but it's
220 there if you need it.
222 Interesting config options |
223 ---------------------------+
225 CT_LOCAL_TARBALLS_DIR:
226 If you already have some tarballs in a direcotry, enter it here. That will
227 speed up the retrieving phase, where crosstool-NG would otherwise download
231 This is where the toolchain will be installed in (and for now, where it
232 will run from). Common use is to add the target tuple in the directory
233 path, such as (see above):
234 /opt/x-tools/${CT_TARGET}
237 An identifier for your toolchain, will take place in the vendor part of the
238 target tuple. It shall *not* contain spaces or dashes. Usually, keep it
239 to a one-word string, or use underscores to separate words if you need.
240 Avoid dots, commas, and special characters.
243 An alias for the toolchian. It will be used as a prefix to the toolchain
244 tools. For example, you will have ${CT_TARGET_ALIAS}-gcc
246 Also, if you think you don't see enough versions, you can try to enable one of
250 Show obsolete versions or tools. Most of the time, you don't want to base
251 your toolchain on too old a version (of gcc, for example). But at times, it
252 can come handy to use such an old version for regression tests. Those old
253 versions are hidden behind CT_OBSOLETE.
256 Show experimental versions or tools. Again, you might not want to base your
257 toolchain on too recent tools (eg. gcc) for production. But if you need a
258 feature present only in a recent version, or a new tool, you can find them
259 hidden behind CT_EXPERIMENTAL.
262 Show broken versions or tools. Some usefull tools are currently broken: they
263 won't compile, run, or worse, cause defects when running. But if you are
264 brave enough, you can try and debug them. They are hidden behind CT_BROKEN,
265 which itself is hidden behind CT_EXPERIMENTAL.
267 Re-building an existing toolchain |
268 ----------------------------------+
270 If you have an existing toolchain, you can re-use the options used to build it
271 to create a new toolchain. That needs a very little bit of effort on your side
272 but is quite easy. The options to build a toolchain are saved in the build log
273 file that is saved within the toolchain. crosstool-NG can extract those options
274 to recreate a new configuration:
275 ct-ng extractconfig </path/to/your/build.log >.config
277 will extract those options, prompt you for the new ones, which you can later
278 edit with menuconfig.
280 Of course, if your build log was compressed, you'd have to use something like:
281 bzcat /path/to/your/build.log.bz2 |ct-ng extractconfig >.config
283 Then, once the configuration has been extracted, run:
286 ________________________
288 Running crosstool-NG /
289 _____________________/
291 To build the toolchain, simply type:
294 This will use the above configuration to retrieve, extract and patch the
295 components, build, install and eventually test your newly built toolchain.
297 You are then free to add the toolchain /bin directory in your PATH to use
300 In any case, you can get some terse help. Just type:
305 Stopping and restarting a build |
306 --------------------------------+
308 If you want to stop the build after a step you are debugging, you can pass the
309 variable STOP to make:
312 Conversely, if you want to restart a build at a specific step you are
313 debugging, you can pass the RESTART variable to make:
314 ct-ng RESTART=some_step
316 Alternatively, you can call make with the name of a step to just do that step:
319 ct-ng RESTART=libc_headers STOP=libc_headers
321 The shortcuts +step_name and step_name+ allow to respectively stop or restart
323 ct-ng +libc_headers and: ct-ng libc_headers+
325 ct-ng STOP=libc_headers and: ct-ng RESTART=libc_headers
327 To obtain the list of acceptable steps, please call:
330 Note that in order to restart a build, you'll have to say 'Y' to the config
331 option CT_DEBUG_CT_SAVE_STEPS, and that the previous build effectively went
334 Testing all toolchains at once |
335 -------------------------------+
337 You can test-build all samples; simply call:
340 Overriding the number of // jobs |
341 ---------------------------------+
343 If you want to override the number of jobs to run in // (the -j option to
344 make), you can either re-enter the menuconfig, or simply add it on the command
348 which tells crosstool-NG to override the number of // jobs to 4.
350 You can see the actions that support overriding the number of // jobs in
351 the help menu. Those are the ones with [.#] after them (eg. build[.#] or
352 regtest[.#], and so on...).
354 _______________________
356 Using the toolchain /
357 ____________________/
359 Using the toolchain is as simple as adding the toolchain's bin directory in
361 export PATH="${PATH}:/your/toolchain/path/bin"
363 and then using the target tuple to tell the build systems to use your
365 ./configure --target=your-target-tuple
367 make CC=your-target-tuple-gcc
369 make CROSS_COMPILE=your-target-tuple-
372 It is strongly advised not to use the toolchain sys-root directory as an
373 install directory for your programs/packages. If you do so, you will not be
374 able to use your toolchain for another project. It is even strongly advised
375 that your toolchain is chmod-ed to read-only once successfully build, so that
376 you don't go polluting your toolchain with your programs/packages' files.
378 Thus, when you build a program/package, install it in a separate directory,
379 eg. /your/root. This directory is the /image/ of what would be in the root file
380 system of your target, and will contain all that your programs/packages have
383 When your root directory is ready, it is still missing some important bits: the
384 toolchain's libraries. To populate your root directory with those libs, just
386 your-target-tuple-populate -s /your/root -d /your/root-populated
388 This will copy /your/root into /your/root-populated, and put the needed and only
389 the needed libraries there. Thus you don't polute /your/root with any cruft that
390 would no longer be needed should you have to remove stuff. /your/root always
391 contains only those things you install in it.
393 You can then use /your/root-populated to build up your file system image, a
394 tarball, or to NFS-mount it from your target, or whatever you need.
396 populate accepts the following options:
399 Use 'src_dir' as the 'source', un-populated root directory
402 Put the 'destination', populated root directory in 'dst_dir'
405 Remove 'dst_dir' if it previously existed
408 Be verbose, and tell what's going on (you can see exactly where libs are
419 There are four kinds of toolchains you could encounter.
421 First off, you must understand the following: when it comes to compilers there
422 are up to four machines involved:
423 1) the machine configuring the toolchain components: the config machine
424 2) the machine building the toolchain components: the build machine
425 3) the machine running the toolchain: the host machine
426 4) the machine the toolchain is generating code for: the target machine
428 We can most of the time assume that the config machine and the build machine
429 are the same. Most of the time, this will be true. The only time it isn't
430 is if you're using distributed compilation (such as distcc). Let's forget
431 this for the sake of simplicity.
433 So we're left with three machines:
438 Any toolchain will involve those three machines. You can be as pretty sure of
439 this as "2 and 2 are 4". Here is how they come into play:
441 1) build == host == target
442 This is a plain native toolchain, targetting the exact same machine as the
443 one it is built on, and running again on this exact same machine. You have
444 to build such a toolchain when you want to use an updated component, such
445 as a newer gcc for example.
446 crosstool-NG calls it "native".
448 2) build == host != target
449 This is a classic cross-toolchain, which is expected to be run on the same
450 machine it is compiled on, and generate code to run on a second machine,
452 crosstool-NG calls it "cross".
454 3) build != host == target
455 Such a toolchain is also a native toolchain, as it targets the same machine
456 as it runs on. But it is build on another machine. You want such a
457 toolchain when porting to a new architecture, or if the build machine is
458 much faster than the host machine.
459 crosstool-NG calls it "cross-native".
461 4) build != host != target
462 This one is called a canadian-toolchain (*), and is tricky. The three
463 machines in play are different. You might want such a toolchain if you
464 have a fast build machine, but the users will use it on another machine,
465 and will produce code to run on a third machine.
466 crosstool-NG calls it "canadian".
468 crosstool-NG can build all these kinds of toolchains (or is aiming at it,
471 (*) The term Canadian Cross came about because at the time that these issues
472 were all being hashed out, Canada had three national political parties.
473 http://en.wikipedia.org/wiki/Cross_compiler
480 Internally, crosstool-NG is script-based. To ease usage, the frontend is
486 The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
487 script with an action will act exactly as if the Makefile was in the current
488 working directory and make was called with the action as rule. Thus:
491 is equivalent to having the Makefile in CWD, and calling:
494 Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
497 ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
498 at configuration time with ./configure.
500 ct-ng also searches for config files, sub-tools, samples, scripts and patches in
501 that library directory.
503 Because of a stupid make behavior/bug I was unable to track down, implicit make
504 rules are disabled: installing with --local would triger those rules, and mconf
510 The kconfig language is a hacked version, vampirised from the toybox project
511 by Rob LANDLEY (http://www.landley.net/code/toybox/), itself coming from the
512 Linux kernel (http://www.kernel.org/), and (heavily) adapted to my needs.
514 The kconfig parsers (conf and mconf) are not installed pre-built, but as
515 source files. Thus you can have the directory where crosstool-NG is installed,
516 exported (via NFS or whatever) and have clients with different architectures
517 use the same crosstool-NG installation, and most notably, the same set of
520 Architecture-specific |
521 ----------------------+
523 Note: this chapter is not really well written, and might thus be a little bit
524 complex to understand. To get a better grasp of what an architecture is, the
525 reader is kindly encouraged to look at the "arch/" sub-directory, and to the
526 existing architectures to see how things are laid out.
528 An architecture is defined by:
530 - a human-readable name, in lower case letters, with numbers as appropriate.
531 The underscore is allowed; space and special characters are not.
533 - a directory in "arch/" named after the architecture, with the same letters
534 as above. Eg.: arch/arm, arch/x86_64
535 This directory contains the following files, and only those files:
536 - a configuration file in kconfig syntax, named "config.in", which shall
537 follow the API defined below.
538 Eg.: arch/arm/config.in
539 - a function script in bash-3.0 syntax, named "functions", which shall
540 follow the API defined below.
541 Eg.: arch/arm/functions
542 - an optional file, named "experimental" (in lower case!), which, if it
543 is present, means that support for this architecture is EXPERIMENTAL.
544 Eg.: arch/arm/experimental
546 The "config.in" file API:
547 > the config option "ARCH_%arch%" (where %arch% is to be replaced with the
548 actual architecture name).
549 That config option must have *neither* a type, *nor* a prompt! Also, it can
550 *not* depend on any other config option (EXPERIMENTAL is managed as above).
554 defines an (terse) help entry for this architecture:
558 The ARM architecture.
560 selects adequate associated config options.
563 select ARCH_SUPPORTS_BOTH_ENDIAN
564 select ARCH_DEFAULT_LE
566 The ARM architecture.
568 > other target-specific options, at your discretion. Note however that to
569 avoid name-clashing, such options shall be prefixed with "ARCH_%arch%",
570 where %arch% is again replaced by the actual architecture name.
571 (Note: due to historical reasons, and lack of time to clean up the code,
572 I may have left some config options that do not completely conform to
573 this, as the architecture name was written all upper case. However, the
574 prefix is unique among architectures, and does not cause harm).
576 The "functions" file API:
577 > the function "CT_DoArchValues"
580 - all variables from the ".config" file,
581 - the two variables "target_endian_eb" and "target_endian_el" which are
582 the endianness suffixes
583 + return value: 0 upon success, !0 upon failure
586 - the environment variable CT_TARGET_ARCH
588 the architecture part of the target tuple.
589 Eg.: "armeb" for big endian ARM
593 - the environment variable CT_TARGET_SYS
595 the sytem part of the target tuple.
596 Eg.: "gnu" for glibc on most architectures
597 "gnueabi" for glibc on an ARM EABI
599 - for glibc-based toolchain: "gnu"
600 - for uClibc-based toolchain: "uclibc"
603 - the environment variable CT_KERNEL_ARCH
605 the architecture name as understandable by the Linux kernel build
607 Eg.: "arm" for an ARM
608 "powerpc" for a PowerPC
614 - the environment variables to configure the cross-gcc
621 - contain (defaults):
622 - CT_ARCH_WITH_ARCH : the gcc ./configure switch to select architecture level ( "--with-arch=${CT_ARCH_ARCH}" )
623 - CT_ARCH_WITH_ABI : the gcc ./configure switch to select ABI level ( "--with-abi=${CT_ARCH_ABI}" )
624 - CT_ARCH_WITH_CPU : the gcc ./configure switch to select CPU instruction set ( "--with-cpu=${CT_ARCH_CPU}" )
625 - CT_ARCH_WITH_TUNE : the gcc ./configure switch to select scheduling ( "--with-tune=${CT_ARCH_TUNE}" )
626 - CT_ARCH_WITH_FPU : the gcc ./configure switch to select FPU type ( "--with-fpu=${CT_ARCH_FPU}" )
627 - CT_ARCH_WITH_FLOAT : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/ )
630 - the environment variables to pass to the cross-gcc to build target binaries
636 - CT_ARCH_FLOAT_CFLAG
637 - CT_ARCH_ENDIAN_CFLAG
638 - contain (defaults):
639 - CT_ARCH_ARCH_CFLAG : the gcc switch to select architecture level ( "-march=${CT_ARCH_ARCH}" )
640 - CT_ARCH_ABI_CFLAG : the gcc switch to select ABI level ( "-mabi=${CT_ARCH_ABI}" )
641 - CT_ARCH_CPU_CFLAG : the gcc switch to select CPU instruction set ( "-mcpu=${CT_ARCH_CPU}" )
642 - CT_ARCH_TUNE_CFLAG : the gcc switch to select scheduling ( "-mtune=${CT_ARCH_TUNE}" )
643 - CT_ARCH_FPU_CFLAG : the gcc switch to select FPU type ( "-mfpu=${CT_ARCH_FPU}" )
644 - CT_ARCH_FLOAT_CFLAG : the gcc switch to choose floating point arithmetics ( "-msoft-float" or /empty/ )
645 - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian ( "-mbig-endian" or "-mlittle-endian" )
649 Adding a new version of a component |
650 ------------------------------------+
652 When a new component, such as the Linux kernel, gcc or any other is released,
653 adding the new version to crosstool-NG is quite easy. There is a script that
654 will do all that for you:
655 tools/addToolVersion.sh
657 Run it with no option to get some help.
662 To Be Written later...