File.........: overview.txt Content......: Overview of how crosstool-NG works. Copyrigth....: (C) 2007 Yann E. MORIN License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5 ________________ / Introduction / _____________/ crosstool-NG aims at building toolchains. Toolchains are an essential component in a software development project. It will compile, assemble and link the code that is being developed. Some pieces of the toolchain will eventually end up in the resulting binary/ies: static libraries are but an example. So, a toolchain is a very sensitive piece of software, as any bug in one of the components, or a poorly configured component, can lead to execution problems, ranging from poor performance, to applications ending unexpectedly, to mis-behaving software (which more than often is hard to detect), to hardware damage, or even to human risks (which is more than regrettable). Toolchains are made of different piece of software, each being quite complex and requiring specially crafted options to build and work seamlessly. This is usually not that easy, even in the not-so-trivial case of native toolchains. The work reaches a higher degree of complexity when it comes to cross- compilation, where it can become quite a nightmare... Some cross-toolchains exist on the internet, and can be used for general development, but they have a number of limitations: - they can be general purpose, in that they are configured for the majority: no optimisation for your specific target, - they can be prepared for a specific target and thus are not easy to use, nor optimised for, or even supporting your target, - they often are using aging components (compiler, C library, etc...) not supporting special features of your shiny new processor; On the other side, these toolchain offer some advantages: - they are ready to use and quite easy to install and setup, - they are proven if used by a wide community. But once you want to get all the juice out of your specific hardware, you will want to build your own toolchain. This is where crosstool-NG comes into play. There are also a number of tools that build toolchains for specific needs, which are not really scalable. Examples are: - buildroot (buildroot.uclibc.org) whose main purpose is to build root file systems, hence the name. But once you have your toolchain with buildroot, part of it is installed in the root-to-be, so if you want to build a whole new root, you either have to save the existing one as a template and restore it later, or restart again from scratch. This is not convenient, - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very similar to buildroot, - other projects (openembedded.org for example), which is again used to build root file systems. crosstool-NG is really targeted at building toolchains, and only toolchains. It is then up to you to use it the way you want. ___________ / History / ________/ crosstool was first 'conceived' by Dan Kegel, who offered it to the community as a set of scripts, a repository of patches, and some pre-configured, general purpose setup files to be used to configure crosstool. This is available at http://www.kegel.com/crosstool, and the subversion repository is hosted on google at http://code.google.com/p/crosstool/. I once managed to add support for uClibc-based toolchains, but it did not make into mainline, mostly because I didn't have time to port the patch forward to the new versions, due in part to the big effort it was taking. So I decided to clean up crosstool in the state it was, re-order the things in place, add appropriate support for what I needed, that is uClibc support and a menu-driven configuration, named the new implementation crosstool-NG, (standing for crosstool Next Generation, as many other comunity projects do, and as a wink at the TV series "Star Trek: The Next Generation" ;-) ) and made it available to the community, in case it was of interest to any one. ___________________________ / Installing crosstool-NG / ________________________/ There are two ways you can use crosstool-NG: - build and install it, then get rid of the sources like you'd do for most programs, - or only build it and run from the source directory. The former should be used if you got crosstool-NG from a packaged tarball, see "Install method", below, while the latter is most useful for developpers that checked the code out from SVN, and want to submit patches, see "The Hacker's way", below. Install method | ---------------+ If you go for the install, then you just follow the classical, but yet easy ./configure way: ./configure --prefix=/some/place make make install export PATH="${PATH}:/some/place/bin" You can then get rid of crosstool-NG source. Next create a directory to serve as a working place, cd in there and run: ct-ng help See below for complete usage. The Hacker's way | -----------------+ If you go the hacker's way, then the usage is a bit different, although very simple: ./configure --local make Now, *do not* remove crosstool-NG sources. They are needed to run crosstool-NG! Stay in the directory holding the sources, and run: ./ct-ng help See below for complete usage. Now, provided you checked-out the code, you can send me your interesting changes by running: svn diff and mailing me the result! :-P Contributed code | -----------------+ Some people contibuted code that couldn't get merged for various reasons. This code is available as patches in the contrib/ sub-directory. These patches are to be applied to the source of crosstool-NG, prior to installing. ____________________________ / Configuring crosstool-NG / _________________________/ crosstool-NG is configured by a configurator presenting a menu-stuctured set of options. These options let you specify the way you want your toolchain built, where you want it installed, what architecture and specific processor it will support, the version of the components you want to use, etc... The value for those options are then stored in a configuration file. The configurator works the same way you configure your Linux kernel. It is assumed you now how to handle this. To enter the menu, type: ct-ng menuconfig Almost every config item has a help entry. Read them carefully. String and number options can refer to environment variables. In such a case, you must use the shell syntax: ${VAR}. You shall neither single- nor double- quote the string/number options. There are three environment variables that are computed by crosstool-NG, and that you can use: CT_TARGET: It represents the target tuple you are building for. You can use it for example in the installation/prefix directory, such as: /opt/x-tools/${CT_TARGET} CT_TOP_DIR: The top directory where crosstool-NG is running. You shouldn't need it in most cases. There is one case where you may need it: if you have local patches and you store them in your running directory, you can refer to them by using CT_TOP_DIR, such as: ${CT_TOP_DIR}/patches.myproject CT_VERSION: The version of crosstool-NG you are using. Not much use for you, but it's there if you need it. Interesting config options | ---------------------------+ CT_LOCAL_TARBALLS_DIR: If you already have some tarballs in a direcotry, enter it here. That will speed up the retrieving phase, where crosstool-NG would otherwise download those tarballs. CT_PREFIX_DIR: This is where the toolchain will be installed in (and for now, where it will run from). Common use is to add the target tuple in the directory path, such as (see above): /opt/x-tools/${CT_TARGET} CT_TARGET_VENDOR: An identifier for your toolchain, will take place in the vendor part of the target tuple. It shall *not* contain spaces or dashes. Usually, keep it to a one-word string, or use underscores to separate words if you need. Avoid dots, commas, and special characters. CT_TARGET_ALIAS: An alias for the toolchian. It will be used as a prefix to the toolchain tools. For example, you will have ${CT_TARGET_ALIAS}-gcc Also, if you think you don't see enough versions, you can try to enable one of those: CT_OBSOLETE: Show obsolete versions or tools. Most of the time, you don't want to base your toolchain on too old a version (of gcc, for example). But at times, it can come handy to use such an old version for regression tests. Those old versions are hidden behind CT_OBSOLETE. CT_EXPERIMENTAL: Show experimental versions or tools. Again, you might not want to base your toolchain on too recent tools (eg. gcc) for production. But if you need a feature present only in a recent version, or a new tool, you can find them hidden behind CT_EXPERIMENTAL. CT_BROKEN: Show broken versions or tools. Some usefull tools are currently broken: they won't compile, run, or worse, cause defects when running. But if you are brave enough, you can try and debug them. They are hidden behind CT_BROKEN, which itself is hidden behind CT_EXPERIMENTAL. Re-building an existing toolchain | ----------------------------------+ If you have an existing toolchain, you can re-use the options used to build it to create a new toolchain. That needs a very little bit of effort on your side but is quite easy. The options to build a toolchain are saved in the build log file that is saved within the toolchain. crosstool-NG can extract those options to recreate a new configuration: ct-ng extractconfig the function "CT_DoArchValues" + parameters: none + environment: - all variables from the ".config" file, - the two variables "target_endian_eb" and "target_endian_el" which are the endianness suffixes + return value: 0 upon success, !0 upon failure + provides: - mandatory - the environment variable CT_TARGET_ARCH - contains: the architecture part of the target tuple. Eg.: "armeb" for big endian ARM "i386" for an i386 + provides: - optional - the environment variable CT_TARGET_SYS - contains: the sytem part of the target tuple. Eg.: "gnu" for glibc on most architectures "gnueabi" for glibc on an ARM EABI - defaults to: - for glibc-based toolchain: "gnu" - for uClibc-based toolchain: "uclibc" + provides: - optional - the environment variable CT_KERNEL_ARCH - contains: the architecture name as understandable by the Linux kernel build system. Eg.: "arm" for an ARM "powerpc" for a PowerPC "i386" for an x86 - defaults to: ${CT_ARCH} + provides: - optional - the environment variables to configure the cross-gcc - CT_ARCH_WITH_ARCH - CT_ARCH_WITH_ABI - CT_ARCH_WITH_CPU - CT_ARCH_WITH_TUNE - CT_ARCH_WITH_FPU - CT_ARCH_WITH_FLOAT - contain (defaults): - CT_ARCH_WITH_ARCH : the gcc ./configure switch to select architecture level ( "--with-arch=${CT_ARCH_ARCH}" ) - CT_ARCH_WITH_ABI : the gcc ./configure switch to select ABI level ( "--with-abi=${CT_ARCH_ABI}" ) - CT_ARCH_WITH_CPU : the gcc ./configure switch to select CPU instruction set ( "--with-cpu=${CT_ARCH_CPU}" ) - CT_ARCH_WITH_TUNE : the gcc ./configure switch to select scheduling ( "--with-tune=${CT_ARCH_TUNE}" ) - CT_ARCH_WITH_FPU : the gcc ./configure switch to select FPU type ( "--with-fpu=${CT_ARCH_FPU}" ) - CT_ARCH_WITH_FLOAT : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/ ) + provides: - optional - the environment variables to pass to the cross-gcc to build target binaries - CT_ARCH_ARCH_CFLAG - CT_ARCH_ABI_CFLAG - CT_ARCH_CPU_CFLAG - CT_ARCH_TUNE_CFLAG - CT_ARCH_FPU_CFLAG - CT_ARCH_FLOAT_CFLAG - CT_ARCH_ENDIAN_CFLAG - contain (defaults): - CT_ARCH_ARCH_CFLAG : the gcc switch to select architecture level ( "-march=${CT_ARCH_ARCH}" ) - CT_ARCH_ABI_CFLAG : the gcc switch to select ABI level ( "-mabi=${CT_ARCH_ABI}" ) - CT_ARCH_CPU_CFLAG : the gcc switch to select CPU instruction set ( "-mcpu=${CT_ARCH_CPU}" ) - CT_ARCH_TUNE_CFLAG : the gcc switch to select scheduling ( "-mtune=${CT_ARCH_TUNE}" ) - CT_ARCH_FPU_CFLAG : the gcc switch to select FPU type ( "-mfpu=${CT_ARCH_FPU}" ) - CT_ARCH_FLOAT_CFLAG : the gcc switch to choose floating point arithmetics ( "-msoft-float" or /empty/ ) - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian ( "-mbig-endian" or "-mlittle-endian" ) - default to: see above. Adding a new version of a conponent | ------------------------------------+ When a new component, such as the Linux kernel, gcc or any other is released, adding the new version to crosstool-NG is quite easy. There is a script that will do all that for you: tools/addToolVersion.sh Run it with no option to get some help. Build scripts | --------------+ To Be Written later...