yann@1: File.........: overview.txt yann@1: Content......: Overview of how ct-ng works. yann@92: Copyrigth....: (C) 2007 Yann E. MORIN yann@192: License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5 yann@92: yann@1: ________________ yann@1: / yann@1: Introduction / yann@1: _____________/ yann@1: yann@1: crosstool-NG aims at building toolchains. Toolchains are an essential component yann@1: in a software development project. It will compile, assemble and link the code yann@1: that is being developped. Some pieces of the toolchain will eventually end up yann@1: in the resulting binary/ies: static libraries are but an example. yann@1: yann@1: So, a toolchain is a very sensitive piece of software, as any bug in one of the yann@1: components, or a poorly configured component, can lead to execution problems, yann@1: ranging from poor performance, to applications ending unexpectedly, to yann@1: mis-behaving software (which more than often is hard to detect), to hardware yann@1: damage, or even to human risks (which is more than regretable). yann@1: yann@1: Toolchains are made of different piece of software, each being quite complex yann@1: and requiring specially crafted options to build and work seamlessly. This yann@1: is usually not that easy, even in the not-so-trivial case of native toolchains. yann@1: The work reaches a higher degree of complexity when it comes to cross- yann@40: compilation, where it can become quite a nightmare... yann@1: yann@40: Some cross-toolchains exist on the internet, and can be used for general yann@1: development, but they have a number of limitations: yann@1: - they can be general purpose, in that they are configured for the majority: yann@1: no optimisation for your specific target, yann@1: - they can be prepared for a specific target and thus are not easy to use, yann@1: nor optimised for, or even supporting your target, yann@1: - they often are using ageing components (compiler, C library, etc...) not yann@1: supporting special features of your shiny new processor; yann@1: On the other side, these toolchain offer some advantages: yann@1: - they are ready to use and quite easy to install and setup, yann@1: - they are proven if used by a wide community. yann@1: yann@1: But once you want to get all the juice out of your specific hardware, you will yann@1: want to build your own toolchain. This is where crosstool-ng comes into play. yann@1: yann@1: There are also a number of tools that builds toolchains for specific needs, yann@1: which is not really scalable. Examples are: yann@1: - buildroot (buildroot.uclibc.org) whose main puprpose is to build root file yann@1: systems, hence the name. But once you have your toolchain with buildroot, yann@1: part of it is installed in the root-to-be, so if you want to build a whole yann@1: new root, you either have to save the existing one as a template and yann@1: restore it later, or restart again from scratch. This is not convenient, yann@1: - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very yann@1: similar to buildroot, yann@1: - other projects (openembeded.org for example), which is again used to yann@1: build root file systems. yann@1: yann@1: crosstool-NG is really targetted at building toolchains, and only toolchains. yann@1: It is then up to you to use it the way you want. yann@1: yann@1: ___________ yann@1: / yann@1: History / yann@1: ________/ yann@1: yann@1: crosstool was first 'conceived' by Dan Kegel, which offered it to the community, yann@1: as a set of scripts, a repository of patches, and some pre-configured, general yann@1: purpose setup files to be used to configure crosstool. This is available at yann@1: www.kegel.com/crosstool, and the subversion repository is hosted on google at yann@1: http://code.google.com/p/crosstool/. yann@1: yann@1: At the time of writing, crosstool only supports building with one C library, yann@1: namely glibc, and one C compiler, gcc; it is cripled with historical support yann@92: for legacy components, and is some kind of a mess to upgrade. Also, submited yann@92: patches take a looong time before they are integrated mainline. yann@1: yann@1: I once managed to add support for uClibc-based toolchains, but it did not make yann@1: into mainline, mostly because I don't have time to port the patch forward to yann@1: the new versions, due in part to the big effort it was taking. yann@1: yann@1: So I decided to clean up crosstool in the state it was, re-order the things yann@1: in place, and add appropriate support for what I needed, that is uClibc yann@1: support. yann@1: yann@1: The only option left to me was rewrite crosstool from scratch. I decided to go yann@1: this way, and name the new implementation ct-ng, standing for crosstool Next yann@1: Generation, as many other comunity projects do, and as a wink at the TV series yann@1: "Star Trek: The Next Generation". ;-) yann@1: yann@168: ____________________________ yann@168: / yann@168: Configuring crosstool-NG / yann@168: _________________________/ yann@168: yann@168: crosstool-NG is configured the same way you configure your Linux kernel: by yann@168: using a curses-based menu. It is assumed you now how to handle this. yann@168: yann@168: To enter the menu, type: yann@192: ct-ng menuconfig yann@168: yann@168: Almost every config item has a help entry. Read it carefully. yann@168: yann@168: String and number options can refer to environment variables. In such a case, yann@192: you must use the shell syntax: ${VAR}. You shall neither single- nor double- yann@192: quote the string options. yann@168: yann@192: There are three environment variables that are computed by crosstool-NG, and yann@168: that you can use: yann@168: yann@168: CT_TARGET: yann@168: It represents the target triplet you are building for. You can use it for yann@168: example in the installation/prefix directory, such as: yann@168: /opt/x-tools/${CT_TARGET} yann@168: yann@168: CT_TOP_DIR: yann@182: The top directory where crosstool-NG is running. You shouldn't need it in yann@182: most cases. There is one case where you may need it: if you have local yann@182: patches and you store them in your running directory, you can refer to them yann@168: by using CT_TOP_DIR, such as: yann@168: ${CT_TOP_DIR}/patches.myproject yann@168: yann@168: CT_VERSION: yann@192: The version of crosstool-NG you are using. Not much use for you, but it's yann@168: there if you need it. yann@168: yann@168: yann@168: Interesting config options | yann@168: ---------------------------* yann@168: yann@168: CT_LOCAL_TARBALLS_DIR: yann@168: If you already have sone tarballs in a direcotry, enter it here. That will yann@168: speed up the retrieving phase, where crosstool-ng would otherwise download yann@168: those tarballs. yann@168: yann@168: CT_PREFIX_DIR: yann@168: This is where the toolchain will be installed in (and for now, where it yann@168: will run from). yann@168: yann@168: CT_TARGET_VENDOR: yann@168: An identifier for your toolchain, will take place in the vendor part of the yann@168: target triplet. It shall *not* contain spaces or dashes. Usually, keep it yann@168: to a one-word string, or use underscores to separate words if you need. yann@168: Avoid dots, commas, and special characters. yann@168: yann@168: CT_TARGET_ALIAS: yann@168: An alias for the toolchian. It will be used as a prefix to the toolchain yann@168: tools. For example, you will have ${CT_TARGET_ALIAS}-gcc yann@168: yann@168: ________________________ yann@168: / yann@168: Running crosstool-NG / yann@168: _____________________/ yann@1: yann@1: ct-ng is configured by a configurator presenting a menu-stuctured set of yann@1: options. These options let you specify the way you want your toolchain built, yann@1: where you want it installed, what architecture and specific processor it yann@1: will support, the version of the components you want to use, etc... The yann@1: value for those options are then stored in a configuration file. yann@1: yann@168: To build the toolchain, simply type: yann@192: ct-ng yann@135: yann@135: This will use the above configuration to retrieve, extract and patch the yann@135: components, build, install and eventually test your newly built toolchain. yann@1: yann@1: You are then free to add the toolchain /bin directory in your PATH to use yann@1: it at will. yann@1: yann@135: In any case, you can get some terse help. Just type: yann@192: ct-ng help yann@135: yann@135: yann@135: Stoping and restarting a build | yann@135: -------------------------------* yann@135: yann@135: If you want to stop the build after a step you are debugging, you can pass the yann@135: variable STOP to make: yann@192: ct-ng STOP=some_step yann@135: yann@135: Conversely, if you want to restart a build at a specific step you are yann@135: debugging, you can pass the RESTART variable to make: yann@192: ct-ng RESTART=some_step yann@135: yann@136: Alternatively, you can call make with the name of a step to just do that step: yann@192: ct-ng libc_headers yann@136: is equivalent to: yann@192: ct-ng RESTART=libs_headers STOP=libc_headers yann@136: yann@136: The shortcuts -step_name and step_name- allow to respectively stop or restart yann@136: at that step. Thus: yann@192: ct-ng -libc_headers and: ct-ng libc_headers- yann@136: are equivalent to: yann@192: ct-ng STOP=libc_headers and: ct-ng RESTART=libc_headers yann@136: yann@181: To obtain the list of acceptable steps, please call: yann@192: ct-ng liststeps yann@181: yann@168: Note that in order to restart a build, you'll have to say 'Y' to the config yann@168: option CT_DEBUG_CT_SAVE_STEPS, and that the previous build effectively went yann@168: that far. yann@92: yann@92: yann@168: Testing all toolchains at once | yann@168: -------------------------------* yann@92: yann@168: You can test-build all samples; simply call: yann@192: ct-ng regtest yann@40: yann@40: ___________________ yann@40: / yann@40: Toolchain types / yann@40: ________________/ yann@40: yann@40: There are four kinds of toolchains you could encounter. yann@40: yann@40: First off, you must understand the following: when it comes to compilers there yann@40: are up to four machines involved: yann@40: 1) the machine configuring the toolchain components: the config machine yann@40: 2) the machine building the toolchain components: the build machine yann@40: 3) the machine running the toolchain: the host machine yann@40: 4) the machine the toolchain is building for: the target machine yann@40: yann@40: We can most of the time assume that the config machine and the build machine yann@40: are the same. Most of the time, this will be true. The only time it isn't yann@40: is if you're using distributed compilation (such as distcc). Let's forget yann@40: this for the sake of simplicity. yann@40: yann@40: So we're left with three machines: yann@40: - build yann@40: - host yann@40: - target yann@40: yann@40: Any toolchain will involve those three machines. You can be as pretty sure of yann@40: this as "2 and 2 are 4". Here is how they come into play: yann@40: yann@40: 1) build == host == target yann@40: This is a plain native toolchain, targetting the exact same machine as the yann@40: one it is built on, and running again on this exact same machine. You have yann@40: to build such a toolchain when you want to use an updated component, such yann@40: as a newer gcc for example. yann@40: ct-ng calls it "native". yann@40: yann@40: 2) build == host != target yann@40: This is a classic cross-toolchain, which is expected to be run on the same yann@40: machine it is compiled on, and generate code to run on a second machine, yann@40: the target. yann@40: ct-ng calls it "cross". yann@40: yann@40: 3) build != host == target yann@40: Such a toolchain is also a native toolchain, as it targets the same machine yann@40: as it runs on. But it is build on another machine. You want such a yann@40: toolchain when porting to a new architecture, or if the build machine is yann@40: much faster than the host machine. yann@40: ct-ng calls it "cross-native". yann@40: yann@40: 4) build != host != target yann@92: This one is called a canadian-toolchain (*), and is tricky. The three yann@40: machines in play are different. You might want such a toolchain if you yann@40: have a fast build machine, but the users will use it on another machine, yann@40: and will produce code to run on a third machine. yann@40: ct-ng calls it "canadian". yann@40: yann@40: ct-ng can build all these kinds of toolchains (or is aiming at it, anyway!) yann@40: yann@40: (*) The term Canadian Cross came about because at the time that these issues yann@40: were all being hashed out, Canada had three national political parties. yann@40: http://en.wikipedia.org/wiki/Cross_compiler yann@40: yann@1: _____________ yann@1: / yann@1: Internals / yann@1: __________/ yann@1: yann@92: Internally, crosstool-NG is script-based. To ease usage, the frontend is yann@92: Makefile-based. yann@92: yann@92: Makefile front-end | yann@92: -------------------* yann@92: yann@182: To Be Written later... yann@92: yann@182: Kconfig parser | yann@182: ---------------* yann@92: yann@92: The kconfig language is a hacked version, vampirised from the toybox project yann@182: by Rob LANDLEY (http://www.landley.net/code/toybox/), itself coming from the yann@182: Linux kernel (http://www.linux.org/ http://www.kernel.org/), and (heavily) yann@182: adapted to my needs. yann@92: