File.........: overview.txt

Content......: Overview of how ct-ng works.

Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>

License......: see COPYING in the root of this package

________________

               /

Introduction  /

_____________/

crosstool-NG aims at building toolchains. Toolchains are an essential component

in a software development project. It will compile, assemble and link the code

that is being developped. Some pieces of the toolchain will eventually end up

in the resulting binary/ies: static libraries are but an example.

So, a toolchain is a very sensitive piece of software, as any bug in one of the

components, or a poorly configured component, can lead to execution problems,

ranging from poor performance, to applications ending unexpectedly, to

mis-behaving software (which more than often is hard to detect), to hardware

damage, or even to human risks (which is more than regretable).

Toolchains are made of different piece of software, each being quite complex

and requiring specially crafted options to build and work seamlessly. This

is usually not that easy, even in the not-so-trivial case of native toolchains.

The work reaches a higher degree of complexity when it comes to cross-

compilation, where it can become quite a nightmare...

Some cross-toolchains exist on the internet, and can be used for general

development, but they have a number of limitations:

  - they can be general purpose, in that they are configured for the majority:

    no optimisation for your specific target,

  - they can be prepared for a specific target and thus are not easy to use,

    nor optimised for, or even supporting your target,

  - they often are using ageing components (compiler, C library, etc...) not

    supporting special features of your shiny new processor;

On the other side, these toolchain offer some advantages:

  - they are ready to use and quite easy to install and setup,

  - they are proven if used by a wide community.

But once you want to get all the juice out of your specific hardware, you will

want to build your own toolchain. This is where crosstool-ng comes into play.

There are also a number of tools that builds toolchains for specific needs,

which is not really scalable. Examples are:

  - buildroot (buildroot.uclibc.org) whose main puprpose is to build root file

    systems, hence the name. But once you have your toolchain with buildroot,

    part of it is installed in the root-to-be, so if you want to build a whole

    new root, you either have to save the existing one as a template and

    restore it later, or restart again from scratch. This is not convenient,

  - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very

    similar to buildroot,

  - other projects (openembeded.org for example), which is again used to

    build root file systems.

crosstool-NG is really targetted at building toolchains, and only toolchains.

It is then up to you to use it the way you want.

___________

          /

History  /

________/

crosstool was first 'conceived' by Dan Kegel, which offered it to the community,

as a set of scripts, a repository of patches, and some pre-configured, general

purpose setup files to be used to configure crosstool. This is available at

www.kegel.com/crosstool, and the subversion repository is hosted on google at

http://code.google.com/p/crosstool/.

At the time of writing, crosstool only supports building with one C library,

namely glibc, and one C compiler, gcc; it is cripled with historical support

for legacy components, and is some kind of a mess to upgrade. Also, submited

patches take a looong time before they are integrated mainline.

I once managed to add support for uClibc-based toolchains, but it did not make

into mainline, mostly because I don't have time to port the patch forward to

the new versions, due in part to the big effort it was taking.

So I decided to clean up crosstool in the state it was, re-order the things

in place, and add appropriate support for what I needed, that is uClibc

support.

The only option left to me was rewrite crosstool from scratch. I decided to go

this way, and name the new implementation ct-ng, standing for crosstool Next

Generation, as many other comunity projects do, and as a wink at the TV series

"Star Trek: The Next Generation". ;-)

_____________

            /

Operation  /

__________/

ct-ng is configured by a configurator presenting a menu-stuctured set of

options. These options let you specify the way you want your toolchain built,

where you want it installed, what architecture and specific processor it

will support, the version of the components you want to use, etc... The

value for those options are then stored in a configuration file.

You then simply run make. It will use this configuration file to retrieve,

extract and patch the components, build, install and test your newly built

toolchain.

You are then free to add the toolchain /bin directory in your PATH to use

it at will.

____________________________

                           /

Configuring crosstool-NG  /

_________________________/

crosstool-NG is configured the same way you configure your Linux kernel: by

using a curses-based menu. It is assumed you now how to handle this.

Almost every config item has a help entry. Read it carefully.

String and number options can refer to environment variables. In such a case,

you  must use the shell syntax: ${VAR}. No such option is ever needed by make.

You need to neither single- nor double-quote the string options.

There are three environment variablea that are computed by crosstool-NG, and

that you can use:

CT_TARGET:

  It represents the target triplet you are building for. You can use it for

  example in the installation/prefix directory, such as:

    /opt/x-tools/${CT_TARGET}

CT_TOP_DIR:

  The top directory where crosstool-NG sits. You shouldn't need it in most

  cases. There is one case where you may need it: if you have local patches

  and you store them in your copy of crosstool-NG, you can refer to them

  by using CT_TOP_DIR, such as:

    ${CT_TOP_DIR}/patches.myproject

CT_VERSION:

  The version of crosstool-NG you are using. Not much help for you, but it's

  there if you need it.

Interesting config options |

---------------------------*

CT_LOCAL_TARBALLS_DIR:

  If you already have sone tarballs in a direcotry, enter it here. That will

  speed up the retrieving phase, where crosstool-ng would otherwise download

  those tarballs.

CT_PREFIX_DIR:

  This is where the toolchain will be installed in (and for now, where it

  will run from).

CT_LOG_FILE:

  The file where *all* log messages will go. Keep the default, in goes in

  ${CT_PREFIX_DIR}/${CT_TARGET}.log

CT_TARGET_VENDOR:

  An identifier for your toolchain, will take place in the vendor part of the

  target triplet. It shall *not* contain spaces or dashes. Usually, keep it

  to a one-word string, or use underscores to separate words if you need.

  Avoid dots, commas, and special characters.

CT_TARGET_ALIAS:

  An alias for the toolchian. It will be used as a prefix to the toolchain

  tools. For example, you will have ${CT_TARGET_ALIAS}-gcc

___________________

                  /

Toolchain types  /

________________/

There are four kinds of toolchains you could encounter.

First off, you must understand the following: when it comes to compilers there

are up to four machines involved:

  1) the machine configuring the toolchain components: the config machine

  2) the machine building the toolchain components:    the build machine

  3) the machine running the toolchain:                the host machine

  4) the machine the toolchain is building for:        the target machine

We can most of the time assume that the config machine and the build machine

are the same. Most of the time, this will be true. The only time it isn't

is if you're using distributed compilation (such as distcc). Let's forget

this for the sake of simplicity.

So we're left with three machines:

 - build

 - host

 - target

Any toolchain will involve those three machines. You can be as pretty sure of

this as "2 and 2 are 4". Here is how they come into play:

1) build == host == target

    This is a plain native toolchain, targetting the exact same machine as the

    one it is built on, and running again on this exact same machine. You have

    to build such a toolchain when you want to use an updated component, such

    as a newer gcc for example.

    ct-ng calls it "native".

2) build == host != target

    This is a classic cross-toolchain, which is expected to be run on the same

    machine it is compiled on, and generate code to run on a second machine,

    the target.

    ct-ng calls it "cross".

3) build != host == target

    Such a toolchain is also a native toolchain, as it targets the same machine

    as it runs on. But it is build on another machine. You want such a

    toolchain when porting to a new architecture, or if the build machine is

    much faster than the host machine.

    ct-ng calls it "cross-native".

4) build != host != target

    This one is called a canadian-toolchain (*), and is tricky. The three

    machines in play are different. You might want such a toolchain if you

    have a fast build machine, but the users will use it on another machine,

    and will produce code to run on a third machine.

    ct-ng calls it "canadian".

ct-ng can build all these kinds of toolchains (or is aiming at it, anyway!)

(*) The term Canadian Cross came about because at the time that these issues

    were all being hashed out, Canada had three national political parties.

    http://en.wikipedia.org/wiki/Cross_compiler

_____________

            /

Internals  /

__________/

Internally, crosstool-NG is script-based. To ease usage, the frontend is

Makefile-based.

Makefile front-end |

-------------------*

The Makefile defines a set of rules to call each action. You can get the

list, along with some terse description, by typing "make help" in your

favourite command line.

The Makefile sets the version variable from the version file in ${CT_TOP_DIR}

which is then available to others in the CT_VERSION environment variable.

The kconfig language is a hacked version, vampirised from the toybox project

by Rob LANDLEY (http://www.landley.net/code/toybox/), adapted to my needs.