docs/overview.txt
author "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Thu Oct 09 19:17:49 2008 +0000 (2008-10-09)
changeset 914 0b164a321177
parent 903 9fb0f81b4416
child 965 45ddf096def1
permissions -rw-r--r--
Remove CT_BROKEN.
Only one component is actually BROKEN (libelf), make it depend on EXPERIMENTAL, but state BROKEN in the prompt.
ltrace (which depends on libelf) is now marked as EXPERIMENTAL, with the reference to the BROKEN libelf in the help entry.

/trunk/docs/overview.txt | 6 0 6 0 ------
/trunk/config/debug/ltrace.in | 7 4 3 0 ++++---
/trunk/config/global/ct-behave.in | 8 0 8 0 --------
/trunk/config/tools/libelf.in | 5 3 2 0 +++--
4 files changed, 7 insertions(+), 19 deletions(-)
yann@1
     1
File.........: overview.txt
yann@197
     2
Content......: Overview of how crosstool-NG works.
yann@92
     3
Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>
yann@192
     4
License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
yann@92
     5
yann@628
     6
____________________
yann@628
     7
                   /
yann@628
     8
Table Of Content  /
yann@628
     9
_________________/
yann@628
    10
yann@628
    11
yann@628
    12
Introduction
yann@628
    13
History
yann@628
    14
Installing crosstool-NG
yann@628
    15
  Install method
yann@628
    16
  The hacker's way
yann@837
    17
  Shell completion
yann@628
    18
  Contributed code
yann@628
    19
Configuring crosstool-NG
yann@628
    20
  Interesting config options
yann@628
    21
  Re-building an existing toolchain
yann@628
    22
Running crosstool-NG
yann@628
    23
  Stopping and restarting a build
yann@628
    24
  Testing all toolchains at once
yann@628
    25
  Overriding the number of // jobs
yann@628
    26
Using the toolchain
yann@628
    27
Toolchain types
yann@628
    28
Internals
yann@628
    29
  Makefile front-end
yann@628
    30
  Kconfig parser
yann@628
    31
  Architecture-specific
yann@628
    32
  Adding a new version of a component
yann@628
    33
  Build scripts
yann@628
    34
yann@1
    35
________________
yann@1
    36
               /
yann@1
    37
Introduction  /
yann@1
    38
_____________/
yann@1
    39
yann@1
    40
crosstool-NG aims at building toolchains. Toolchains are an essential component
yann@1
    41
in a software development project. It will compile, assemble and link the code
rpjday@436
    42
that is being developed. Some pieces of the toolchain will eventually end up
yann@1
    43
in the resulting binary/ies: static libraries are but an example.
yann@1
    44
yann@1
    45
So, a toolchain is a very sensitive piece of software, as any bug in one of the
yann@1
    46
components, or a poorly configured component, can lead to execution problems,
yann@1
    47
ranging from poor performance, to applications ending unexpectedly, to
yann@1
    48
mis-behaving software (which more than often is hard to detect), to hardware
rpjday@436
    49
damage, or even to human risks (which is more than regrettable).
yann@1
    50
yann@1
    51
Toolchains are made of different piece of software, each being quite complex
yann@1
    52
and requiring specially crafted options to build and work seamlessly. This
yann@1
    53
is usually not that easy, even in the not-so-trivial case of native toolchains.
yann@1
    54
The work reaches a higher degree of complexity when it comes to cross-
yann@40
    55
compilation, where it can become quite a nightmare...
yann@1
    56
yann@40
    57
Some cross-toolchains exist on the internet, and can be used for general
yann@1
    58
development, but they have a number of limitations:
yann@1
    59
  - they can be general purpose, in that they are configured for the majority:
yann@1
    60
    no optimisation for your specific target,
yann@1
    61
  - they can be prepared for a specific target and thus are not easy to use,
yann@1
    62
    nor optimised for, or even supporting your target,
rpjday@436
    63
  - they often are using aging components (compiler, C library, etc...) not
yann@1
    64
    supporting special features of your shiny new processor;
yann@1
    65
On the other side, these toolchain offer some advantages:
yann@1
    66
  - they are ready to use and quite easy to install and setup,
yann@1
    67
  - they are proven if used by a wide community.
yann@1
    68
yann@1
    69
But once you want to get all the juice out of your specific hardware, you will
yann@197
    70
want to build your own toolchain. This is where crosstool-NG comes into play.
yann@1
    71
rpjday@436
    72
There are also a number of tools that build toolchains for specific needs,
rpjday@436
    73
which are not really scalable. Examples are:
rpjday@436
    74
  - buildroot (buildroot.uclibc.org) whose main purpose is to build root file
yann@1
    75
    systems, hence the name. But once you have your toolchain with buildroot,
yann@1
    76
    part of it is installed in the root-to-be, so if you want to build a whole
yann@1
    77
    new root, you either have to save the existing one as a template and
yann@1
    78
    restore it later, or restart again from scratch. This is not convenient,
yann@1
    79
  - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very
yann@1
    80
    similar to buildroot,
yann@702
    81
  - other projects (openembedded.org for example), which are again used to
yann@1
    82
    build root file systems.
yann@1
    83
rpjday@436
    84
crosstool-NG is really targeted at building toolchains, and only toolchains.
yann@1
    85
It is then up to you to use it the way you want.
yann@1
    86
yann@1
    87
___________
yann@1
    88
          /
yann@1
    89
History  /
yann@1
    90
________/
yann@1
    91
rpjday@436
    92
crosstool was first 'conceived' by Dan Kegel, who offered it to the community
yann@1
    93
as a set of scripts, a repository of patches, and some pre-configured, general
yann@1
    94
purpose setup files to be used to configure crosstool. This is available at
yann@203
    95
http://www.kegel.com/crosstool, and the subversion repository is hosted on
yann@203
    96
google at http://code.google.com/p/crosstool/.
yann@1
    97
yann@1
    98
I once managed to add support for uClibc-based toolchains, but it did not make
yann@437
    99
into mainline, mostly because I didn't have time to port the patch forward to
yann@1
   100
the new versions, due in part to the big effort it was taking.
yann@1
   101
yann@1
   102
So I decided to clean up crosstool in the state it was, re-order the things
yann@437
   103
in place, add appropriate support for what I needed, that is uClibc support
yann@437
   104
and a menu-driven configuration, named the new implementation crosstool-NG,
yann@437
   105
(standing for crosstool Next Generation, as many other comunity projects do,
yann@437
   106
and as a wink at the TV series "Star Trek: The Next Generation" ;-) ) and
yann@437
   107
made it available to the community, in case it was of interest to any one.
yann@1
   108
yann@294
   109
___________________________
yann@294
   110
                          /
yann@294
   111
Installing crosstool-NG  /
yann@294
   112
________________________/
yann@294
   113
yann@294
   114
There are two ways you can use crosstool-NG:
yann@294
   115
 - build and install it, then get rid of the sources like you'd do for most
yann@294
   116
   programs,
yann@294
   117
 - or only build it and run from the source directory.
yann@294
   118
yann@294
   119
The former should be used if you got crosstool-NG from a packaged tarball, see
rpjday@436
   120
"Install method", below, while the latter is most useful for developpers that
yann@294
   121
checked the code out from SVN, and want to submit patches, see "The Hacker's
yann@294
   122
way", below.
yann@294
   123
yann@294
   124
Install method |
yann@294
   125
---------------+
yann@294
   126
yann@294
   127
If you go for the install, then you just follow the classical, but yet easy
yann@294
   128
./configure way:
yann@294
   129
  ./configure --prefix=/some/place
yann@294
   130
  make
yann@294
   131
  make install
yann@294
   132
  export PATH="${PATH}:/some/place/bin"
yann@294
   133
yann@294
   134
You can then get rid of crosstool-NG source. Next create a directory to serve
yann@294
   135
as a working place, cd in there and run:
yann@294
   136
  ct-ng help
yann@294
   137
yann@294
   138
See below for complete usage.
yann@294
   139
yann@294
   140
The Hacker's way |
yann@294
   141
-----------------+
yann@294
   142
yann@294
   143
If you go the hacker's way, then the usage is a bit different, although very
yann@294
   144
simple:
yann@294
   145
  ./configure --local
yann@294
   146
  make
yann@294
   147
yann@294
   148
Now, *do not* remove crosstool-NG sources. They are needed to run crosstool-NG!
yann@294
   149
Stay in the directory holding the sources, and run:
yann@294
   150
  ./ct-ng help
yann@294
   151
yann@294
   152
See below for complete usage.
yann@294
   153
yann@294
   154
Now, provided you checked-out the code, you can send me your interesting changes
yann@294
   155
by running:
yann@294
   156
  svn diff
yann@294
   157
yann@294
   158
and mailing me the result! :-P
yann@294
   159
yann@837
   160
Shell completion |
yann@837
   161
-----------------+
yann@837
   162
yann@837
   163
crosstool-NG comes with a shell script fragment that defines bash-compatible
yann@837
   164
completion. That shell fragment is currently not installed automatically, but
yann@837
   165
this is planned.
yann@837
   166
yann@837
   167
To install the shell script fragment, you have two options:
yann@837
   168
 - install system-wide, most probably by copying ct-ng.comp into
yann@837
   169
   /etc/bash_completion.d/
yann@837
   170
 - install for a single user, by copying ct-ng.comp into ${HOME}/ and
yann@837
   171
   sourcing this file from your ${HOME}/.bashrc
yann@837
   172
yann@456
   173
Contributed code |
yann@456
   174
-----------------+
yann@456
   175
yann@456
   176
Some people contibuted code that couldn't get merged for various reasons. This
yann@456
   177
code is available as patches in the contrib/ sub-directory. These patches are
yann@456
   178
to be applied to the source of crosstool-NG, prior to installing.
yann@456
   179
yann@620
   180
An easy way to use contributed code is to pass the --with-contrib= option to
yann@620
   181
./configure. The possible values depend upon which contributions are packaged
yann@620
   182
with your version, but you can get with it with passing one of those two
yann@620
   183
special values:
yann@620
   184
  --with-contrib=list
yann@620
   185
    will list all available contributions
yann@620
   186
yann@620
   187
  --with-contrib=all
yann@620
   188
    will select all avalaible contributions
yann@620
   189
yann@620
   190
There is no guarantee that a particuliar contribution applies to the current
yann@620
   191
version of crosstool-ng, or that it will work at all. Use contributions at
yann@620
   192
your own risk.
yann@620
   193
yann@168
   194
____________________________
yann@168
   195
                           /
yann@168
   196
Configuring crosstool-NG  /
yann@168
   197
_________________________/
yann@168
   198
yann@620
   199
crosstool-NG is configured with a configurator presenting a menu-stuctured set
yann@620
   200
of options. These options let you specify the way you want your toolchain
yann@620
   201
built, where you want it installed, what architecture and specific processor it
yann@277
   202
will support, the version of the components you want to use, etc... The
yann@277
   203
value for those options are then stored in a configuration file.
yann@277
   204
yann@476
   205
The configurator works the same way you configure your Linux kernel. It is
yann@277
   206
assumed you now how to handle this.
yann@168
   207
yann@168
   208
To enter the menu, type:
yann@192
   209
  ct-ng menuconfig
yann@168
   210
yann@203
   211
Almost every config item has a help entry. Read them carefully.
yann@168
   212
yann@168
   213
String and number options can refer to environment variables. In such a case,
yann@192
   214
you must use the shell syntax: ${VAR}. You shall neither single- nor double-
yann@294
   215
quote the string/number options.
yann@168
   216
yann@192
   217
There are three environment variables that are computed by crosstool-NG, and
yann@168
   218
that you can use:
yann@168
   219
yann@168
   220
CT_TARGET:
yann@335
   221
  It represents the target tuple you are building for. You can use it for
yann@168
   222
  example in the installation/prefix directory, such as:
yann@168
   223
    /opt/x-tools/${CT_TARGET}
yann@168
   224
yann@168
   225
CT_TOP_DIR:
yann@182
   226
  The top directory where crosstool-NG is running. You shouldn't need it in
yann@182
   227
  most cases. There is one case where you may need it: if you have local
yann@182
   228
  patches and you store them in your running directory, you can refer to them
yann@168
   229
  by using CT_TOP_DIR, such as:
yann@168
   230
    ${CT_TOP_DIR}/patches.myproject
yann@168
   231
yann@168
   232
CT_VERSION:
yann@192
   233
  The version of crosstool-NG you are using. Not much use for you, but it's
yann@168
   234
  there if you need it.
yann@168
   235
yann@168
   236
Interesting config options |
yann@476
   237
---------------------------+
yann@168
   238
yann@168
   239
CT_LOCAL_TARBALLS_DIR:
yann@277
   240
  If you already have some tarballs in a direcotry, enter it here. That will
yann@197
   241
  speed up the retrieving phase, where crosstool-NG would otherwise download
yann@168
   242
  those tarballs.
yann@168
   243
yann@168
   244
CT_PREFIX_DIR:
yann@168
   245
  This is where the toolchain will be installed in (and for now, where it
yann@437
   246
  will run from). Common use is to add the target tuple in the directory
yann@277
   247
  path, such as (see above):
yann@277
   248
    /opt/x-tools/${CT_TARGET}
yann@168
   249
yann@168
   250
CT_TARGET_VENDOR:
yann@168
   251
  An identifier for your toolchain, will take place in the vendor part of the
yann@335
   252
  target tuple. It shall *not* contain spaces or dashes. Usually, keep it
yann@168
   253
  to a one-word string, or use underscores to separate words if you need.
yann@168
   254
  Avoid dots, commas, and special characters.
yann@168
   255
yann@168
   256
CT_TARGET_ALIAS:
yann@168
   257
  An alias for the toolchian. It will be used as a prefix to the toolchain
yann@168
   258
  tools. For example, you will have ${CT_TARGET_ALIAS}-gcc
yann@168
   259
yann@246
   260
Also, if you think you don't see enough versions, you can try to enable one of
yann@246
   261
those:
yann@246
   262
yann@246
   263
CT_OBSOLETE:
yann@246
   264
  Show obsolete versions or tools. Most of the time, you don't want to base
yann@246
   265
  your toolchain on too old a version (of gcc, for example). But at times, it
yann@246
   266
  can come handy to use such an old version for regression tests. Those old
yann@294
   267
  versions are hidden behind CT_OBSOLETE.
yann@246
   268
yann@246
   269
CT_EXPERIMENTAL:
yann@246
   270
  Show experimental versions or tools. Again, you might not want to base your
yann@246
   271
  toolchain on too recent tools (eg. gcc) for production. But if you need a
yann@246
   272
  feature present only in a recent version, or a new tool, you can find them
yann@246
   273
  hidden behind CT_EXPERIMENTAL.
yann@246
   274
yann@276
   275
Re-building an existing toolchain |
yann@276
   276
----------------------------------+
yann@276
   277
yann@276
   278
If you have an existing toolchain, you can re-use the options used to build it
yann@276
   279
to create a new toolchain. That needs a very little bit of effort on your side
yann@894
   280
but is quite easy. The options to build a toolchain are saved with the
yann@894
   281
toolchain, and you can retrieve this configuration by running:
yann@894
   282
  ${CT_TARGET}-config
yann@276
   283
yann@894
   284
This will dump the configuration to stdout, so to rebuild a toolchain with this
yann@894
   285
configuration, the following is all you need to do:
yann@894
   286
  ${CT_TARGET}-config >.config
yann@276
   287
yann@894
   288
Then, you can review and change the configuration by running:
yann@894
   289
  ct-ng menuconfig
yann@276
   290
yann@168
   291
________________________
yann@168
   292
                       /
yann@168
   293
Running crosstool-NG  /
yann@168
   294
_____________________/
yann@1
   295
yann@168
   296
To build the toolchain, simply type:
yann@203
   297
  ct-ng build
yann@135
   298
yann@135
   299
This will use the above configuration to retrieve, extract and patch the
yann@135
   300
components, build, install and eventually test your newly built toolchain.
yann@1
   301
yann@1
   302
You are then free to add the toolchain /bin directory in your PATH to use
yann@1
   303
it at will.
yann@1
   304
yann@135
   305
In any case, you can get some terse help. Just type:
yann@192
   306
  ct-ng help
yann@203
   307
or:
yann@203
   308
  man 1 ct-ng
yann@135
   309
rpjday@436
   310
Stopping and restarting a build |
yann@476
   311
--------------------------------+
yann@135
   312
yann@135
   313
If you want to stop the build after a step you are debugging, you can pass the
yann@135
   314
variable STOP to make:
yann@192
   315
  ct-ng STOP=some_step
yann@135
   316
yann@135
   317
Conversely, if you want to restart a build at a specific step you are
yann@135
   318
debugging, you can pass the RESTART variable to make:
yann@192
   319
  ct-ng RESTART=some_step
yann@135
   320
yann@136
   321
Alternatively, you can call make with the name of a step to just do that step:
yann@192
   322
  ct-ng libc_headers
yann@136
   323
is equivalent to:
yann@620
   324
  ct-ng RESTART=libc_headers STOP=libc_headers
yann@136
   325
yann@304
   326
The shortcuts +step_name and step_name+ allow to respectively stop or restart
yann@136
   327
at that step. Thus:
yann@304
   328
  ct-ng +libc_headers        and:    ct-ng libc_headers+
yann@136
   329
are equivalent to:
yann@192
   330
  ct-ng STOP=libc_headers    and:    ct-ng RESTART=libc_headers
yann@136
   331
yann@181
   332
To obtain the list of acceptable steps, please call:
yann@544
   333
  ct-ng list-steps
yann@181
   334
yann@168
   335
Note that in order to restart a build, you'll have to say 'Y' to the config
yann@168
   336
option CT_DEBUG_CT_SAVE_STEPS, and that the previous build effectively went
yann@168
   337
that far.
yann@92
   338
yann@168
   339
Testing all toolchains at once |
yann@476
   340
-------------------------------+
yann@92
   341
yann@168
   342
You can test-build all samples; simply call:
yann@192
   343
  ct-ng regtest
yann@40
   344
yann@335
   345
Overriding the number of // jobs |
yann@476
   346
---------------------------------+
yann@335
   347
yann@335
   348
If you want to override the number of jobs to run in // (the -j option to
yann@335
   349
make), you can either re-enter the menuconfig, or simply add it on the command
yann@335
   350
line, as such:
yann@335
   351
  ct-ng build.4
yann@335
   352
yann@335
   353
which tells crosstool-NG to override the number of // jobs to 4.
yann@335
   354
yann@335
   355
You can see the actions that support overriding the number of // jobs in
yann@335
   356
the help menu. Those are the ones with [.#] after them (eg. build[.#] or
yann@335
   357
regtest[.#], and so on...).
yann@335
   358
yann@227
   359
_______________________
yann@227
   360
                      /
yann@227
   361
Using the toolchain  /
yann@227
   362
____________________/
yann@227
   363
yann@227
   364
Using the toolchain is as simple as adding the toolchain's bin directory in
yann@227
   365
your PATH, such as:
yann@227
   366
  export PATH="${PATH}:/your/toolchain/path/bin"
yann@227
   367
yann@335
   368
and then using the target tuple to tell the build systems to use your
yann@227
   369
toolchain:
yann@335
   370
  ./configure --target=your-target-tuple
yann@294
   371
or
yann@335
   372
  make CC=your-target-tuple-gcc
yann@294
   373
or
yann@335
   374
  make CROSS_COMPILE=your-target-tuple-
yann@294
   375
and so on...
yann@227
   376
yann@476
   377
It is strongly advised not to use the toolchain sys-root directory as an
yann@620
   378
install directory for your programs/packages. If you do so, you will not be
yann@476
   379
able to use your toolchain for another project. It is even strongly advised
yann@476
   380
that your toolchain is chmod-ed to read-only once successfully build, so that
yann@620
   381
you don't go polluting your toolchain with your programs/packages' files.
yann@476
   382
yann@476
   383
Thus, when you build a program/package, install it in a separate directory,
yann@476
   384
eg. /your/root. This directory is the /image/ of what would be in the root file
yann@620
   385
system of your target, and will contain all that your programs/packages have
yann@476
   386
installed.
yann@476
   387
yann@227
   388
When your root directory is ready, it is still missing some important bits: the
yann@227
   389
toolchain's libraries. To populate your root directory with those libs, just
yann@227
   390
run:
yann@335
   391
  your-target-tuple-populate -s /your/root -d /your/root-populated
yann@227
   392
yann@227
   393
This will copy /your/root into /your/root-populated, and put the needed and only
yann@227
   394
the needed libraries there. Thus you don't polute /your/root with any cruft that
yann@227
   395
would no longer be needed should you have to remove stuff. /your/root always
yann@227
   396
contains only those things you install in it.
yann@227
   397
yann@227
   398
You can then use /your/root-populated to build up your file system image, a
yann@227
   399
tarball, or to NFS-mount it from your target, or whatever you need.
yann@227
   400
yann@294
   401
populate accepts the following options:
yann@294
   402
yann@294
   403
 -s [src_dir]
yann@294
   404
    Use 'src_dir' as the 'source', un-populated root directory
yann@294
   405
yann@294
   406
 -d [dst_dir]
yann@294
   407
    Put the 'destination', populated root directory in 'dst_dir'
yann@294
   408
yann@294
   409
 -f
yann@294
   410
    Remove 'dst_dir' if it previously existed
yann@294
   411
yann@294
   412
 -v
yann@294
   413
    Be verbose, and tell what's going on (you can see exactly where libs are
yann@294
   414
    coming from).
yann@294
   415
yann@294
   416
 -h
yann@294
   417
    Print the help
yann@294
   418
yann@40
   419
___________________
yann@40
   420
                  /
yann@40
   421
Toolchain types  /
yann@40
   422
________________/
yann@40
   423
yann@40
   424
There are four kinds of toolchains you could encounter.
yann@40
   425
yann@40
   426
First off, you must understand the following: when it comes to compilers there
yann@40
   427
are up to four machines involved:
yann@40
   428
  1) the machine configuring the toolchain components: the config machine
yann@40
   429
  2) the machine building the toolchain components:    the build machine
yann@40
   430
  3) the machine running the toolchain:                the host machine
yann@203
   431
  4) the machine the toolchain is generating code for: the target machine
yann@40
   432
yann@40
   433
We can most of the time assume that the config machine and the build machine
yann@40
   434
are the same. Most of the time, this will be true. The only time it isn't
yann@40
   435
is if you're using distributed compilation (such as distcc). Let's forget
yann@40
   436
this for the sake of simplicity.
yann@40
   437
yann@40
   438
So we're left with three machines:
yann@40
   439
 - build
yann@40
   440
 - host
yann@40
   441
 - target
yann@40
   442
yann@40
   443
Any toolchain will involve those three machines. You can be as pretty sure of
yann@40
   444
this as "2 and 2 are 4". Here is how they come into play:
yann@40
   445
yann@40
   446
1) build == host == target
yann@40
   447
    This is a plain native toolchain, targetting the exact same machine as the
yann@40
   448
    one it is built on, and running again on this exact same machine. You have
yann@40
   449
    to build such a toolchain when you want to use an updated component, such
yann@40
   450
    as a newer gcc for example.
yann@197
   451
    crosstool-NG calls it "native".
yann@40
   452
yann@40
   453
2) build == host != target
yann@40
   454
    This is a classic cross-toolchain, which is expected to be run on the same
yann@40
   455
    machine it is compiled on, and generate code to run on a second machine,
yann@40
   456
    the target.
yann@197
   457
    crosstool-NG calls it "cross".
yann@40
   458
yann@40
   459
3) build != host == target
yann@40
   460
    Such a toolchain is also a native toolchain, as it targets the same machine
yann@40
   461
    as it runs on. But it is build on another machine. You want such a
yann@40
   462
    toolchain when porting to a new architecture, or if the build machine is
yann@40
   463
    much faster than the host machine.
yann@197
   464
    crosstool-NG calls it "cross-native".
yann@40
   465
yann@40
   466
4) build != host != target
yann@92
   467
    This one is called a canadian-toolchain (*), and is tricky. The three
yann@40
   468
    machines in play are different. You might want such a toolchain if you
yann@40
   469
    have a fast build machine, but the users will use it on another machine,
yann@40
   470
    and will produce code to run on a third machine.
yann@197
   471
    crosstool-NG calls it "canadian".
yann@40
   472
yann@197
   473
crosstool-NG can build all these kinds of toolchains (or is aiming at it,
yann@197
   474
anyway!)
yann@40
   475
yann@40
   476
(*) The term Canadian Cross came about because at the time that these issues
yann@40
   477
    were all being hashed out, Canada had three national political parties.
yann@40
   478
    http://en.wikipedia.org/wiki/Cross_compiler
yann@40
   479
yann@1
   480
_____________
yann@1
   481
            /
yann@1
   482
Internals  /
yann@1
   483
__________/
yann@1
   484
yann@92
   485
Internally, crosstool-NG is script-based. To ease usage, the frontend is
yann@92
   486
Makefile-based.
yann@92
   487
yann@92
   488
Makefile front-end |
yann@476
   489
-------------------+
yann@92
   490
yann@203
   491
The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
yann@203
   492
script with an action will act exactly as if the Makefile was in the current
yann@203
   493
working directory and make was called with the action as rule. Thus:
yann@203
   494
  ct-ng menuconfig
yann@294
   495
yann@203
   496
is equivalent to having the Makefile in CWD, and calling:
yann@203
   497
  make menuconfig
yann@203
   498
yann@203
   499
Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
yann@203
   500
traditional command.
yann@203
   501
yann@203
   502
ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
yann@203
   503
at configuration time with ./configure.
yann@203
   504
yann@437
   505
ct-ng also searches for config files, sub-tools, samples, scripts and patches in
yann@203
   506
that library directory.
yann@92
   507
yann@294
   508
Because of a stupid make behavior/bug I was unable to track down, implicit make
yann@294
   509
rules are disabled: installing with --local would triger those rules, and mconf
yann@294
   510
was unbuildable.
yann@294
   511
yann@182
   512
Kconfig parser |
yann@476
   513
---------------+
yann@92
   514
yann@92
   515
The kconfig language is a hacked version, vampirised from the toybox project
yann@182
   516
by Rob LANDLEY (http://www.landley.net/code/toybox/), itself coming from the
yann@294
   517
Linux kernel (http://www.kernel.org/), and (heavily) adapted to my needs.
yann@92
   518
yann@203
   519
The kconfig parsers (conf and mconf) are not installed pre-built, but as
yann@203
   520
source files. Thus you can have the directory where crosstool-NG is installed,
yann@203
   521
exported (via NFS or whatever) and have clients with different architectures
yann@203
   522
use the same crosstool-NG installation, and most notably, the same set of
yann@203
   523
patches.
yann@203
   524
yann@381
   525
Architecture-specific |
yann@476
   526
----------------------+
yann@381
   527
yann@628
   528
Note: this chapter is not really well written, and might thus be a little bit
yann@628
   529
complex to understand. To get a better grasp of what an architecture is, the
yann@628
   530
reader is kindly encouraged to look at the "arch/" sub-directory, and to the
yann@628
   531
existing architectures to see how things are laid out.
yann@628
   532
yann@381
   533
An architecture is defined by:
yann@381
   534
yann@381
   535
 - a human-readable name, in lower case letters, with numbers as appropriate.
yann@628
   536
   The underscore is allowed; space and special characters are not.
yann@628
   537
     Eg.: arm, x86_64
yann@903
   538
 - a file in "config/arch/", named after the architecture's name, and suffixed
yann@903
   539
   with ".in".
yann@903
   540
     Eg.: config/arch/arm.in
yann@903
   541
 - a file in "scripts/build/arch/", named after the architecture's name, and
yann@903
   542
   suffixed with ".sh".
yann@903
   543
     Eg.: scripts/build/arch/arm.sh
yann@628
   544
yann@903
   545
The architecture's ".in" file API:
yann@628
   546
 > the config option "ARCH_%arch%" (where %arch% is to be replaced with the
yann@628
   547
   actual architecture name).
yann@628
   548
   That config option must have *neither* a type, *nor* a prompt! Also, it can
yann@628
   549
   *not* depend on any other config option (EXPERIMENTAL is managed as above).
yann@628
   550
     Eg.:
yann@628
   551
       config ARCH_arm
yann@630
   552
   + mandatory:
yann@702
   553
       defines a (terse) help entry for this architecture:
yann@630
   554
       Eg.:
yann@630
   555
         config ARCH_arm
yann@630
   556
           help
yann@630
   557
             The ARM architecture.
yann@628
   558
   + optional:
yann@628
   559
       selects adequate associated config options.
yann@628
   560
       Eg.:
yann@628
   561
         config ARCH_arm
yann@628
   562
           select ARCH_SUPPORTS_BOTH_ENDIAN
yann@628
   563
           select ARCH_DEFAULT_LE
yann@630
   564
           help
yann@630
   565
             The ARM architecture.
yann@628
   566
yann@628
   567
 > other target-specific options, at your discretion. Note however that to
yann@628
   568
   avoid name-clashing, such options shall be prefixed with "ARCH_%arch%",
yann@628
   569
   where %arch% is again replaced by the actual architecture name.
yann@628
   570
   (Note: due to historical reasons, and lack of time to clean up the code,
yann@628
   571
    I may have left some config options that do not completely conform to
yann@628
   572
    this, as the architecture name was written all upper case. However, the
yann@628
   573
    prefix is unique among architectures, and does not cause harm).
yann@381
   574
yann@903
   575
The architecture's ".sh" file API:
yann@383
   576
 > the function "CT_DoArchValues"
yann@381
   577
   + parameters: none
yann@381
   578
   + environment:
yann@901
   579
     - all variables from the ".config" file,
yann@901
   580
     - the two variables "target_endian_eb" and "target_endian_el" which are
yann@901
   581
       the endianness suffixes
yann@381
   582
   + return value: 0 upon success, !0 upon failure
yann@381
   583
   + provides:
yann@391
   584
     - mandatory
yann@383
   585
     - the environment variable CT_TARGET_ARCH
yann@389
   586
     - contains:
yann@389
   587
       the architecture part of the target tuple.
yann@389
   588
       Eg.: "armeb" for big endian ARM
yann@389
   589
            "i386" for an i386
yann@389
   590
   + provides:
yann@391
   591
     - optional
yann@389
   592
     - the environment variable CT_TARGET_SYS
yann@456
   593
     - contains:
yann@383
   594
       the sytem part of the target tuple.
yann@383
   595
       Eg.: "gnu" for glibc on most architectures
yann@383
   596
            "gnueabi" for glibc on an ARM EABI
yann@383
   597
     - defaults to:
yann@383
   598
       - for glibc-based toolchain: "gnu"
yann@383
   599
       - for uClibc-based toolchain: "uclibc"
yann@383
   600
   + provides:
yann@383
   601
     - optional
yann@391
   602
     - the environment variable CT_KERNEL_ARCH
yann@383
   603
     - contains:
yann@391
   604
       the architecture name as understandable by the Linux kernel build
yann@391
   605
       system.
yann@391
   606
       Eg.: "arm" for an ARM
yann@391
   607
            "powerpc" for a PowerPC
yann@391
   608
            "i386" for an x86
yann@383
   609
     - defaults to:
yann@391
   610
       ${CT_ARCH}
yann@391
   611
   + provides:
yann@391
   612
     - optional
yann@767
   613
     - the environment variables to configure the cross-gcc (defaults)
yann@767
   614
       - CT_ARCH_WITH_ARCH    : the gcc ./configure switch to select architecture level         ( "--with-arch=${CT_ARCH_ARCH}"   )
yann@767
   615
       - CT_ARCH_WITH_ABI     : the gcc ./configure switch to select ABI level                  ( "--with-abi=${CT_ARCH_ABI}"     )
yann@767
   616
       - CT_ARCH_WITH_CPU     : the gcc ./configure switch to select CPU instruction set        ( "--with-cpu=${CT_ARCH_CPU}"     )
yann@767
   617
       - CT_ARCH_WITH_TUNE    : the gcc ./configure switch to select scheduling                 ( "--with-tune=${CT_ARCH_TUNE}"   )
yann@767
   618
       - CT_ARCH_WITH_FPU     : the gcc ./configure switch to select FPU type                   ( "--with-fpu=${CT_ARCH_FPU}"     )
yann@767
   619
       - CT_ARCH_WITH_FLOAT   : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/  )
yann@391
   620
   + provides:
yann@391
   621
     - optional
yann@767
   622
     - the environment variables to pass to the cross-gcc to build target binaries (defaults)
yann@391
   623
       - CT_ARCH_ARCH_CFLAG   : the gcc switch to select architecture level                     ( "-march=${CT_ARCH_ARCH}"            )
yann@456
   624
       - CT_ARCH_ABI_CFLAG    : the gcc switch to select ABI level                              ( "-mabi=${CT_ARCH_ABI}"              )
yann@391
   625
       - CT_ARCH_CPU_CFLAG    : the gcc switch to select CPU instruction set                    ( "-mcpu=${CT_ARCH_CPU}"              )
yann@391
   626
       - CT_ARCH_TUNE_CFLAG   : the gcc switch to select scheduling                             ( "-mtune=${CT_ARCH_TUNE}"            )
yann@391
   627
       - CT_ARCH_FPU_CFLAG    : the gcc switch to select FPU type                               ( "-mfpu=${CT_ARCH_FPU}"              )
yann@391
   628
       - CT_ARCH_FLOAT_CFLAG  : the gcc switch to choose floating point arithmetics             ( "-msoft-float" or /empty/           )
yann@391
   629
       - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian                   ( "-mbig-endian" or "-mlittle-endian" )
yann@391
   630
     - default to:
yann@391
   631
       see above.
yann@767
   632
   + provides:
yann@767
   633
     - optional
yann@767
   634
     - the environement variables to configure the core and final compiler, specific to this architecture:
yann@767
   635
       - CT_ARCH_CC_CORE_EXTRA_CONFIG   : additional, architecture specific core gcc ./configure flags
yann@767
   636
       - CT_ARCH_CC_EXTRA_CONFIG        : additional, architecture specific final gcc ./configure flags
yann@767
   637
     - default to:
yann@767
   638
       - all empty
yann@767
   639
   + provides:
yann@767
   640
     - optional
yann@767
   641
     - the architecture-specific CFLAGS and LDFLAGS:
yann@767
   642
       - CT_ARCH_TARGET_CLFAGS
yann@767
   643
       - CT_ARCH_TARGET_LDFLAGS
yann@767
   644
     - default to:
yann@767
   645
       - all empty
yann@628
   646
yann@903
   647
You can have a look at "config/arch/arm.in" and "scripts/build/arch/arm.sh" for
yann@903
   648
a quite complete example of what an actual architecture description looks like.
yann@901
   649
yann@890
   650
Kernel specific |
yann@890
   651
----------------+
yann@890
   652
yann@890
   653
A kernel is defined by:
yann@890
   654
yann@890
   655
 - a human-readable name, in lower case letters, with numbers as appropriate.
yann@890
   656
   The underscore is allowed; space and special characters are not (although
yann@890
   657
   they are internally replaced with underscores.
yann@890
   658
     Eg.: linux, bare-metal
yann@890
   659
 - a file in "config/kernel/", named after the kernel name, and suffixed with
yann@890
   660
   ".in".
yann@890
   661
     Eg.: config/kernel/linux.in, config/kernel/bare-metal.in
yann@901
   662
 - a file in "scripts/build/kernel/", named after the kernel name, and suffixed
yann@901
   663
   with ".sh".
yann@901
   664
     Eg.: scripts/build/kernel/linux.sh, scripts/build/kernel/bare-metal.sh
yann@890
   665
yann@890
   666
The kernel's ".in" file must contain:
yann@890
   667
 > an optional lines containing exactly "# EXPERIMENTAL", starting on the
yann@890
   668
   first column, and without any following space or other character.
yann@890
   669
   If this line is present, then this kernel is considered EXPERIMENTAL,
yann@890
   670
   and correct dependency on EXPERIMENTAL will be set.
yann@901
   671
yann@890
   672
 > the config option "KERNEL_%kernel_name%" (where %kernel_name% is to be
yann@890
   673
   replaced with the actual kernel name, with all special characters and
yann@890
   674
   spaces replaced by underscores).
yann@890
   675
   That config option must have *neither* a type, *nor* a prompt! Also, it can
yann@890
   676
   *not* depends on EXPERIMENTAL.
yann@890
   677
     Eg.: KERNEL_linux, KERNEL_bare_metal
yann@890
   678
   + mandatory:
yann@890
   679
       defines a (terse) help entry for this kernel.
yann@890
   680
       Eg.:
yann@890
   681
         config KERNEL_bare_metal
yann@890
   682
           help
yann@890
   683
             Build a compiler for use without any kernel.
yann@890
   684
   + optional:
yann@890
   685
       selects adequate associated config options.
yann@890
   686
       Eg.:
yann@890
   687
         config KERNEL_bare_metal
yann@890
   688
           select BARE_METAL
yann@890
   689
           help
yann@890
   690
             Build a compiler for use without any kernel.
yann@890
   691
yann@890
   692
 > other kernel specific options, at your discretion. Note however that, to
yann@890
   693
   avoid name-clashing, such options should be prefixed with
yann@890
   694
   "KERNEL_%kernel_name%", where %kernel_name% is again tp be replaced with
yann@890
   695
   the actual kernel name.
yann@890
   696
   (Note: due to historical reasons, and lack of time to clean up the code,
yann@890
   697
    I may have left some config options that do not completely conform to
yann@890
   698
    this, as the kernel name was written all upper case. However, the prefix
yann@890
   699
    is unique among kernels, and does not cause harm).
yann@890
   700
yann@901
   701
The kernel's ".sh" file API:
yann@901
   702
 > is a bash script fragment
yann@901
   703
yann@901
   704
 > defines the function "do_print_filename":
yann@901
   705
   + parameters: none
yann@901
   706
   + environment:
yann@901
   707
      - all variables from the ".config" file,
yann@901
   708
   + return value: ignored
yann@901
   709
   + behavior: output the kernel's tarball filename, with adequate suffix,
yann@901
   710
     on stdout.
yann@901
   711
       Eg.: linux-2.6.26.5.tar.bz2
yann@901
   712
yann@901
   713
 > defines the function "do_kernel_get":
yann@901
   714
   + parameters: none
yann@901
   715
   + environment:
yann@901
   716
      - all variables from the ".config" file.
yann@901
   717
   + return value: 0 for success, !0 for failure.
yann@901
   718
   + behavior: download the kernel's sources, and store the tarball into
yann@901
   719
     "${CT_TARBALLS_DIR}". To this end, a functions is available, that
yann@901
   720
     abstracts downloading tarballs:
yann@901
   721
     - CT_DoGet <tarball_base_name> <URL1 [URL...]>
yann@901
   722
       Eg.: CT_DoGet linux-2.6.26.5 ftp://ftp.kernel.org/pub/linux/kernel/v2.6
yann@901
   723
     Note: retrieving sources from svn, cvs, git and the likes is not supported
yann@901
   724
     by CT_DoGet. You'll have to do this by hand, as it is done for eglibc in
yann@901
   725
     "scripts/build/libc/eglibc.sh"
yann@901
   726
yann@901
   727
 > defines the function "do_kernel_extract":
yann@901
   728
   + parameters: none
yann@901
   729
   + environment:
yann@901
   730
      - all variables from the ".config" file,
yann@901
   731
   + return value: 0 for success, !0 for failure.
yann@901
   732
   + behavior: extract the kernel's tarball into "${CT_SRC_DIR}", and apply
yann@901
   733
     required patches. To this end, a function is available, that abstracts
yann@901
   734
     extracting tarballs:
yann@901
   735
     - CT_ExtractAndPatch <tarball_base_name>
yann@901
   736
       Eg.: CT_ExtractAndPatch linux-2.6.26.5
yann@901
   737
yann@901
   738
 > defines the function "do_kernel_headers":
yann@901
   739
   + parameters: none
yann@901
   740
   + environment:
yann@901
   741
      - all variables from the ".config" file,
yann@901
   742
   + return value: 0 for success, !0 for failure.
yann@901
   743
   + behavior: install the kernel headers (if any) in "${CT_SYSROOT_DIR}/usr/include"
yann@901
   744
yann@901
   745
 > defines any kernel-specific helper functions
yann@901
   746
   These functions, if any, must be prefixed with "do_kernel_%CT_KERNEL%_",
yann@901
   747
   where '%CT_KERNEL%' is to be replaced with the actual kernel name, to avoid
yann@901
   748
   any name-clashing.
yann@901
   749
yann@901
   750
You can have a look at "config/kernel/linux.in" and "scripts/build/kernel/linux.sh"
yann@903
   751
as an example of what a complex kernel description looks like.
yann@901
   752
yann@620
   753
Adding a new version of a component |
yann@476
   754
------------------------------------+
yann@476
   755
yann@476
   756
When a new component, such as the Linux kernel, gcc or any other is released,
yann@476
   757
adding the new version to crosstool-NG is quite easy. There is a script that
yann@476
   758
will do all that for you:
yann@476
   759
  tools/addToolVersion.sh
yann@476
   760
yann@476
   761
Run it with no option to get some help.
yann@381
   762
yann@203
   763
Build scripts |
yann@476
   764
--------------+
yann@203
   765
yann@203
   766
To Be Written later...