yann@1
|
1 |
File.........: overview.txt
|
yann@197
|
2 |
Content......: Overview of how crosstool-NG works.
|
yann@92
|
3 |
Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>
|
yann@192
|
4 |
License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
|
yann@92
|
5 |
|
yann@1
|
6 |
________________
|
yann@1
|
7 |
/
|
yann@1
|
8 |
Introduction /
|
yann@1
|
9 |
_____________/
|
yann@1
|
10 |
|
yann@1
|
11 |
crosstool-NG aims at building toolchains. Toolchains are an essential component
|
yann@1
|
12 |
in a software development project. It will compile, assemble and link the code
|
yann@1
|
13 |
that is being developped. Some pieces of the toolchain will eventually end up
|
yann@1
|
14 |
in the resulting binary/ies: static libraries are but an example.
|
yann@1
|
15 |
|
yann@1
|
16 |
So, a toolchain is a very sensitive piece of software, as any bug in one of the
|
yann@1
|
17 |
components, or a poorly configured component, can lead to execution problems,
|
yann@1
|
18 |
ranging from poor performance, to applications ending unexpectedly, to
|
yann@1
|
19 |
mis-behaving software (which more than often is hard to detect), to hardware
|
yann@1
|
20 |
damage, or even to human risks (which is more than regretable).
|
yann@1
|
21 |
|
yann@1
|
22 |
Toolchains are made of different piece of software, each being quite complex
|
yann@1
|
23 |
and requiring specially crafted options to build and work seamlessly. This
|
yann@1
|
24 |
is usually not that easy, even in the not-so-trivial case of native toolchains.
|
yann@1
|
25 |
The work reaches a higher degree of complexity when it comes to cross-
|
yann@40
|
26 |
compilation, where it can become quite a nightmare...
|
yann@1
|
27 |
|
yann@40
|
28 |
Some cross-toolchains exist on the internet, and can be used for general
|
yann@1
|
29 |
development, but they have a number of limitations:
|
yann@1
|
30 |
- they can be general purpose, in that they are configured for the majority:
|
yann@1
|
31 |
no optimisation for your specific target,
|
yann@1
|
32 |
- they can be prepared for a specific target and thus are not easy to use,
|
yann@1
|
33 |
nor optimised for, or even supporting your target,
|
yann@1
|
34 |
- they often are using ageing components (compiler, C library, etc...) not
|
yann@1
|
35 |
supporting special features of your shiny new processor;
|
yann@1
|
36 |
On the other side, these toolchain offer some advantages:
|
yann@1
|
37 |
- they are ready to use and quite easy to install and setup,
|
yann@1
|
38 |
- they are proven if used by a wide community.
|
yann@1
|
39 |
|
yann@1
|
40 |
But once you want to get all the juice out of your specific hardware, you will
|
yann@197
|
41 |
want to build your own toolchain. This is where crosstool-NG comes into play.
|
yann@1
|
42 |
|
yann@1
|
43 |
There are also a number of tools that builds toolchains for specific needs,
|
yann@1
|
44 |
which is not really scalable. Examples are:
|
yann@1
|
45 |
- buildroot (buildroot.uclibc.org) whose main puprpose is to build root file
|
yann@1
|
46 |
systems, hence the name. But once you have your toolchain with buildroot,
|
yann@1
|
47 |
part of it is installed in the root-to-be, so if you want to build a whole
|
yann@1
|
48 |
new root, you either have to save the existing one as a template and
|
yann@1
|
49 |
restore it later, or restart again from scratch. This is not convenient,
|
yann@1
|
50 |
- ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very
|
yann@1
|
51 |
similar to buildroot,
|
yann@1
|
52 |
- other projects (openembeded.org for example), which is again used to
|
yann@1
|
53 |
build root file systems.
|
yann@1
|
54 |
|
yann@1
|
55 |
crosstool-NG is really targetted at building toolchains, and only toolchains.
|
yann@1
|
56 |
It is then up to you to use it the way you want.
|
yann@1
|
57 |
|
yann@1
|
58 |
___________
|
yann@1
|
59 |
/
|
yann@1
|
60 |
History /
|
yann@1
|
61 |
________/
|
yann@1
|
62 |
|
yann@1
|
63 |
crosstool was first 'conceived' by Dan Kegel, which offered it to the community,
|
yann@1
|
64 |
as a set of scripts, a repository of patches, and some pre-configured, general
|
yann@1
|
65 |
purpose setup files to be used to configure crosstool. This is available at
|
yann@203
|
66 |
http://www.kegel.com/crosstool, and the subversion repository is hosted on
|
yann@203
|
67 |
google at http://code.google.com/p/crosstool/.
|
yann@1
|
68 |
|
yann@1
|
69 |
At the time of writing, crosstool only supports building with one C library,
|
yann@1
|
70 |
namely glibc, and one C compiler, gcc; it is cripled with historical support
|
yann@92
|
71 |
for legacy components, and is some kind of a mess to upgrade. Also, submited
|
yann@203
|
72 |
patches take a loooong time before they are integrated mainline.
|
yann@1
|
73 |
|
yann@1
|
74 |
I once managed to add support for uClibc-based toolchains, but it did not make
|
yann@1
|
75 |
into mainline, mostly because I don't have time to port the patch forward to
|
yann@1
|
76 |
the new versions, due in part to the big effort it was taking.
|
yann@1
|
77 |
|
yann@1
|
78 |
So I decided to clean up crosstool in the state it was, re-order the things
|
yann@1
|
79 |
in place, and add appropriate support for what I needed, that is uClibc
|
yann@203
|
80 |
support. That was a disaster, as inclusion into mainline is slow as hell,
|
yann@203
|
81 |
and the changes were so numerous.
|
yann@1
|
82 |
|
yann@1
|
83 |
The only option left to me was rewrite crosstool from scratch. I decided to go
|
yann@197
|
84 |
this way, and name the new implementation crosstool-NG, standing for crosstool
|
yann@197
|
85 |
Next Generation, as many other comunity projects do, and as a wink at the TV
|
yann@197
|
86 |
series "Star Trek: The Next Generation". ;-)
|
yann@1
|
87 |
|
yann@168
|
88 |
____________________________
|
yann@168
|
89 |
/
|
yann@168
|
90 |
Configuring crosstool-NG /
|
yann@168
|
91 |
_________________________/
|
yann@168
|
92 |
|
yann@168
|
93 |
crosstool-NG is configured the same way you configure your Linux kernel: by
|
yann@168
|
94 |
using a curses-based menu. It is assumed you now how to handle this.
|
yann@168
|
95 |
|
yann@168
|
96 |
To enter the menu, type:
|
yann@192
|
97 |
ct-ng menuconfig
|
yann@168
|
98 |
|
yann@203
|
99 |
Almost every config item has a help entry. Read them carefully.
|
yann@168
|
100 |
|
yann@168
|
101 |
String and number options can refer to environment variables. In such a case,
|
yann@192
|
102 |
you must use the shell syntax: ${VAR}. You shall neither single- nor double-
|
yann@192
|
103 |
quote the string options.
|
yann@168
|
104 |
|
yann@192
|
105 |
There are three environment variables that are computed by crosstool-NG, and
|
yann@168
|
106 |
that you can use:
|
yann@168
|
107 |
|
yann@168
|
108 |
CT_TARGET:
|
yann@168
|
109 |
It represents the target triplet you are building for. You can use it for
|
yann@168
|
110 |
example in the installation/prefix directory, such as:
|
yann@168
|
111 |
/opt/x-tools/${CT_TARGET}
|
yann@168
|
112 |
|
yann@168
|
113 |
CT_TOP_DIR:
|
yann@182
|
114 |
The top directory where crosstool-NG is running. You shouldn't need it in
|
yann@182
|
115 |
most cases. There is one case where you may need it: if you have local
|
yann@182
|
116 |
patches and you store them in your running directory, you can refer to them
|
yann@168
|
117 |
by using CT_TOP_DIR, such as:
|
yann@168
|
118 |
${CT_TOP_DIR}/patches.myproject
|
yann@168
|
119 |
|
yann@168
|
120 |
CT_VERSION:
|
yann@192
|
121 |
The version of crosstool-NG you are using. Not much use for you, but it's
|
yann@168
|
122 |
there if you need it.
|
yann@168
|
123 |
|
yann@168
|
124 |
|
yann@168
|
125 |
Interesting config options |
|
yann@168
|
126 |
---------------------------*
|
yann@168
|
127 |
|
yann@168
|
128 |
CT_LOCAL_TARBALLS_DIR:
|
yann@168
|
129 |
If you already have sone tarballs in a direcotry, enter it here. That will
|
yann@197
|
130 |
speed up the retrieving phase, where crosstool-NG would otherwise download
|
yann@168
|
131 |
those tarballs.
|
yann@168
|
132 |
|
yann@168
|
133 |
CT_PREFIX_DIR:
|
yann@168
|
134 |
This is where the toolchain will be installed in (and for now, where it
|
yann@168
|
135 |
will run from).
|
yann@168
|
136 |
|
yann@168
|
137 |
CT_TARGET_VENDOR:
|
yann@168
|
138 |
An identifier for your toolchain, will take place in the vendor part of the
|
yann@168
|
139 |
target triplet. It shall *not* contain spaces or dashes. Usually, keep it
|
yann@168
|
140 |
to a one-word string, or use underscores to separate words if you need.
|
yann@168
|
141 |
Avoid dots, commas, and special characters.
|
yann@168
|
142 |
|
yann@168
|
143 |
CT_TARGET_ALIAS:
|
yann@168
|
144 |
An alias for the toolchian. It will be used as a prefix to the toolchain
|
yann@168
|
145 |
tools. For example, you will have ${CT_TARGET_ALIAS}-gcc
|
yann@168
|
146 |
|
yann@168
|
147 |
________________________
|
yann@168
|
148 |
/
|
yann@168
|
149 |
Running crosstool-NG /
|
yann@168
|
150 |
_____________________/
|
yann@1
|
151 |
|
yann@197
|
152 |
crosstool-NG is configured by a configurator presenting a menu-stuctured set of
|
yann@1
|
153 |
options. These options let you specify the way you want your toolchain built,
|
yann@1
|
154 |
where you want it installed, what architecture and specific processor it
|
yann@1
|
155 |
will support, the version of the components you want to use, etc... The
|
yann@1
|
156 |
value for those options are then stored in a configuration file.
|
yann@1
|
157 |
|
yann@168
|
158 |
To build the toolchain, simply type:
|
yann@203
|
159 |
ct-ng build
|
yann@135
|
160 |
|
yann@135
|
161 |
This will use the above configuration to retrieve, extract and patch the
|
yann@135
|
162 |
components, build, install and eventually test your newly built toolchain.
|
yann@1
|
163 |
|
yann@1
|
164 |
You are then free to add the toolchain /bin directory in your PATH to use
|
yann@1
|
165 |
it at will.
|
yann@1
|
166 |
|
yann@135
|
167 |
In any case, you can get some terse help. Just type:
|
yann@192
|
168 |
ct-ng help
|
yann@203
|
169 |
or:
|
yann@203
|
170 |
man 1 ct-ng
|
yann@135
|
171 |
|
yann@135
|
172 |
|
yann@135
|
173 |
Stoping and restarting a build |
|
yann@135
|
174 |
-------------------------------*
|
yann@135
|
175 |
|
yann@135
|
176 |
If you want to stop the build after a step you are debugging, you can pass the
|
yann@135
|
177 |
variable STOP to make:
|
yann@192
|
178 |
ct-ng STOP=some_step
|
yann@135
|
179 |
|
yann@135
|
180 |
Conversely, if you want to restart a build at a specific step you are
|
yann@135
|
181 |
debugging, you can pass the RESTART variable to make:
|
yann@192
|
182 |
ct-ng RESTART=some_step
|
yann@135
|
183 |
|
yann@136
|
184 |
Alternatively, you can call make with the name of a step to just do that step:
|
yann@192
|
185 |
ct-ng libc_headers
|
yann@136
|
186 |
is equivalent to:
|
yann@192
|
187 |
ct-ng RESTART=libs_headers STOP=libc_headers
|
yann@136
|
188 |
|
yann@136
|
189 |
The shortcuts -step_name and step_name- allow to respectively stop or restart
|
yann@136
|
190 |
at that step. Thus:
|
yann@192
|
191 |
ct-ng -libc_headers and: ct-ng libc_headers-
|
yann@136
|
192 |
are equivalent to:
|
yann@192
|
193 |
ct-ng STOP=libc_headers and: ct-ng RESTART=libc_headers
|
yann@136
|
194 |
|
yann@181
|
195 |
To obtain the list of acceptable steps, please call:
|
yann@192
|
196 |
ct-ng liststeps
|
yann@181
|
197 |
|
yann@168
|
198 |
Note that in order to restart a build, you'll have to say 'Y' to the config
|
yann@168
|
199 |
option CT_DEBUG_CT_SAVE_STEPS, and that the previous build effectively went
|
yann@168
|
200 |
that far.
|
yann@92
|
201 |
|
yann@92
|
202 |
|
yann@168
|
203 |
Testing all toolchains at once |
|
yann@168
|
204 |
-------------------------------*
|
yann@92
|
205 |
|
yann@168
|
206 |
You can test-build all samples; simply call:
|
yann@192
|
207 |
ct-ng regtest
|
yann@40
|
208 |
|
yann@40
|
209 |
___________________
|
yann@40
|
210 |
/
|
yann@40
|
211 |
Toolchain types /
|
yann@40
|
212 |
________________/
|
yann@40
|
213 |
|
yann@40
|
214 |
There are four kinds of toolchains you could encounter.
|
yann@40
|
215 |
|
yann@40
|
216 |
First off, you must understand the following: when it comes to compilers there
|
yann@40
|
217 |
are up to four machines involved:
|
yann@40
|
218 |
1) the machine configuring the toolchain components: the config machine
|
yann@40
|
219 |
2) the machine building the toolchain components: the build machine
|
yann@40
|
220 |
3) the machine running the toolchain: the host machine
|
yann@203
|
221 |
4) the machine the toolchain is generating code for: the target machine
|
yann@40
|
222 |
|
yann@40
|
223 |
We can most of the time assume that the config machine and the build machine
|
yann@40
|
224 |
are the same. Most of the time, this will be true. The only time it isn't
|
yann@40
|
225 |
is if you're using distributed compilation (such as distcc). Let's forget
|
yann@40
|
226 |
this for the sake of simplicity.
|
yann@40
|
227 |
|
yann@40
|
228 |
So we're left with three machines:
|
yann@40
|
229 |
- build
|
yann@40
|
230 |
- host
|
yann@40
|
231 |
- target
|
yann@40
|
232 |
|
yann@40
|
233 |
Any toolchain will involve those three machines. You can be as pretty sure of
|
yann@40
|
234 |
this as "2 and 2 are 4". Here is how they come into play:
|
yann@40
|
235 |
|
yann@40
|
236 |
1) build == host == target
|
yann@40
|
237 |
This is a plain native toolchain, targetting the exact same machine as the
|
yann@40
|
238 |
one it is built on, and running again on this exact same machine. You have
|
yann@40
|
239 |
to build such a toolchain when you want to use an updated component, such
|
yann@40
|
240 |
as a newer gcc for example.
|
yann@197
|
241 |
crosstool-NG calls it "native".
|
yann@40
|
242 |
|
yann@40
|
243 |
2) build == host != target
|
yann@40
|
244 |
This is a classic cross-toolchain, which is expected to be run on the same
|
yann@40
|
245 |
machine it is compiled on, and generate code to run on a second machine,
|
yann@40
|
246 |
the target.
|
yann@197
|
247 |
crosstool-NG calls it "cross".
|
yann@40
|
248 |
|
yann@40
|
249 |
3) build != host == target
|
yann@40
|
250 |
Such a toolchain is also a native toolchain, as it targets the same machine
|
yann@40
|
251 |
as it runs on. But it is build on another machine. You want such a
|
yann@40
|
252 |
toolchain when porting to a new architecture, or if the build machine is
|
yann@40
|
253 |
much faster than the host machine.
|
yann@197
|
254 |
crosstool-NG calls it "cross-native".
|
yann@40
|
255 |
|
yann@40
|
256 |
4) build != host != target
|
yann@92
|
257 |
This one is called a canadian-toolchain (*), and is tricky. The three
|
yann@40
|
258 |
machines in play are different. You might want such a toolchain if you
|
yann@40
|
259 |
have a fast build machine, but the users will use it on another machine,
|
yann@40
|
260 |
and will produce code to run on a third machine.
|
yann@197
|
261 |
crosstool-NG calls it "canadian".
|
yann@40
|
262 |
|
yann@197
|
263 |
crosstool-NG can build all these kinds of toolchains (or is aiming at it,
|
yann@197
|
264 |
anyway!)
|
yann@40
|
265 |
|
yann@40
|
266 |
(*) The term Canadian Cross came about because at the time that these issues
|
yann@40
|
267 |
were all being hashed out, Canada had three national political parties.
|
yann@40
|
268 |
http://en.wikipedia.org/wiki/Cross_compiler
|
yann@40
|
269 |
|
yann@1
|
270 |
_____________
|
yann@1
|
271 |
/
|
yann@1
|
272 |
Internals /
|
yann@1
|
273 |
__________/
|
yann@1
|
274 |
|
yann@92
|
275 |
Internally, crosstool-NG is script-based. To ease usage, the frontend is
|
yann@92
|
276 |
Makefile-based.
|
yann@92
|
277 |
|
yann@92
|
278 |
Makefile front-end |
|
yann@92
|
279 |
-------------------*
|
yann@92
|
280 |
|
yann@203
|
281 |
The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
|
yann@203
|
282 |
script with an action will act exactly as if the Makefile was in the current
|
yann@203
|
283 |
working directory and make was called with the action as rule. Thus:
|
yann@203
|
284 |
ct-ng menuconfig
|
yann@203
|
285 |
is equivalent to having the Makefile in CWD, and calling:
|
yann@203
|
286 |
make menuconfig
|
yann@203
|
287 |
|
yann@203
|
288 |
Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
|
yann@203
|
289 |
traditional command.
|
yann@203
|
290 |
|
yann@203
|
291 |
ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
|
yann@203
|
292 |
at configuration time with ./configure.
|
yann@203
|
293 |
|
yann@203
|
294 |
ct-ng also search for config files, sub-tools, samples, scripts and patches in
|
yann@203
|
295 |
that library directory.
|
yann@92
|
296 |
|
yann@182
|
297 |
Kconfig parser |
|
yann@182
|
298 |
---------------*
|
yann@92
|
299 |
|
yann@92
|
300 |
The kconfig language is a hacked version, vampirised from the toybox project
|
yann@182
|
301 |
by Rob LANDLEY (http://www.landley.net/code/toybox/), itself coming from the
|
yann@182
|
302 |
Linux kernel (http://www.linux.org/ http://www.kernel.org/), and (heavily)
|
yann@182
|
303 |
adapted to my needs.
|
yann@92
|
304 |
|
yann@203
|
305 |
The kconfig parsers (conf and mconf) are not installed pre-built, but as
|
yann@203
|
306 |
source files. Thus you can have the directory where crosstool-NG is installed,
|
yann@203
|
307 |
exported (via NFS or whatever) and have clients with different architectures
|
yann@203
|
308 |
use the same crosstool-NG installation, and most notably, the same set of
|
yann@203
|
309 |
patches.
|
yann@203
|
310 |
|
yann@203
|
311 |
Build scripts |
|
yann@203
|
312 |
--------------*
|
yann@203
|
313 |
|
yann@203
|
314 |
To Be Written later...
|