docs/8 - Internals.txt
author "Yann E. MORIN" <yann.morin.1998@free.fr>
Sat Mar 17 23:43:59 2012 +0100 (2012-03-17)
changeset 2920 d5f5cc186d26
parent 2564 5d4e91c0343e
permissions -rw-r--r--
kernel/linux: update long-term versions

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
yann@2076
     1
File.........: 8 - Internals.txt
yann@2908
     2
Copyright....: (C) 2010 Yann E. MORIN <yann.morin.1998@free.fr>
yann@2076
     3
License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
yann@2076
     4
yann@2076
     5
yann@2076
     6
Internals  /
yann@2076
     7
__________/
yann@2076
     8
yann@2076
     9
yann@2076
    10
Internally, crosstool-NG is script-based. To ease usage, the frontend is
yann@2076
    11
Makefile-based.
yann@2076
    12
yann@2076
    13
yann@2076
    14
Makefile front-end |
yann@2076
    15
-------------------+
yann@2076
    16
yann@2076
    17
The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
yann@2076
    18
script with an action will act exactly as if the Makefile was in the current
yann@2076
    19
working directory and make was called with the action as rule. Thus:
yann@2076
    20
  ct-ng menuconfig
yann@2076
    21
yann@2076
    22
is equivalent to having the Makefile in CWD, and calling:
yann@2076
    23
  make menuconfig
yann@2076
    24
yann@2076
    25
Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
yann@2076
    26
traditional command.
yann@2076
    27
yann@2076
    28
ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
yann@2076
    29
at configuration time with ./configure.
yann@2076
    30
yann@2076
    31
ct-ng also searches for config files, sub-tools, samples, scripts and patches in
yann@2076
    32
that library directory.
yann@2076
    33
yann@2076
    34
Because of a stupid make behavior/bug I was unable to track down, implicit make
antony@2564
    35
rules are disabled: installing with --local would trigger those rules, and mconf
yann@2076
    36
was unbuildable.
yann@2076
    37
yann@2076
    38
yann@2076
    39
Kconfig parser |
yann@2076
    40
---------------+
yann@2076
    41
yann@2076
    42
The kconfig language is a hacked version, vampirised from the Linux kernel
yann@2076
    43
(http://www.kernel.org/), and (heavily) adapted to my needs.
yann@2076
    44
yann@2076
    45
The list of the most notable changes (at least the ones I remember) follows:
yann@2076
    46
- the CONFIG_ prefix has been replaced with CT_
yann@2076
    47
- a leading | in prompts is skipped, and subsequent leading spaces are not
yann@2076
    48
  trimmed; otherwise leading spaces are silently trimmed
yann@2076
    49
- removed the warning about undefined environment variable
yann@2076
    50
yann@2076
    51
The kconfig parsers (conf and mconf) are not installed pre-built, but as
yann@2076
    52
source files. Thus you can have the directory where crosstool-NG is installed,
yann@2076
    53
exported (via NFS or whatever) and have clients with different architectures
yann@2076
    54
use the same crosstool-NG installation, and most notably, the same set of
yann@2076
    55
patches.
yann@2076
    56
yann@2076
    57
yann@2076
    58
Architecture-specific |
yann@2076
    59
----------------------+
yann@2076
    60
yann@2076
    61
Note: this chapter is not really well written, and might thus be a little bit
yann@2076
    62
complex to understand. To get a better grasp of what an architecture is, the
yann@2076
    63
reader is kindly encouraged to look at the "arch/" sub-directory, and to the
yann@2076
    64
existing architectures to see how things are laid out.
yann@2076
    65
yann@2076
    66
An architecture is defined by:
yann@2076
    67
yann@2076
    68
 - a human-readable name, in lower case letters, with numbers as appropriate.
yann@2076
    69
   The underscore is allowed; space and special characters are not.
yann@2076
    70
     Eg.: arm, x86_64
yann@2076
    71
 - a file in "config/arch/", named after the architecture's name, and suffixed
yann@2076
    72
   with ".in".
yann@2076
    73
     Eg.: config/arch/arm.in
yann@2076
    74
 - a file in "scripts/build/arch/", named after the architecture's name, and
yann@2076
    75
   suffixed with ".sh".
yann@2076
    76
     Eg.: scripts/build/arch/arm.sh
yann@2076
    77
yann@2076
    78
The architecture's ".in" file API:
yann@2076
    79
 > the config option "ARCH_%arch%" (where %arch% is to be replaced with the
yann@2076
    80
   actual architecture name).
yann@2076
    81
   That config option must have *neither* a type, *nor* a prompt! Also, it can
yann@2076
    82
   *not* depend on any other config option (EXPERIMENTAL is managed as above).
yann@2076
    83
     Eg.:
yann@2076
    84
       config ARCH_arm
yann@2076
    85
   + mandatory:
yann@2076
    86
       defines a (terse) help entry for this architecture:
yann@2076
    87
       Eg.:
yann@2076
    88
         config ARCH_arm
yann@2076
    89
           help
yann@2076
    90
             The ARM architecture.
yann@2076
    91
   + optional:
yann@2076
    92
       selects adequate associated config options.
yann@2076
    93
       Note: 64-bit architectures *shall* select ARCH_64
yann@2076
    94
       Eg.:
yann@2076
    95
         config ARCH_arm
yann@2076
    96
           select ARCH_SUPPORTS_BOTH_ENDIAN
yann@2076
    97
           select ARCH_DEFAULT_LE
yann@2076
    98
           help
yann@2076
    99
             The ARM architecture.
yann@2076
   100
       Eg.:
yann@2076
   101
         config ARCH_x86_64
yann@2076
   102
            select ARCH_64
yann@2076
   103
            help
yann@2076
   104
              The x86_64 architecture.
yann@2076
   105
yann@2076
   106
 > other target-specific options, at your discretion. Note however that to
yann@2076
   107
   avoid name-clashing, such options shall be prefixed with "ARCH_%arch%",
yann@2076
   108
   where %arch% is again replaced by the actual architecture name.
yann@2076
   109
   (Note: due to historical reasons, and lack of time to clean up the code,
yann@2076
   110
    I may have left some config options that do not completely conform to
yann@2076
   111
    this, as the architecture name was written all upper case. However, the
yann@2076
   112
    prefix is unique among architectures, and does not cause harm).
yann@2076
   113
yann@2076
   114
The architecture's ".sh" file API:
yann@2076
   115
 > the function "CT_DoArchTupleValues"
yann@2076
   116
   + parameters: none
yann@2076
   117
   + environment:
yann@2076
   118
     - all variables from the ".config" file,
yann@2076
   119
     - the two variables "target_endian_eb" and "target_endian_el" which are
yann@2076
   120
       the endianness suffixes
yann@2076
   121
   + return value: 0 upon success, !0 upon failure
yann@2076
   122
   + provides:
yann@2076
   123
     - mandatory
yann@2076
   124
     - the environment variable CT_TARGET_ARCH
yann@2076
   125
     - contains:
yann@2076
   126
       the architecture part of the target tuple.
yann@2076
   127
       Eg.: "armeb" for big endian ARM
yann@2076
   128
            "i386" for an i386
yann@2076
   129
   + provides:
yann@2076
   130
     - optional
yann@2076
   131
     - the environment variable CT_TARGET_SYS
yann@2076
   132
     - contains:
antony@2564
   133
       the system part of the target tuple.
yann@2076
   134
       Eg.: "gnu" for glibc on most architectures
yann@2076
   135
            "gnueabi" for glibc on an ARM EABI
yann@2076
   136
     - defaults to:
yann@2076
   137
       - for glibc-based toolchain: "gnu"
yann@2076
   138
       - for uClibc-based toolchain: "uclibc"
yann@2076
   139
   + provides:
yann@2076
   140
     - optional
yann@2076
   141
     - the environment variables to configure the cross-gcc (defaults)
yann@2076
   142
       - CT_ARCH_WITH_ARCH    : the gcc ./configure switch to select architecture level         ( "--with-arch=${CT_ARCH_ARCH}"   )
yann@2076
   143
       - CT_ARCH_WITH_ABI     : the gcc ./configure switch to select ABI level                  ( "--with-abi=${CT_ARCH_ABI}"     )
yann@2076
   144
       - CT_ARCH_WITH_CPU     : the gcc ./configure switch to select CPU instruction set        ( "--with-cpu=${CT_ARCH_CPU}"     )
yann@2076
   145
       - CT_ARCH_WITH_TUNE    : the gcc ./configure switch to select scheduling                 ( "--with-tune=${CT_ARCH_TUNE}"   )
yann@2076
   146
       - CT_ARCH_WITH_FPU     : the gcc ./configure switch to select FPU type                   ( "--with-fpu=${CT_ARCH_FPU}"     )
yann@2076
   147
       - CT_ARCH_WITH_FLOAT   : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/  )
yann@2076
   148
   + provides:
yann@2076
   149
     - optional
yann@2076
   150
     - the environment variables to pass to the cross-gcc to build target binaries (defaults)
yann@2076
   151
       - CT_ARCH_ARCH_CFLAG   : the gcc switch to select architecture level                     ( "-march=${CT_ARCH_ARCH}"            )
yann@2076
   152
       - CT_ARCH_ABI_CFLAG    : the gcc switch to select ABI level                              ( "-mabi=${CT_ARCH_ABI}"              )
yann@2076
   153
       - CT_ARCH_CPU_CFLAG    : the gcc switch to select CPU instruction set                    ( "-mcpu=${CT_ARCH_CPU}"              )
yann@2076
   154
       - CT_ARCH_TUNE_CFLAG   : the gcc switch to select scheduling                             ( "-mtune=${CT_ARCH_TUNE}"            )
yann@2076
   155
       - CT_ARCH_FPU_CFLAG    : the gcc switch to select FPU type                               ( "-mfpu=${CT_ARCH_FPU}"              )
yann@2076
   156
       - CT_ARCH_FLOAT_CFLAG  : the gcc switch to choose floating point arithmetics             ( "-msoft-float" or /empty/           )
yann@2076
   157
       - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian                   ( "-mbig-endian" or "-mlittle-endian" )
yann@2076
   158
     - default to:
yann@2076
   159
       see above.
yann@2076
   160
   + provides:
yann@2076
   161
     - optional
antony@2564
   162
     - the environment variables to configure the core and final compiler, specific to this architecture:
yann@2076
   163
       - CT_ARCH_CC_CORE_EXTRA_CONFIG   : additional, architecture specific core gcc ./configure flags
yann@2076
   164
       - CT_ARCH_CC_EXTRA_CONFIG        : additional, architecture specific final gcc ./configure flags
yann@2076
   165
     - default to:
yann@2076
   166
       - all empty
yann@2076
   167
   + provides:
yann@2076
   168
     - optional
yann@2076
   169
     - the architecture-specific CFLAGS and LDFLAGS:
yann@2076
   170
       - CT_ARCH_TARGET_CLFAGS
yann@2076
   171
       - CT_ARCH_TARGET_LDFLAGS
yann@2076
   172
     - default to:
yann@2076
   173
       - all empty
yann@2076
   174
yann@2076
   175
You can have a look at "config/arch/arm.in" and "scripts/build/arch/arm.sh" for
yann@2076
   176
a quite complete example of what an actual architecture description looks like.
yann@2076
   177
yann@2076
   178
yann@2076
   179
Kernel specific |
yann@2076
   180
----------------+
yann@2076
   181
yann@2076
   182
A kernel is defined by:
yann@2076
   183
yann@2076
   184
 - a human-readable name, in lower case letters, with numbers as appropriate.
yann@2076
   185
   The underscore is allowed; space and special characters are not (although
yann@2076
   186
   they are internally replaced with underscores.
yann@2076
   187
     Eg.: linux, bare-metal
yann@2076
   188
 - a file in "config/kernel/", named after the kernel name, and suffixed with
yann@2076
   189
   ".in".
yann@2076
   190
     Eg.: config/kernel/linux.in, config/kernel/bare-metal.in
yann@2076
   191
 - a file in "scripts/build/kernel/", named after the kernel name, and suffixed
yann@2076
   192
   with ".sh".
yann@2076
   193
     Eg.: scripts/build/kernel/linux.sh, scripts/build/kernel/bare-metal.sh
yann@2076
   194
yann@2076
   195
The kernel's ".in" file must contain:
yann@2076
   196
 > an optional lines containing exactly "# EXPERIMENTAL", starting on the
yann@2076
   197
   first column, and without any following space or other character.
yann@2076
   198
   If this line is present, then this kernel is considered EXPERIMENTAL,
yann@2076
   199
   and correct dependency on EXPERIMENTAL will be set.
yann@2076
   200
yann@2076
   201
 > the config option "KERNEL_%kernel_name%" (where %kernel_name% is to be
yann@2076
   202
   replaced with the actual kernel name, with all special characters and
yann@2076
   203
   spaces replaced by underscores).
yann@2076
   204
   That config option must have *neither* a type, *nor* a prompt! Also, it can
yann@2076
   205
   *not* depends on EXPERIMENTAL.
yann@2076
   206
     Eg.: KERNEL_linux, KERNEL_bare_metal
yann@2076
   207
   + mandatory:
yann@2076
   208
       defines a (terse) help entry for this kernel.
yann@2076
   209
       Eg.:
yann@2076
   210
         config KERNEL_bare_metal
yann@2076
   211
           help
yann@2076
   212
             Build a compiler for use without any kernel.
yann@2076
   213
   + optional:
yann@2076
   214
       selects adequate associated config options.
yann@2076
   215
       Eg.:
yann@2076
   216
         config KERNEL_bare_metal
yann@2076
   217
           select BARE_METAL
yann@2076
   218
           help
yann@2076
   219
             Build a compiler for use without any kernel.
yann@2076
   220
yann@2076
   221
 > other kernel specific options, at your discretion. Note however that, to
yann@2076
   222
   avoid name-clashing, such options should be prefixed with
yann@2076
   223
   "KERNEL_%kernel_name%", where %kernel_name% is again tp be replaced with
yann@2076
   224
   the actual kernel name.
yann@2076
   225
   (Note: due to historical reasons, and lack of time to clean up the code,
yann@2076
   226
    I may have left some config options that do not completely conform to
yann@2076
   227
    this, as the kernel name was written all upper case. However, the prefix
yann@2076
   228
    is unique among kernels, and does not cause harm).
yann@2076
   229
yann@2076
   230
The kernel's ".sh" file API:
yann@2076
   231
 > is a bash script fragment
yann@2076
   232
yann@2076
   233
 > defines the function CT_DoKernelTupleValues
yann@2076
   234
   + see the architecture's CT_DoArchTupleValues, except for:
yann@2076
   235
   + set the environment variable CT_TARGET_KERNEL, the kernel part of the
yann@2076
   236
     target tuple
yann@2076
   237
   + return value: ignored
yann@2076
   238
yann@2076
   239
 > defines the function "do_kernel_get":
yann@2076
   240
   + parameters: none
yann@2076
   241
   + environment:
yann@2076
   242
      - all variables from the ".config" file.
yann@2076
   243
   + return value: 0 for success, !0 for failure.
yann@2076
   244
   + behavior: download the kernel's sources, and store the tarball into
yann@2076
   245
     "${CT_TARBALLS_DIR}". To this end, a functions is available, that
yann@2076
   246
     abstracts downloading tarballs:
yann@2076
   247
     - CT_DoGet <tarball_base_name> <URL1 [URL...]>
yann@2076
   248
       Eg.: CT_DoGet linux-2.6.26.5 ftp://ftp.kernel.org/pub/linux/kernel/v2.6
yann@2076
   249
     Note: retrieving sources from svn, cvs, git and the likes is not supported
yann@2076
   250
     by CT_DoGet. You'll have to do this by hand, as it is done for eglibc in
yann@2076
   251
     "scripts/build/libc/eglibc.sh"
yann@2076
   252
yann@2076
   253
 > defines the function "do_kernel_extract":
yann@2076
   254
   + parameters: none
yann@2076
   255
   + environment:
yann@2076
   256
      - all variables from the ".config" file,
yann@2076
   257
   + return value: 0 for success, !0 for failure.
yann@2076
   258
   + behavior: extract the kernel's tarball into "${CT_SRC_DIR}", and apply
yann@2076
   259
     required patches. To this end, a function is available, that abstracts
yann@2076
   260
     extracting tarballs:
yann@2076
   261
     - CT_ExtractAndPatch <tarball_base_name>
yann@2076
   262
       Eg.: CT_ExtractAndPatch linux-2.6.26.5
yann@2076
   263
yann@2076
   264
 > defines the function "do_kernel_headers":
yann@2076
   265
   + parameters: none
yann@2076
   266
   + environment:
yann@2076
   267
      - all variables from the ".config" file,
yann@2076
   268
   + return value: 0 for success, !0 for failure.
yann@2076
   269
   + behavior: install the kernel headers (if any) in "${CT_SYSROOT_DIR}/usr/include"
yann@2076
   270
yann@2076
   271
 > defines any kernel-specific helper functions
yann@2076
   272
   These functions, if any, must be prefixed with "do_kernel_%CT_KERNEL%_",
yann@2076
   273
   where '%CT_KERNEL%' is to be replaced with the actual kernel name, to avoid
yann@2076
   274
   any name-clashing.
yann@2076
   275
yann@2076
   276
You can have a look at "config/kernel/linux.in" and "scripts/build/kernel/linux.sh"
yann@2076
   277
as an example of what a complex kernel description looks like.
yann@2076
   278
yann@2076
   279
yann@2076
   280
Adding a new version of a component |
yann@2076
   281
------------------------------------+
yann@2076
   282
yann@2076
   283
When a new component, such as the Linux kernel, gcc or any other is released,
yann@2076
   284
adding the new version to crosstool-NG is quite easy. There is a script that
yann@2076
   285
will do all that for you:
yann@2076
   286
  scripts/addToolVersion.sh
yann@2076
   287
yann@2076
   288
Run it with no option to get some help.
yann@2076
   289
yann@2076
   290
yann@2076
   291
Build scripts |
yann@2076
   292
--------------+
yann@2076
   293
yann@2076
   294
To Be Written later...