docs/8 - Internals.txt
author "Yann E. MORIN" <yann.morin.1998@free.fr>
Thu Jan 31 21:07:48 2013 +0100 (2013-01-31)
branch1.18
changeset 3179 45a89104a20c
parent 2564 5d4e91c0343e
permissions -rw-r--r--
1.18: update version to 1.18.0+hg

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
     1 File.........: 8 - Internals.txt
     2 Copyright....: (C) 2010 Yann E. MORIN <yann.morin.1998@free.fr>
     3 License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
     4 
     5 
     6 Internals  /
     7 __________/
     8 
     9 
    10 Internally, crosstool-NG is script-based. To ease usage, the frontend is
    11 Makefile-based.
    12 
    13 
    14 Makefile front-end |
    15 -------------------+
    16 
    17 The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
    18 script with an action will act exactly as if the Makefile was in the current
    19 working directory and make was called with the action as rule. Thus:
    20   ct-ng menuconfig
    21 
    22 is equivalent to having the Makefile in CWD, and calling:
    23   make menuconfig
    24 
    25 Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
    26 traditional command.
    27 
    28 ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
    29 at configuration time with ./configure.
    30 
    31 ct-ng also searches for config files, sub-tools, samples, scripts and patches in
    32 that library directory.
    33 
    34 Because of a stupid make behavior/bug I was unable to track down, implicit make
    35 rules are disabled: installing with --local would trigger those rules, and mconf
    36 was unbuildable.
    37 
    38 
    39 Kconfig parser |
    40 ---------------+
    41 
    42 The kconfig language is a hacked version, vampirised from the Linux kernel
    43 (http://www.kernel.org/), and (heavily) adapted to my needs.
    44 
    45 The list of the most notable changes (at least the ones I remember) follows:
    46 - the CONFIG_ prefix has been replaced with CT_
    47 - a leading | in prompts is skipped, and subsequent leading spaces are not
    48   trimmed; otherwise leading spaces are silently trimmed
    49 - removed the warning about undefined environment variable
    50 
    51 The kconfig parsers (conf and mconf) are not installed pre-built, but as
    52 source files. Thus you can have the directory where crosstool-NG is installed,
    53 exported (via NFS or whatever) and have clients with different architectures
    54 use the same crosstool-NG installation, and most notably, the same set of
    55 patches.
    56 
    57 
    58 Architecture-specific |
    59 ----------------------+
    60 
    61 Note: this chapter is not really well written, and might thus be a little bit
    62 complex to understand. To get a better grasp of what an architecture is, the
    63 reader is kindly encouraged to look at the "arch/" sub-directory, and to the
    64 existing architectures to see how things are laid out.
    65 
    66 An architecture is defined by:
    67 
    68  - a human-readable name, in lower case letters, with numbers as appropriate.
    69    The underscore is allowed; space and special characters are not.
    70      Eg.: arm, x86_64
    71  - a file in "config/arch/", named after the architecture's name, and suffixed
    72    with ".in".
    73      Eg.: config/arch/arm.in
    74  - a file in "scripts/build/arch/", named after the architecture's name, and
    75    suffixed with ".sh".
    76      Eg.: scripts/build/arch/arm.sh
    77 
    78 The architecture's ".in" file API:
    79  > the config option "ARCH_%arch%" (where %arch% is to be replaced with the
    80    actual architecture name).
    81    That config option must have *neither* a type, *nor* a prompt! Also, it can
    82    *not* depend on any other config option (EXPERIMENTAL is managed as above).
    83      Eg.:
    84        config ARCH_arm
    85    + mandatory:
    86        defines a (terse) help entry for this architecture:
    87        Eg.:
    88          config ARCH_arm
    89            help
    90              The ARM architecture.
    91    + optional:
    92        selects adequate associated config options.
    93        Note: 64-bit architectures *shall* select ARCH_64
    94        Eg.:
    95          config ARCH_arm
    96            select ARCH_SUPPORTS_BOTH_ENDIAN
    97            select ARCH_DEFAULT_LE
    98            help
    99              The ARM architecture.
   100        Eg.:
   101          config ARCH_x86_64
   102             select ARCH_64
   103             help
   104               The x86_64 architecture.
   105 
   106  > other target-specific options, at your discretion. Note however that to
   107    avoid name-clashing, such options shall be prefixed with "ARCH_%arch%",
   108    where %arch% is again replaced by the actual architecture name.
   109    (Note: due to historical reasons, and lack of time to clean up the code,
   110     I may have left some config options that do not completely conform to
   111     this, as the architecture name was written all upper case. However, the
   112     prefix is unique among architectures, and does not cause harm).
   113 
   114 The architecture's ".sh" file API:
   115  > the function "CT_DoArchTupleValues"
   116    + parameters: none
   117    + environment:
   118      - all variables from the ".config" file,
   119      - the two variables "target_endian_eb" and "target_endian_el" which are
   120        the endianness suffixes
   121    + return value: 0 upon success, !0 upon failure
   122    + provides:
   123      - mandatory
   124      - the environment variable CT_TARGET_ARCH
   125      - contains:
   126        the architecture part of the target tuple.
   127        Eg.: "armeb" for big endian ARM
   128             "i386" for an i386
   129    + provides:
   130      - optional
   131      - the environment variable CT_TARGET_SYS
   132      - contains:
   133        the system part of the target tuple.
   134        Eg.: "gnu" for glibc on most architectures
   135             "gnueabi" for glibc on an ARM EABI
   136      - defaults to:
   137        - for glibc-based toolchain: "gnu"
   138        - for uClibc-based toolchain: "uclibc"
   139    + provides:
   140      - optional
   141      - the environment variables to configure the cross-gcc (defaults)
   142        - CT_ARCH_WITH_ARCH    : the gcc ./configure switch to select architecture level         ( "--with-arch=${CT_ARCH_ARCH}"   )
   143        - CT_ARCH_WITH_ABI     : the gcc ./configure switch to select ABI level                  ( "--with-abi=${CT_ARCH_ABI}"     )
   144        - CT_ARCH_WITH_CPU     : the gcc ./configure switch to select CPU instruction set        ( "--with-cpu=${CT_ARCH_CPU}"     )
   145        - CT_ARCH_WITH_TUNE    : the gcc ./configure switch to select scheduling                 ( "--with-tune=${CT_ARCH_TUNE}"   )
   146        - CT_ARCH_WITH_FPU     : the gcc ./configure switch to select FPU type                   ( "--with-fpu=${CT_ARCH_FPU}"     )
   147        - CT_ARCH_WITH_FLOAT   : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/  )
   148    + provides:
   149      - optional
   150      - the environment variables to pass to the cross-gcc to build target binaries (defaults)
   151        - CT_ARCH_ARCH_CFLAG   : the gcc switch to select architecture level                     ( "-march=${CT_ARCH_ARCH}"            )
   152        - CT_ARCH_ABI_CFLAG    : the gcc switch to select ABI level                              ( "-mabi=${CT_ARCH_ABI}"              )
   153        - CT_ARCH_CPU_CFLAG    : the gcc switch to select CPU instruction set                    ( "-mcpu=${CT_ARCH_CPU}"              )
   154        - CT_ARCH_TUNE_CFLAG   : the gcc switch to select scheduling                             ( "-mtune=${CT_ARCH_TUNE}"            )
   155        - CT_ARCH_FPU_CFLAG    : the gcc switch to select FPU type                               ( "-mfpu=${CT_ARCH_FPU}"              )
   156        - CT_ARCH_FLOAT_CFLAG  : the gcc switch to choose floating point arithmetics             ( "-msoft-float" or /empty/           )
   157        - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian                   ( "-mbig-endian" or "-mlittle-endian" )
   158      - default to:
   159        see above.
   160    + provides:
   161      - optional
   162      - the environment variables to configure the core and final compiler, specific to this architecture:
   163        - CT_ARCH_CC_CORE_EXTRA_CONFIG   : additional, architecture specific core gcc ./configure flags
   164        - CT_ARCH_CC_EXTRA_CONFIG        : additional, architecture specific final gcc ./configure flags
   165      - default to:
   166        - all empty
   167    + provides:
   168      - optional
   169      - the architecture-specific CFLAGS and LDFLAGS:
   170        - CT_ARCH_TARGET_CLFAGS
   171        - CT_ARCH_TARGET_LDFLAGS
   172      - default to:
   173        - all empty
   174 
   175 You can have a look at "config/arch/arm.in" and "scripts/build/arch/arm.sh" for
   176 a quite complete example of what an actual architecture description looks like.
   177 
   178 
   179 Kernel specific |
   180 ----------------+
   181 
   182 A kernel is defined by:
   183 
   184  - a human-readable name, in lower case letters, with numbers as appropriate.
   185    The underscore is allowed; space and special characters are not (although
   186    they are internally replaced with underscores.
   187      Eg.: linux, bare-metal
   188  - a file in "config/kernel/", named after the kernel name, and suffixed with
   189    ".in".
   190      Eg.: config/kernel/linux.in, config/kernel/bare-metal.in
   191  - a file in "scripts/build/kernel/", named after the kernel name, and suffixed
   192    with ".sh".
   193      Eg.: scripts/build/kernel/linux.sh, scripts/build/kernel/bare-metal.sh
   194 
   195 The kernel's ".in" file must contain:
   196  > an optional lines containing exactly "# EXPERIMENTAL", starting on the
   197    first column, and without any following space or other character.
   198    If this line is present, then this kernel is considered EXPERIMENTAL,
   199    and correct dependency on EXPERIMENTAL will be set.
   200 
   201  > the config option "KERNEL_%kernel_name%" (where %kernel_name% is to be
   202    replaced with the actual kernel name, with all special characters and
   203    spaces replaced by underscores).
   204    That config option must have *neither* a type, *nor* a prompt! Also, it can
   205    *not* depends on EXPERIMENTAL.
   206      Eg.: KERNEL_linux, KERNEL_bare_metal
   207    + mandatory:
   208        defines a (terse) help entry for this kernel.
   209        Eg.:
   210          config KERNEL_bare_metal
   211            help
   212              Build a compiler for use without any kernel.
   213    + optional:
   214        selects adequate associated config options.
   215        Eg.:
   216          config KERNEL_bare_metal
   217            select BARE_METAL
   218            help
   219              Build a compiler for use without any kernel.
   220 
   221  > other kernel specific options, at your discretion. Note however that, to
   222    avoid name-clashing, such options should be prefixed with
   223    "KERNEL_%kernel_name%", where %kernel_name% is again tp be replaced with
   224    the actual kernel name.
   225    (Note: due to historical reasons, and lack of time to clean up the code,
   226     I may have left some config options that do not completely conform to
   227     this, as the kernel name was written all upper case. However, the prefix
   228     is unique among kernels, and does not cause harm).
   229 
   230 The kernel's ".sh" file API:
   231  > is a bash script fragment
   232 
   233  > defines the function CT_DoKernelTupleValues
   234    + see the architecture's CT_DoArchTupleValues, except for:
   235    + set the environment variable CT_TARGET_KERNEL, the kernel part of the
   236      target tuple
   237    + return value: ignored
   238 
   239  > defines the function "do_kernel_get":
   240    + parameters: none
   241    + environment:
   242       - all variables from the ".config" file.
   243    + return value: 0 for success, !0 for failure.
   244    + behavior: download the kernel's sources, and store the tarball into
   245      "${CT_TARBALLS_DIR}". To this end, a functions is available, that
   246      abstracts downloading tarballs:
   247      - CT_DoGet <tarball_base_name> <URL1 [URL...]>
   248        Eg.: CT_DoGet linux-2.6.26.5 ftp://ftp.kernel.org/pub/linux/kernel/v2.6
   249      Note: retrieving sources from svn, cvs, git and the likes is not supported
   250      by CT_DoGet. You'll have to do this by hand, as it is done for eglibc in
   251      "scripts/build/libc/eglibc.sh"
   252 
   253  > defines the function "do_kernel_extract":
   254    + parameters: none
   255    + environment:
   256       - all variables from the ".config" file,
   257    + return value: 0 for success, !0 for failure.
   258    + behavior: extract the kernel's tarball into "${CT_SRC_DIR}", and apply
   259      required patches. To this end, a function is available, that abstracts
   260      extracting tarballs:
   261      - CT_ExtractAndPatch <tarball_base_name>
   262        Eg.: CT_ExtractAndPatch linux-2.6.26.5
   263 
   264  > defines the function "do_kernel_headers":
   265    + parameters: none
   266    + environment:
   267       - all variables from the ".config" file,
   268    + return value: 0 for success, !0 for failure.
   269    + behavior: install the kernel headers (if any) in "${CT_SYSROOT_DIR}/usr/include"
   270 
   271  > defines any kernel-specific helper functions
   272    These functions, if any, must be prefixed with "do_kernel_%CT_KERNEL%_",
   273    where '%CT_KERNEL%' is to be replaced with the actual kernel name, to avoid
   274    any name-clashing.
   275 
   276 You can have a look at "config/kernel/linux.in" and "scripts/build/kernel/linux.sh"
   277 as an example of what a complex kernel description looks like.
   278 
   279 
   280 Adding a new version of a component |
   281 ------------------------------------+
   282 
   283 When a new component, such as the Linux kernel, gcc or any other is released,
   284 adding the new version to crosstool-NG is quite easy. There is a script that
   285 will do all that for you:
   286   scripts/addToolVersion.sh
   287 
   288 Run it with no option to get some help.
   289 
   290 
   291 Build scripts |
   292 --------------+
   293 
   294 To Be Written later...