GCC Cross-Compiler

From OSDev.wiki
Jump to navigation Jump to search
Difficulty level

Beginner

Introduction

About

Generally speaking, a cross-compiler is a compiler that runs on platform A (the host), but generates executables for platform B (the target). These two platforms may (but do not need to) differ in CPU, operating system, and/or executable format.

What isn't covered

If

  • ... you are using Gentoo Linux,
  • ... you want to make a Canadian Cross,
  • ... you want to make a cross-compiler to compile applications (rather than OS development),
  • ... you want to make a cross-compiler for a 64-bit target (TARGET=x86_64-elf) using a version older than GCC 4.3.2/Binutils 2.17.

... then please refer to the See Also at the bottom of this page.

Why do we need one?

Creating a dedicated (cross-)compiler for your OS development work can save you many headaches. If

  • ... your system compiler drags in references to alloca() or other OS-dependent things,
  • ... the linker (ld) complains about "PE operation on non-PE file",
  • ... your compiler and your assembler can't agree on binary formats ("unresolved reference to _kmain()"),
  • ... your bootloader stubbornly insists that it can't read your kernel binary,
  • ... you are using Mac OS,

... the most effective solution is to set up a dedicated cross-compiler. It also avoids accidentally using files available to your system compiler that are not applicable to your OS development work (standard includes, C runtime (crt0.o), etc.)

At the very least, using a cross-compiler places you on the same playground as other users: you can rest assured that any problems you might encounter are not specific to your compiler setup.

See also Why do I need a Cross Compiler?

How this document is organized

We describe a sequence of steps, starting with nothing but your system compiler and ending with a native compiler for the target system. You might not need all those steps (when you want to compile your hobbyist OS into a binary, Step 1 is all you need). Actually, at this point only Step 1 is actually covered.

Please note that we build everything out of the source directory tree, as is considered good practice. Some packages only support building outside, some only inside and some both (but may not offer extensive checking with make). Building GCC inside the source directory tree fails miserably, at least for older versions.

Requirements

  • A host system with a working GCC installation, and enough memory as well as hard drive space. How much qualifies as "enough" is depending on the versions of the software involved, but GCC is a big piece of software, so don't be surprised when 128 or 256 MByte are not sufficient.
  • A bash shell or comparable environment. If you are not using a bash shell, you might have to modify some of the command lines below. If you have just installed the basic Cygwin package, you have to run the setup.exe again and install the following packages:
    • GCC (even if you have something like MinGW installed)
    • Make
    • Flex
    • Bison
  • If you plan to use GCC 4.3.0 or a later version, you will also have to install the following (using your system's package management):
    • GNU GMP (libgmp-devel on Cygwin, libgmp3-dev on apt-based systems, dev-libs/gmp on Gentoo, gmp-devel on Fedora, or use this tutorial on MSYS or MinGW.
    • GNU MPFR (libmpfr-devel on Cygwin, libmpfr-dev on apt-based systems, dev-libs/mpfr on Gentoo, mpfr-devel on Fedora)
  • If you plan to use GCC 4.5.0 or a later version, you will also have to install the following (using your system's package management):
    • MPC (libmpc-devel on Cygwin, libmpc-dev on apt-based systems, dev-libs/mpc on Gentoo, libmpc-devel on Fedora)
  • MinGW + MSYS is an option, and as it addresses the native Windows API instead of a POSIX emulation layer, results in a slightly faster but not quite as stable toolchain. Some software packages will not build properly under MSYS as they were not designed for use with Windows. As far as this tutorial is concerned, everything that applies to Cygwin also applies to MSYS unless otherwise specified.

Note: Cygwin includes your Windows %PATH% in its bash $PATH. If you were using DJGPP before, this could result in confusion as e.g. calling "gcc" on the Cygwin bash command line would still call the DJGPP compiler. After uninstalling DJGPP, you should delete the DJGPP environment variable and clear the C:\djgpp entry (or wherever you installed it) from your %PATH%.

Tested on...

For a full list of tested version combinations, see Cross-Compiler Successful Builds.

If you are compiling for the arm-elf target, make sure you don't use version 2.20 of Binutils as it fails to compile.

When compiling GCC 4.3 or higher on OS X 10.4 and 10.5, you may get unresolved symbol errors related to libiconv. This is because the version shipped with OS X is seriously out of date. Install a new version (compile it yourself or use macports) and add --with-libiconv-prefix=/opt/local (or /usr/local if you compiled it yourself) to GCC's ./configure line. Alternatively you may place the libiconv source in gcc-x.y.z/libiconv and it will be compiled as part of the GCC compilation process. (This trick also works for mpfr and gmp).

Step 1 - Bootstrap

We build a toolset running on your host that can turn source code into object files for your target system.

We need the binutils and the GCC packages from http://ftp.gnu.org/gnu/. Download them to /usr/src (or wherever you think appropriate), and unpack them. (Cygwin users: Do not try to build a cross-compiler from Cygwin's source packages. Use the vanilla source packages from gnu.org; the Cygwin packages are known to break in a cross-compiler build.)

You don't have to download the whole big gcc-x.x.x package - gcc-core is sufficient to build the C compiler. If you also want C++, download both gcc-core and gcc-g++, and unpack them in the same directory. (The remaining 12 megabyte or so in the main package are for Fortran, ADA, Java, Objective-C, and the test suite.)

Note: The versioning scheme used is that each fullstop separates a full number, i.e. binutils 2.20.0 is newer than 2.9.0. This may be confusing, if you have not encountered this (quite common) versioning scheme yet, when looking at an alphanumerically sorted list of tarballs: The file at the bottom of the list is not the latest version!

Preparation

export PREFIX=/usr/local/cross
export TARGET=i586-elf
cd /usr/src
mkdir build-binutils build-gcc

The prefix will configure the build process so that all the files of your cross-compiler environment end up in /usr/local/cross. You can change that prefix to whatever you like (e.g., /opt/cross or $HOME/cross would be options). Technically, you could even install directly to /usr, so that your cross-compiler would reside alongside your system compiler, but that is not recommended for several reasons (like risking to overwrite your system compiler if you get TARGET wrong, or getting into conflict with your system's package management).

This tutorial has been shown to work in the same way for the x86_64-elf target, for building 64 bit executables (GCC 4.3.x). In order to do this, simply export TARGET as x86_64-elf instead of i586-elf.

MacOS users, beware

The makefiles of binutils and GCC use the `$(CC)` variable to invoke the compiler. On MacOS, this resolves to `gcc` by default, which is actually not the "real thing", but `llvm-gcc`. That wouldn't be so bad, but `llvm-gcc` is not able to compile the GCC sources into a functional binary. (Bugs have been reported, but that doesn't help you here and now.)

MacOS does have a "real" GCC installed, too. You just have to tell `make` to use it instead of `llvm-gcc`: 

# This is only necessary for MacOS users.
export CC=/usr/bin/gcc-4.2
export CXX=/usr/bin/g++-4.2
export CPP=/usr/bin/cpp-4.2
export LD=/usr/bin/gcc-4.2

You might want to unset these exports once you compiled and installed the cross compiler, as it might confuse other builds. Do not make these permanent...!

binutils

cd /usr/src/build-binutils
../binutils-x.xx/configure --target=$TARGET --prefix=$PREFIX --disable-nls
make all
make install

This compiles the binutils (assembler, disassembler, and various other useful stuff), runnable on your system but handling code in the format specified by $TARGET.

--disable-nls tells binutils not to include native language support. This is basically optional, but reduces dependencies and compile time. It will also result in English-language diagnostics, which the people on the Forum understand when you ask your questions. ;-)

gcc

Now, you can build GCC. 

cd /usr/src/build-gcc
export PATH=$PATH:$PREFIX/bin
../gcc-x.x.x/configure --target=$TARGET --prefix=$PREFIX --disable-nls \
    --enable-languages=c,c++ --without-headers
make all-gcc
make install-gcc

The path has to be extended since GCC needs the binutils we built earlier at some point of the build process. You might want to add these extensions to your $PATH permanently, so you won't have to use fully qualified path names every time you call your cross-compiler.

--disable-nls is the same as for binutils above.

--without-headers tells GCC not to rely on any C library (standard or runtime) being present for the target.

--enable-languages tells GCC not to compile all the other language frontends it supports, but only C (and optionally C++).

Summary

Now you have a "naked" cross-compiler. It does not have access to a C library or C runtime yet, so you cannot use any of the standard includes or create runnable binaries. But it is quite sufficient to compile your self-made kernel.

Usage

Once you are finished, your toolset resides in /usr/local/cross. For example, you have a gcc executable in /usr/local/cross/bin/$TARGET-gcc (and /usr/local/cross/$TARGET/gcc as well), which spits out binaries for your TARGET. Add /usr/local/cross/bin to your PATH environment variable, so that gcc invokes your system compiler, and $TARGET-gcc invokes your cross-compiler. (Note that you have to prefix any of the binutils just as well, i.e. $TARGET-ld, $TARGET-ar, $TARGET-objdump etc.)


libgcc

As a useful addition, you may also like to build libgcc, the GCC low-level runtime library. Linking against libgcc provides integer, floating point, decimal, stack unwinding (useful for exception handling) and other support functions. Once you have built and installed the GCC Cross-Compiler, keep your Bash window open and type: 

make all-target-libgcc
make install-target-libgcc

Troubleshooting

In general, verify that you typed the "make" commands precisely. If your shell crashes during a "make" you need to re-enter any PATH export before running "make" again. If a compilation seems to have gotten really messed up, type "make clean", and then start the make process over again (but you probably do not need to rerun "configure").

line 11: $'\r': command not found

You unpacked the source with WinZIP or something else that converts all text files to the Windows CRLF format. Use 7-Zip or the Cygwin tar program (tar xjf packagename.tar.bz2) to do your unpacking.

Note: On some browsers, when you download a "tarball" that ends in ".tar.bz2", your browser may rename it to ".tar.tar" -- don't worry, it will still work, just rename it back to ".tar.bz2".

i586-elf-ar not found

You forgot to set the executable path ($PATH) to include $PREFIX/bin.

Error: junk at end of line, first unrecognized character is ','

This, in combination with lots of other assembly-level error messages (like, Warning: .type pseudo-op used outside of .def/.endef ignored, or Error: unknown pseudo-op: '.local' ) results when you did not correctly set the --prefix=$PREFIX during the binutils configure.

Another possibility is that you did configure, compile and install your cross-compiler correctly, but don't actually use it. Check the "Usage" section above.

If you try compiling in 64-bit windows, you will receive a "Unknown host machine type" error when running configure. To fix this, scroll up in your shell until right after you entered the configure command and you will see a website which will show you where to download updated files to guess host type. Put them in the root directory of where your source files are located. With GCC version 3.4.0 you will have to override the host environment though, as it does not support being compiled with x86_64-unknown-cygwin . Add the command line argument --host=i686-unknown-cygwin to the configure line for GCC. --CjMovie (Too many edits to get this right...)

Configure: error: invalid feature name: nls

If you are using Cygwin, it must be set to use Unix/binary as the file mode. You can set this by running setup.exe and selecting the appropriate mode.

*** Multiple patterns ...

This error can occur when you're using Cygwin (it could also occur on other instances). The most common cause is the cross-usage of UNIX-style paths and DOS-like paths (e.g. /usr/src and C:\MyFolder). If you were using a custom directory (e.g. C:\Cross-Compiler) to build the cross-compiler instead of the /usr/src directory, usually using the default folders (/usr/src, /usr/...) fixes these problems.

But I'm using Cygwin on Windows and I don't have any /usr/src folder on my hard-drive!

If you didn't notice, Cygwin is a tool that works on Windows but acts UNIX-like, so if you run the 'Cygwin.bat' file in your Cygwin folder, you will get the BASH shell. If you snoop around in your Cygwin folder, you'll see a usr folder with a src folder inside (these folders will be used by Cygwin if you follow this tutorial).

Step 2 - C Library

In the second step, we will build the C library for your target system.

The cross-compiler from Step 1 will spit errors whenever you want to #include any of the standard headers (except for a select few that actually are platform-independent, and generated by the compiler itself). This is quite correct - you don't have a standard library for the target system yet!

The C standard defines two different kinds of executing environments - "freestanding" and "hosted". While the definition might be rather fuzzy for the average application programmer, it is pretty clear-cut when you're doing OS development: A kernel is "freestanding", everything you do in user space is "hosted".

A "freestanding" environment needs to provide only a subset of the C library: float.h, iso646.h, limits.h, stdarg.h, stdbool.h, stddef.h, and stdint.h (as of C99). All of these consist of typedef s and #define s "only", so you can implement them without a single .c file in sight.

You will probably want to add a "hosted" C library, so you can use all the nice #include s it provides. You have two choices - writing your own (not recommended, it's one big chunk of work all in its own), or porting an existing one.

You have to realize that a standard library needs to call the kernel in many places. While malloc() can pass out chunks of memory it manages internally, it has to call upon the kernel to actually get any memory it can manage. printf() sure can format your output, but it still has to call upon the kernel to actually print that output anywhere. fopen() does all the C-specific file management for you, but it has to call upon the kernel to actually provide access to a file. And so it goes on and on.

Well-known available C libraries are the GNU libc and newlib. Newlib is most likely the easier one to port. Another alternative would be building on PDPCLIB (which is C90-compliant and public domain), or seeing if what Solar finished of the PDCLib already is good enough for you (the latter one being aimed at the C99 standard and maximum ease of portability for the very purpose of hobbyist OS development, but currently still missing floats, wide char, and locale support).

Once you have done that, go back to the above how-to, and recompile your GCC environment using the --with-headers option to tell it where to find the headers of your C library. You now have access to the standard library, but you still can't compile standalone executables, since you are still without a C (C++) runtime.

You can try the OS Specific Toolchain or Porting Newlib tutorials for help with porting a C library to your OS.

Step 3 - Full Cross-Compiler

In the third step, we will build a "complete" cross-compiler, which can create not only object files, but standalone executables. For that, you will need a C (C++) runtime that sets up the process environment for an executable, i.e. all the stuff that happens before int main(). This is highly platform-specific, so we can only give an overview of the steps involved.

See OS Specific Toolchain for a tutorial on creating a full cross compiler.

Step 4 - Native Compiler

In the fourth step, we will "bootstrap" a native GCC, that not only compiles for the target, but also runs on the target. This is basically the ticket for leaving your host OS behind, and doing all your development work on your own OS! (Of course, you need an editor and a shell environment to actually enjoy this, but you sure did that first thing after finishing Step 3 above, didn't you...?)

...to be extended. This is the only place where --disable-shared really makes sense.

The --disable-shared option

This is an option to the GCC configure process that you might or might not use. It's a bit tricky...

with --disable-shared...

...every single executable in your cross-compiler environment will be statically linked, which makes them huge. On the other hand, you can move around those executables, as there are no dependencies (which is what you want in step 4 above, to "bootstrap" your first native compiler). However, if you compiled any of the libraries that are linked in with a more advanced -march, the resulting executable is going to be only for that arch or better. This is usually no problem - unless you're doing a Canadian Cross where you might want to put the executable on a slower (less-featured) machine. Bad luck...

without --disable-shared...

...the executables will use dynamic linking, and will be much smaller. Now comes the tricky part: Not only the system libraries are dynamically linked, but also those implementing generic functionality of the cross-compilation environment. This is done by hard-coding the library paths into the executables... meaning that you can not move them to another directory. This is usually not necessary - unless you're doing a Canadian Cross where you might want to build the executables in a directory different from the one you want to install them in (e.g. due to access limitations on the build machine). Bad luck...

For a Canadian Cross, the best idea is not to use the option, and to copy the file to the exact same directory.

See Also

Articles

External Links

Retrieved from "https://osdev.wiki/wiki/GCC_Cross-Compiler?oldid=6855"