-*- mode: text -*- Cross-Compiling GLIBC Jim Blandy Introduction Most GNU tools have a simple build procedure: you run their 'configure' script, and then you run 'make'. Unfortunately, the process of cross-compiling the GNU C library is quite a bit more involved: 1) Build a cross-compiler, with certain facilities disabled. 2) Configure the C library using the compiler you built in step 1). Build a few of the C run-time object files, but not the rest of the library. Install the library's header files and the run-time object files, and create a dummy libc.so. 3) Build a second cross-compiler, using the header files and object files you installed in step 2. 4) Configure, build, and install a fresh C library, using the compiler built in step 3. 5) Build a third cross-compiler, based on the C library built in step 4. The reason for this complexity is that, although GCC and the GNU C library are distributed separately, they are not actually independent of each other: GCC requires the C library's headers and some object files to compile its own libraries, while the C library depends on GCC's libraries. GLIBC includes features and bug fixes to the stock GNU C library that simplify this process, but the fundamental interdependency stands. In this document, we explain how to cross-compile an GLIBC/GCC pair from source. Our intended audience is developers who are already familiar with the GNU toolchain and comfortable working with cross-development tools. While we do present a worked example to accompany the explanation, for clarity's sake we do not cover many of the options available to cross-toolchain users. Preparation GLIBC requires recent versions of the GNU binutils, GCC, and the Linux kernel. The web page documents the current requirements, and lists patches needed for certain target architectures. As of this writing, these build instructions have been tested with binutils 2.22.51, GCC 4.6.2, and Linux 3.1. First, let's set some variables, to simplify later commands. We'll build GLIBC and GCC for an ARM target, known to the Linux kernel as 'arm', and we'll do the build on an Intel x86_64 Linux box: $ build=x86_64-pc-linux-gnu $ host=$build $ target=arm-none-linux-gnueabi $ linux_arch=arm We're using the aforementioned versions of Binutils, GCC, and Linux: $ binutilsv=binutils-2.22.51 $ gccv=gcc-4.6.2 $ linuxv=linux-3.1 We're carrying out the entire process under '~/cross-build', which contains unpacked source trees for binutils, gcc, and linux kernel, along with GLIBC svn trunk (which can be checked-out with 'svn co http://www.eglibc.org/svn/trunk eglibc'): $ top=$HOME/cross-build/$target $ src=$HOME/cross-build/src $ ls $src binutils-2.22.51 glibc gcc-4.6.2 linux-3.1 We're going to place our build directories in a subdirectory 'obj', we'll install the cross-development toolchain in 'tools', and we'll place our sysroot (containing files to be installed on the target system) in 'sysroot': $ obj=$top/obj $ tools=$top/tools $ sysroot=$top/sysroot Binutils Configuring and building binutils for the target is straightforward: $ mkdir -p $obj/binutils $ cd $obj/binutils $ $src/$binutilsv/configure \ > --target=$target \ > --prefix=$tools \ > --with-sysroot=$sysroot $ make $ make install The First GCC For our work, we need a cross-compiler targeting an ARM Linux system. However, that configuration includes the shared library 'libgcc_s.so', which is compiled against the GLIBC headers (which we haven't installed yet) and linked against 'libc.so' (which we haven't built yet). Fortunately, there are configuration options for GCC which tell it not to build 'libgcc_s.so'. The '--without-headers' option is supposed to take care of this, but its implementation is incomplete, so you must also configure with the '--with-newlib' option. While '--with-newlib' appears to mean "Use the Newlib C library", its effect is to tell the GCC build machinery, "Don't assume there is a C library available." We also need to disable some of the libraries that would normally be built along with GCC, and specify that only the compiler for the C language is needed. So, we create a build directory, configure, make, and install. $ mkdir -p $obj/gcc1 $ cd $obj/gcc1 $ $src/$gccv/configure \ > --target=$target \ > --prefix=$tools \ > --without-headers --with-newlib \ > --disable-shared --disable-threads --disable-libssp \ > --disable-libgomp --disable-libmudflap --disable-libquadmath \ > --disable-decimal-float --disable-libffi \ > --enable-languages=c $ PATH=$tools/bin:$PATH make $ PATH=$tools/bin:$PATH make install Linux Kernel Headers To configure GLIBC, we also need Linux kernel headers in place. Fortunately, the Linux makefiles have a target that installs them for us. Since the process does modify the source tree a bit, we make a copy first: $ cp -r $src/$linuxv $obj/linux $ cd $obj/linux Now we're ready to install the headers into the sysroot: $ PATH=$tools/bin:$PATH \ > make headers_install \ > ARCH=$linux_arch CROSS_COMPILE=$target- \ > INSTALL_HDR_PATH=$sysroot/usr GLIBC Headers and Preliminary Objects Using the cross-compiler we've just built, we can now configure GLIBC well enough to install the headers and build the object files that the full cross-compiler will need: $ mkdir -p $obj/glibc-headers $ cd $obj/glibc-headers $ BUILD_CC=gcc \ > CC=$tools/bin/$target-gcc \ > CXX=$tools/bin/$target-g++ \ > AR=$tools/bin/$target-ar \ > RANLIB=$tools/bin/$target-ranlib \ > $src/glibc/libc/configure \ > --prefix=/usr \ > --with-headers=$sysroot/usr/include \ > --build=$build \ > --host=$target \ > --disable-profile --without-gd --without-cvs \ > --enable-add-ons=nptl,libidn,../ports The option '--prefix=/usr' may look strange, but you should never configure GLIBC with a prefix other than '/usr': in various places, GLIBC's build system checks whether the prefix is '/usr', and does special handling only if that is the case. Unless you use this prefix, you will get a sysroot that does not use the standard Linux directory layouts and cannot be used as a basis for the root filesystem on your target system compatibly with normal GLIBC installations. The '--with-headers' option tells GLIBC where the Linux headers have been installed. The '--enable-add-ons=nptl,libidn,../ports' option tells GLIBC to look for the listed glibc add-ons. Most notably the ports add-on (located just above the libc sources in the GLIBC svn tree) is required to support ARM targets. We can now use the 'install-headers' makefile target to install the headers: $ make install-headers install_root=$sysroot \ > install-bootstrap-headers=yes The 'install_root' variable indicates where the files should actually be installed; its value is treated as the parent of the '--prefix' directory we passed to the configure script, so the headers will go in '$sysroot/usr/include'. The 'install-bootstrap-headers' variable requests special handling for certain tricky header files. Next, there are a few object files needed to link shared libraries, which we build and install by hand: $ mkdir -p $sysroot/usr/lib $ make csu/subdir_lib $ cp csu/crt1.o csu/crti.o csu/crtn.o $sysroot/usr/lib Finally, 'libgcc_s.so' requires a 'libc.so' to link against. However, since we will never actually execute its code, it doesn't matter what it contains. So, treating '/dev/null' as a C source file, we produce a dummy 'libc.so' in one step: $ $tools/bin/$target-gcc -nostdlib -nostartfiles -shared -x c /dev/null \ > -o $sysroot/usr/lib/libc.so The Second GCC With the GLIBC headers and selected object files installed, we can now build a GCC that is capable of compiling GLIBC. We configure, build, and install the second GCC, again building only the C compiler, and avoiding libraries we won't use: $ mkdir -p $obj/gcc2 $ cd $obj/gcc2 $ $src/$gccv/configure \ > --target=$target \ > --prefix=$tools \ > --with-sysroot=$sysroot \ > --disable-libssp --disable-libgomp --disable-libmudflap \ > --disable-libffi --disable-libquadmath \ > --enable-languages=c $ PATH=$tools/bin:$PATH make $ PATH=$tools/bin:$PATH make install GLIBC, Complete With the second compiler built and installed, we're now ready for the full GLIBC build: $ mkdir -p $obj/glibc $ cd $obj/glibc $ BUILD_CC=gcc \ > CC=$tools/bin/$target-gcc \ > CXX=$tools/bin/$target-g++ \ > AR=$tools/bin/$target-ar \ > RANLIB=$tools/bin/$target-ranlib \ > $src/glibc/libc/configure \ > --prefix=/usr \ > --with-headers=$sysroot/usr/include \ > --with-kconfig=$obj/linux/scripts/kconfig \ > --build=$build \ > --host=$target \ > --disable-profile --without-gd --without-cvs \ > --enable-add-ons=nptl,libidn,../ports Note the additional '--with-kconfig' option. This tells GLIBC where to find the host config tools used by the kernel 'make config' and 'make menuconfig'. These tools can be re-used by GLIBC for its own 'make *config' support, which will create 'option-groups.config' for you. But first make sure those tools have been built by running some dummy 'make *config' calls in the kernel directory: $ cd $obj/linux $ PATH=$tools/bin:$PATH make config \ > ARCH=$linux_arch CROSS_COMPILE=$target- \ $ PATH=$tools/bin:$PATH make menuconfig \ > ARCH=$linux_arch CROSS_COMPILE=$target- \ Now we can configure and build the full GLIBC: $ cd $obj/glibc $ PATH=$tools/bin:$PATH make defconfig $ PATH=$tools/bin:$PATH make menuconfig $ PATH=$tools/bin:$PATH make $ PATH=$tools/bin:$PATH make install install_root=$sysroot At this point, we have a complete GLIBC installation in '$sysroot', with header files, library files, and most of the C runtime startup files in place. The Third GCC Finally, we recompile GCC against this full installation, enabling whatever languages and libraries we would like to use: $ mkdir -p $obj/gcc3 $ cd $obj/gcc3 $ $src/$gccv/configure \ > --target=$target \ > --prefix=$tools \ > --with-sysroot=$sysroot \ > --enable-__cxa_atexit \ > --disable-libssp --disable-libgomp --disable-libmudflap \ > --enable-languages=c,c++ $ PATH=$tools/bin:$PATH make $ PATH=$tools/bin:$PATH make install The '--enable-__cxa_atexit' option tells GCC what sort of C++ destructor support to expect from the C library; it's required with GLIBC. And since GCC's installation process isn't designed to help construct sysroot trees, we must manually copy certain libraries into place in the sysroot. $ cp -d $tools/$target/lib/libgcc_s.so* $sysroot/lib $ cp -d $tools/$target/lib/libstdc++.so* $sysroot/usr/lib Trying Things Out At this point, '$tools' contains a cross toolchain ready to use the GLIBC installation in '$sysroot': $ cat > hello.c < #include > int > main (int argc, char **argv) > { > puts ("Hello, world!"); > return 0; > } > EOF $ $tools/bin/$target-gcc -Wall hello.c -o hello $ cat > c++-hello.cc < #include > int > main (int argc, char **argv) > { > std::cout << "Hello, C++ world!" << std::endl; > return 0; > } > EOF $ $tools/bin/$target-g++ -Wall c++-hello.cc -o c++-hello We can use 'readelf' to verify that these are indeed executables for our target, using our dynamic linker: $ $tools/bin/$target-readelf -hl hello ELF Header: ... Type: EXEC (Executable file) Machine: ARM ... Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align PHDR 0x000034 0x10000034 0x10000034 0x00100 0x00100 R E 0x4 INTERP 0x000134 0x00008134 0x00008134 0x00013 0x00013 R 0x1 [Requesting program interpreter: /lib/ld-linux.so.3] LOAD 0x000000 0x00008000 0x00008000 0x0042c 0x0042c R E 0x8000 ... Looking at the dynamic section of the installed 'libgcc_s.so', we see that the 'NEEDED' entry for the C library does include the '.6' suffix, indicating that was linked against our fully build GLIBC, and not our dummy 'libc.so': $ $tools/bin/$target-readelf -d $sysroot/lib/libgcc_s.so.1 Dynamic section at offset 0x1083c contains 24 entries: Tag Type Name/Value 0x00000001 (NEEDED) Shared library: [libc.so.6] 0x0000000e (SONAME) Library soname: [libgcc_s.so.1] ... And on the target machine, we can run our programs: $ $sysroot/lib/ld.so.1 --library-path $sysroot/lib:$sysroot/usr/lib \ > ./hello Hello, world! $ $sysroot/lib/ld.so.1 --library-path $sysroot/lib:$sysroot/usr/lib \ > ./c++-hello Hello, C++ world!