Automated Build Using CircleCI

Difficulty level
Medium

This tutorial will show you how to set up an automated build environment using CircleCI, which will automatically build your project on every push to a repository, and make the resulting files available on the web. In the tutorial it is assumed that your kernel / operating system project is hosted at GitHub, from where it can be cloned and built using a freestanding GCC Cross-Compiler toolchain.

Setting up your project on CircleCI

First you will need to sign up at CircleCI. Navigate to their homepage and sign in. You will then have the possibility to connect with your GitHub account. GitHub will ask you whether you would like to grant CircleCI access to your account. Once you have confirmed, you can enter your CircleCI settings. Choose a project to build to enable automated builds whenever you push to that project.

Add a build script for the toolchain

The first step towards building your project should be building the toolchain, and thus in particular a GCC Cross-Compiler. The most convenient way to do this is to create a small script that will download the sources for binutils and gcc (and optionally additional sources, such as newlib or gdb), configure them for your chosen target and build the tools. The following script is an example, which builds a toolchain suitable for using a subset of C++ (only the freestanding part):

#!/bin/sh
set -e

# Set the versions we will be using.
binutils_version="2.27"
gcc_version="6.3.0"
newlib_version="2.4.0"

# This script expects the target triplet (e.g. i786-pc-elf) as command line argument.
target=$1

# The tools will be installed in ~/cross/$target.
prefix=~/cross/$target

# First check whether the toolchain was already built on a previous run of this script.
if [ ! -d $prefix ]
then
	mkdir -p /tmp/toolchain
	cd /tmp/toolchain

	# Download gcc sources if they are not yet downloaded.
	if [ ! -f gcc-$gcc_version.tar.bz2 ]
	then
		wget -c -O gcc-$gcc_version.tar.bz2 ftp://ftp.gnu.org/gnu/gcc/gcc-$gcc_version/gcc-$gcc_version.tar.bz2
		tar -xf gcc-$gcc_version.tar.bz2
	fi

	# Download binutils sources if they are not yet downloaded.
	if [ ! -f binutils-$binutils_version.tar.bz2 ]
	then
		wget -c -O binutils-$binutils_version.tar.bz2 ftp://ftp.gnu.org/gnu/binutils/binutils-$binutils_version.tar.bz2
		tar -xf binutils-$binutils_version.tar.bz2
	fi

	# Optional: download newlib sources if they are not yet downloaded.
	if [ ! -f newlib-$newlib_version.tar.gz ]
	then
		wget -c -O newlib-$newlib_version.tar.gz ftp://sources.redhat.com/pub/newlib/newlib-$newlib_version.tar.gz
		tar -xf newlib-$newlib_version.tar.gz
	fi

	# Create build paths.
	mkdir -p /tmp/toolchain/build-binutils
	mkdir -p /tmp/toolchain/build-gcc
	mkdir -p /tmp/toolchain/build-newlib

	# Build binutils.
	cd /tmp/toolchain/build-binutils
	sudo rm -rf *
	/tmp/toolchain/binutils-$binutils_version/configure --target=$target --prefix=$prefix --disable-nls 2>&1
	make all 2>&1
	make install 2>&1
	sudo rm -rf *

	# Build gcc and libgcc.
	cd /tmp/toolchain/build-gcc
	/tmp/toolchain/gcc-$gcc_version/configure --target=$target --prefix=$prefix --disable-nls --enable-languages=c,c++ --enable-libstdcxx --without-headers 2>&1
	make all-gcc 2>&1
	make install-gcc 2>&1
	make all-target-libgcc 2>&1
	make install-target-libgcc 2>&1

	# Make sure that our cross compiler will be found by creating links.
	# Alternative: Add the $prefix/bin directory to your $PATH.
	sudo ln -s -f $prefix/bin/* /usr/local/bin/

	# Optional: Build newlib. This is necessary only for the next, also optional build step.
	cd /tmp/toolchain/build-newlib
	sudo rm -rf *
	/tmp/toolchain/newlib-$newlib_version/configure --target=$target --prefix=$prefix 2>&1
	make all 2>&1
	make install 2>&1
	sudo rm -rf *

	# Optional: Build libstdc++. This is done in order to install the freestanding headers for using the C++11, C++14, C++17 standards.
	cd /tmp/toolchain/build-gcc
	/tmp/toolchain/gcc-$gcc_version/configure --target=$target --prefix=$prefix --disable-nls --enable-languages=c,c++ --enable-libstdcxx --without-headers --with-newlib 2>&1
	make all-target-libstdc++-v3 2>&1
	make install-target-libstdc++-v3 2>&1
	sudo rm -rf *
fi

# Also if the cross compiler has not been freshly build, link it so that it will be found.
sudo ln -s -f $prefix/bin/* /usr/local/bin/

Note that this is only an example script. You will most likely have to modify it according to your needs. Also you should test locally whether it actually works and generates the compiler you will use later. Once you have done this, add it to your source repository. Here we will assume that you have created a directory called ci in the root of your repository, and placed your script as a file named toolchain.sh in this folder.