Porting Newlib: Difference between revisions

m
no edit summary
[unchecked revision][unchecked revision]
No edit summary
mNo edit summary
 
(19 intermediate revisions by 11 users not shown)
Line 3:
 
Porting newlib is one of the easiest ways to get a simple C library into your operating system without an excessive amount of effort. As an added bonus, once complete you can port the toolchain (GCC/binutils) to your OS - and who wouldn't want to do that?
 
''This article was written with x86 in mind. It has been extended to armv8 through tips and notes in the troubleshooting section.''
 
== Introduction ==
Line 10 ⟶ 12:
== Preparation ==
 
Download newlib source (I'm using 2.25.0-1) from [ftp://sources.redhat.com/pub/newlib/index.html this ftp server].
 
=== Download source code of Automake and Autoconf ===
Install autotools 1.12 or earlier. newlib relies on "Cygnus-style trees", a feature which was removed in 1.13.
 
Acquire Automake (v1.11) and Autoconf (v2.65) from here:
[https://ftp.gnu.org/gnu/automake/automake-1.11.tar.gz]
[https://ftp.gnu.org/gnu/autoconf/autoconf-2.65.tar.gz]
 
''Note'': The newlib source is organized using "Cygnus style," which is unsupported in Automake versions 1.12 and beyond.
Therefore, to be able to build newlib, you need a version less than or equal to 1.11.
 
Untar both of the archives:
<syntaxhighlight lang="bash">
tar xvf automake-1.11.tar.gz
tar xvf autoconf-2.65.tar.gz
</syntaxhighlight>
 
Create a destination folder:
<syntaxhighlight lang="bash">
mkdir ~/bin
</syntaxhighlight>
 
Create a build folder:
<syntaxhighlight lang="bash">
mkdir build
cd build
</syntaxhighlight>
 
Configure automake first:
<syntaxhighlight lang="bash">
../automake-1.11/configure --prefix="~/bin"
</syntaxhighlight>
 
Make and install
 
<syntaxhighlight lang="bash">
make && make install
</syntaxhighlight>
Now lets configure autoconf
<syntaxhighlight lang="bash">
../autoconf-2.65/configure --prefix=~/bin
</syntaxhighlight>
Then make and install:
<syntaxhighlight lang="bash">
make && make install
</syntaxhighlight>
 
You should now have the proper binaries in ~/bin!
 
To add these binaries to your path temporarily
<syntaxhighlight lang="bash">
export PATH=~/bin:$PATH
</syntaxhighlight>
 
== System Calls ==
 
First of all you need to support a set of 17 system calls that act as 'glue' between newlib and your OS. These calls are the typical "_exit", "open", "read/write", "execve" (et al). See the [http://sourceware.org/newlib/libc.html#Syscalls Red Hat newlib C library] documentation.
See the [http://sourceware.org/newlib/libc.html#Syscalls Red Hat newlib C library] documentation for an overview of necessary calls.
 
Newlib uses a very specific hierarchy of syscalls, many of which can be supplied by more than one file.
My kernel exposes all the system calls on interrupt 0x80 (128d) so I just had to put a bit of inline assembly into each stub to do what I needed it to do. It's up to you how to implement them in relation to your kernel.
This can quickly lead to symbol redefinition or symbol missing errors when linking with the library.
The normal way that newlib expects you to define syscalls, which you may see elsewhere is to define the underscored symbols (e.g. _open instead of open).
In this case, newlib will call the underscored versions using wrappers defined in newlib/libc/syscalls/.
Our different (simplified) approach is to define the syscalls directly. No wrappers.
To do this the newlib_cflags variable must be set to "" in configure.host (default for some platforms, like x86), which will prevent the wrappers from being compiled.
 
Implementing the syscalls is usually quite trivial, my kernel exposes all the system calls on interrupt 0x80 (128d) so I just had to put a bit of inline assembly into each stub to do what I needed it to do.
It's up to you how to implement them in relation to your kernel.
 
== Porting Newlib ==
Line 26 ⟶ 87:
Same as for binutils in [[OS Specific Toolchain]].
 
=== newlib/configure.host ===
 
Tell newlib which system-specific directory to use for our particular target. In the section starting 'Get the source directories to use for the host ... case "${host}" in', add a section:
 
<sourcesyntaxhighlight lang="bash">
i[3-7]86-*-myos*)
sys_dir=myos
;;
</syntaxhighlight>
</source>
 
configure.host contains two switch clauses, make sure that your variables are not overwritten later!
For example, for aarch64 platforms it sets the syscall_dir variable after us, breaking the library.
 
=== newlib/libc/sys/configure.in ===
Line 40 ⟶ 104:
Tell the newlib build system that it also needs to configure our myos-specific host directory. In the <tt>case ${sys_dir} in</tt> list, simply add
 
<sourcesyntaxhighlight lang="autoconfbash">
myos) AC_CONFIG_SUBDIRS(myos) ;;
</syntaxhighlight>
</source>
 
'''Note:''' After this, you need to run <tt>autoconf (precisely version 2.64)</tt> in the libc/sys directory.
Line 54 ⟶ 118:
This file creates crt0.o, which is included in every application. It should define the symbol _start, and then call the main() function, possibly after setting up process-space segment selectors and pushing argc and argv onto the stack. A simple implementation is:
 
<sourcesyntaxhighlight lang="C">
#include <fcntl.h>
 
Line 64 ⟶ 128:
exit(ex);
}
</syntaxhighlight>
</source>
 
'''Note:''' add in argc and argv support based on how you handle them in your OS
Line 72 ⟶ 136:
This file should contain the implementations for each glue function newlib requires.
 
<sourcesyntaxhighlight lang="C">
/* note these headers are all provided by newlib - you don't need to provide them */
#include <sys/stat.h>
Line 102 ⟶ 166:
int write(int file, char *ptr, int len);
int gettimeofday(struct timeval *p, struct timezone *z);
</syntaxhighlight>
</source>
 
'''Note''': You may split this up into multiple files, just don't forget to link against all of them in Makefile.am.
Line 110 ⟶ 174:
Configure script for our system directory.
 
<sourcesyntaxhighlight lang="bash">
AC_PREREQ(2.59)
AC_INIT([newlib], [NEWLIB_VERSION])
Line 118 ⟶ 182:
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
</syntaxhighlight>
</source>
 
=== newlib/libc/sys/myos/Makefile.am ===
Line 124 ⟶ 188:
A Makefile template for this directory:
 
<sourcesyntaxhighlight lang="make">
AUTOMAKE_OPTIONS = cygnus
INCLUDES = $(NEWLIB_CFLAGS) $(CROSS_CFLAGS) $(TARGET_CFLAGS)
Line 150 ⟶ 214:
ACLOCAL_AMFLAGS = -I ../../..
CONFIG_STATUS_DEPENDENCIES = $(newlib_basedir)/configure.host
</syntaxhighlight>
</source>
 
'''Note''': After this, you need to run <tt>autoconf</tt> in the newlib/libc/sys/ directory, and <tt>autoreconf</tt> in the newlib/libc/sys/myos directory.
 
'''Note''': <tt>autoconf</tt> and <tt>autoreconf</tt> will only run with automake version 1.14 (exact) and autoconf version 2.64 (exactly) (applies to newlib v2.2.0-1 source pulled on March 10, 2015)''
 
=== Signal handling ===
Line 162 ⟶ 224:
Alternatively, you can provide your own implementation. To do this, you need to define your own version of signal() in syscalls.c. A typical implementation would register the handler somewhere in kernel space, so that issuing a signal from another process causes the corresponding function to be called in the receiving process (this will also require some nifty stack-playing in the receiving process, as you are basically interrupting the program flow in the middle). You then need to provide a kill() function in syscalls.c which actually sends signals to another process. Newlib will still define a raise() function for you, but it is just a stub which calls kill() with the current process id. To switch newlib to this mode, you need to #define the SIGNAL_PROVIDED macro when compiling. A simple way to do this is to add the line:
 
<sourcesyntaxhighlight lang="bash">
newlib_cflags="${newlib_cflags} -DSIGNAL_PROVIDED"
</syntaxhighlight>
</source>
 
to your host's entry in <tt>configure.host</tt>. It would probably also make sense to provide sigaction(), and provide signal() as a wrapper for it. Note that [http://pubs.opengroup.org/onlinepubs/9699919799/functions/sigaction.html the Open Group's] definition of sigaction states that 1) sigaction supersedes signal, and 2) an application designed shouldn't use both to manipulate the same signal.
Line 175 ⟶ 237:
'''Note:''' there must be a better way then this.
 
<sourcesyntaxhighlight lang="bash">
# newlib setup
CURRDIR=$(pwd)
 
# make symlinkdssymlinks (a bad hack) to make newlib work
cd ~/cross/bin/ # this is where the bootstrapped generic cross compiler toolchain (i686-elf-xxx) is installed in,
# change this based on your development environment.
Line 190 ⟶ 252:
# return
cd $CURRDIR
</syntaxhighlight>
</source>
 
Then run the following commands to build newlib
 
<sourcesyntaxhighlight lang="bash">
mkdir build-newlib
cd build-newlib
Line 200 ⟶ 262:
make all
make DESTDIR=${SYSROOT} install
</syntaxhighlight>
</source>
 
'''Note:''' SYSROOT is where all your OS-specific toolchains will be installed in. It will look like a miniature version of the Linux filesystem, but have your OS-specific toolchains in; I am using ~/myos as my SYSROOT directory.
 
'''Note:''' By default, newlib is configured to not support %lld/u format specifiers in printf()/scanf() (i.e. it assumes %lld to mean the same as %ld). In order to override this, should it matter, one must add --enable-newlib-io-long-long to the configure invocation
 
For some reason, the newer versions of newlib (at least for me) didn't put the libraries in a location where other utilities like binutils could find.
So here's another hack to fix this:
 
<sourcesyntaxhighlight lang="bash">
cp -ar $SYSROOT/usr/i386i686-myos/* $SYSROOT/usr/
</syntaxhighlight>
</source>
 
After building all of this, your freshly built libc will be installed in your SYSROOT directory! Now you can progress to building your own [[OS Specific Toolchain]].
Line 215 ⟶ 279:
'''Important Note:''' I found that for newlib to properly work, you have to link against libc, libg, libm, and libnosys - hence when porting gcc, in
 
<sourcesyntaxhighlight lang="C">
#define LIB_SPEC ...
</syntaxhighlight>
</source>
 
in gcc/configureconfig/myos.h,
 
make sure you put
 
<sourcesyntaxhighlight lang="C">
#define LIB_SPEC "-lc -lg -lm -lnosys"
</syntaxhighlight>
</source>
 
at the bare minimum.
Line 240 ⟶ 304:
 
'''Note:''' I used a lot of hacks in this article, if you find a better way to do something, please contribute to the page. Thank you.
 
== Troubleshooting ==
 
=== Autotools ===
* Whenever you modifiy configure.ac/in files, or not auto-generated Makefile.in files, you must run the appropriate autoconf/automake/autoreconf command
* autoreconf is a tool that automatically calls autotools as required to process the present working working directory. Autoreconf will use the versions of autotools in your path, so make sure to prepend(!) your custom build of autotools to the path variable. There are also supposedly environment variables that can be set.
* *.ac or *.in files are modified by pattern substitution. Lone spaces or tabs WILL cause issues later, since Makefiles are whitespace sensitive and the whitespace is never removed
 
=== Library Implementation ===
* In some situations, the <code>-DMISSING_SYSCALL_NAMES</code> flag must be set in `newlib_cflags` so that certain functions can call your syscalls as the underscore variant. I.e. sbrk() instead of _sbrk() in your syscalls.c. Otherwise at compile time, the symbol _sbrk will be reported as missing.
* The crt0.o object is overridden by the one generated by <code>libgloss/aarch64</code>. To use your crt file, make sure to override the existing one in your sysroot with the version from the <code>$NEWLIB_BUILD_DIR/$TARGET/newlib/</code> folder after installing newlib.
 
=== Build System Tips ===
* When porting aarch64 (and perhaps other platforms), not having *-elf at the end of your target string can lead to a circular dependency in one of the Makefiles, causing the build to fail on copying the .spec files. Make sure to add a match statement in the <code>case "${target}" in</code> case in `libgloss/aarch64/configure.in`.
* The build system was rewritten in later versions of NEWLIB. The instructions regarding the automake files will no longer apply. For the most part, it just means leaving out those parts, but be prepared to do your own detective work!
 
=== General Tips ===
* Copy-pasting may introduce aforementioned missing newlines or spurious spaces.
* Don't move your build artifacts. This may break dependencies.
* If you are stuck, try taking a look at other NEWLIB ports, such as for Jin Xue's https://github.com/Jimx-/lyos, for example.
 
== See Also ==
Line 247 ⟶ 331:
* [[OS Specific Toolchain]]
 
[[Category:Porting]]
[[Category:C]]
[[Category:Standard Libraries]]
[[Category:Tutorials]]