System Calls

This page is a work in progress.

This page may thus be incomplete. Its content may be changed in the near future.

System Calls are used to call a kernel service from user land. The goal is to be able to switch from user mode to kernel mode, with the associated privileges. Provided system calls depends on the nature of your kernel.

Available Methods

System Calls via Interrupts

The most common way to implement system calls is using a software interrupt. It is probably the most portable way to implement system calls. Linux traditionaly use interrupt 0x80 for this purpose.

To do this, you will have to create your interrupt handler in Assembly. This is because several common compilers do not support interrupt handlers, and most of the C compilers that do support interrupts make the interrupt function of the form

IntHandler:
    PUSHAD
    ; Code
    POPAD
    IRETD

Which will give you problems when you want to return values to the user. One method to fix this is by writing an handler that simply calls another function, so that registers are not preserved.

IntHandler:
    CALL C_Function 
    IRETD

Many protected mode OSes use EAX to hold the function code. DOS uses the AX register to store the function code - AH for the service and AL for functions of the service, or AH for the functions if there are no services. For example, let's say you have read() and write(). The codes are 1 for read() and 2 for write() from the interrupt 0A9h (an arbitrary choice, possibly wrong). You can write

IntA9Handler:
    CMP AH, 1
    JNE .write
    CALL _read
    JMP .done
.write:
    CMP AH, 2
    JNE .badcode
    CALL _write
    JMP .done
.badcode:
    MOV EAX, 0FFFFFFFFh
.done:
    IRETD

More parameters are usually passed through other registers such as EBX, ESI, etc.

System Calls via Sysenter/Sysexit (Intel)

On Intel CPU, starting from the Pentium II, a new instruction pair sysenter/sysexit has appeared. It allows a faster switch from user mode to kernel mode, by limiting the overhead of changing mode.

A similar instruction pair has been created by AMD: Syscall/Sysret. However the behaviour of these instructions are different from Intel's.

System Calls via Trap

Some OSes implement system calls by triggering a CPU Trap in a determined fashion such that they can recognize it as a system call. This solution is adopted on some hardware by Solaris, by L4, and probably others.

For example, L4 use a "LOCK NOP" instruction on x86. Since it is not permitted to perfrom a lock on the "NOP" instruction a trap is triggered. The problem with this approach is that there is no guarantee the "LOCK NOP" will have the same behavior on futur x86 CPU. They should probably have used the "UD2" instruction, since it is defined for this purposes.

System Calls via Call Gates (Intel)

The 80386 family of processors offer various call gates as part of the GDT. The call gate is a far pointer that can be called similar to calling a normal function. Very few operating systems use call gates.

Strategies Conlusion

The system call strategy depends on the platform. You may want to use different strategy depending on the architecture, and even switch strategy depending on the hardware performance.

On the user land side

While the developper can trigger manually the system call, it is probably a good idea to provide a library to encapsulate such a call. Therefore you wil be able to switch the system call technique without impacting the user applications.

See Also

Threads

• System call implementation

External Links

• System Call on Wikipedia.
• sandpile.org - Re: SYSENTER SYSCALL - An explanation on using sysenter / sysexit / syscall.
• Asking the kernel how to make a syscall - Just the same, with notes on how the L4 microkernel is impacted by this.
• Sysenter Based System Call Mechanism in Linux 2.6 - Explains the how and why of why Linux changed their System Call procedure for Pentium II+ machines.
• FreeBSD Developers' Handbook - System Calls - Discusses System Calls in FreeBSD from the usermode perspective.