Calling Conventions

Calling external functions in C, and calling C functions from other languages, is a common issue in OS programming, especially where the other language is assembly. This page will concentrate primarily on the latter case, but some consideration is made for other languages as well.

Some of what is described here is imposed by the x86 architecture, some is special to the GNU GCC toolchain. Some is configurable, and you could be making your own GCC target to support a different calling convention. Currently, this page makes no effort of differentiating which is what.

Basics

As a general rule, a function which follows the C calling conventions, and is appropriately declared (see below) in the C headers, can be called as a normal C function. Most of the burden for following the calling rules falls upon the assembly program.

External References

In order to call a foreign function from C, it must have a correct C prototype. Thus, is if the function fee() takes the arguments fie, foe, and fum, in C calling order, and returns an integer value, then the corresponding header file should have the following prototype:

int fee(int fie, char foe, double fum)

Similarly, an global variables in the assembly code must be declared extern:

extern int frotz;

C functions in assembly or other languages must be declared as appropriate for the language. For example, in NASM, the C function

int foo(int bar, char baz, double quux);

would be declared

extern foo

Also, in most assembly languages, a function or variable that it to be exported must be declared global:

global foo
global frotz

Name Mangling

In some object formats (a.out), the name of a C function is automagically mangled by prepending it with an underscore ('_'). Thus, to call a C function foo() in assembly with such a format, you must define it as extern _foo instead of extern foo. This requirement does not apply to most modern formats such as COFF, PE, and ELF.

C++ name mangling is much more severe, as the C++ compiler encodes the type information from the parameter list into the symbol. (This is what enables function overloading in C++ in the first place.) The binutils package contains the tool c++filt that can be used to determine the correct mangled name.

Registers

The general register EBX, ESI, EDI, EBP, DS, ES, and SS, must be preserved by the called function. If you use them, you must save them first and restore them afterwards. Conversely, EAX and EDX are used for return values, and thus should not be preserved. The other registers do not need to be saved by the called function, but if they are in use by the calling function, then the calling function should save them before the call is made, and restored afterwards.

Passing Function Arguments

GCC/x86 passes function arguments on the stack. These arguments are pushed in reverse order from their order in the argument list. Furthermore, since the x86 protected-mode stack operations operate on doubleword (32-bit) values, the values are always pushed as a doubleword, even if the actual value is less than a full doubleword. Thus, for function foo(), the value of quux (a 48-bit FP value) is pushed first as two doublewords, low-dword first; the value of baz is pushed as the first byte of in doubleword; and then finally bar is pushed as a doubleword.

To pass arguments to a C function, the calling function must push the argument values as described above. Thus, to call foo() from a NASM assembly program, you would do something like this

push eax   ; low dword of quux
push edx   ; high dword of quux
push bl    ; baz
push ecx   ; bar
call foo

Accessing Function Arguments

In the GCC/x86 C calling convention, the first thing any function that accepts formal arguments should do is push the value of EBP (the frame base pointer of the calling function), then copy the value of ESP to EBP. This sets the function's own frame pointer, which is used to track both the arguments and (in C, or in any properly reentrant assembly code) the local variables.

To access arguments passed by a C function, you need to use the EBP an offset equal to 4 * (n + 2), where n is the number of the parameter in the argument list (not the number in the order it was pushed by), zero-indexed. The +2 is an added offset for the calling function's saved frame pointer and return pointer (pushed automatically by CALL, and popped by RET).

Thus, in function fee, to move fie into EAX, foe into BL, and fum into EAX and EDX, you would write (in NASM):

mov ecx, [ebp + 8]  ; fie
mov bl,  [ebp + 12] ; foe
mov edx, [ebp + 16] ; high dword of fum
mov eax, [ebp + 20] ; low dword of fum

As stated earlier, return values in GCC are passed using EAX and EDX. If a value exceeds 64 bits, it must be passed as a pointer.

See Also

External Links

• DJGPP FAQ: GCC calling conventions
• Linux Assembly Language HOWTO chapter 5
• http://my.execpc.com/~geezer/osd/libc/index.htm