James Molloy's Tutorial Known Bugs: Difference between revisions

m
[unchecked revision][unchecked revision]
m (→‎Problem: Not setting a stack: added links to Stack article)
 
(23 intermediate revisions by 13 users not shown)
Line 1:
Several sources - including this Wiki - point to [http://www.jamesmolloy.co.uk/tutorial_html/ James Molloy's Roll your own toy UNIX-clone OS] Tutorial as a starting point. This is wellfine, but the tutorial has some well-known weak points that cause trouble for people again and again. It's not uncommon that well-established members traced back mysterious bugs to early parts of their operating systems based on this tutorial. Nonetheless, it's one of the best introductory tutorials out there even if it has the occasional landmine. This article is meant to preempt issues arising from following the tutorial and to aid those that have encountered such problems. It is generally recommended to be sceptical of its advise on how to design your kernel and compare its information against this wiki. Some issues are quite subtle and only experts will recognize them.
 
== Before you follow the tutorial ==
Line 13:
== Problem: __cdecl calling convention ==
 
The tutorial states that the <tt>__cdecl</tt> calling convention is used. This is, however, a Windows term. and yourYour cross-compiler uses a similar calling convention but it is called the System V ABI for i386. It is advisable to understand this calling convention in depth, especially parts about how the parameters on the stack are clobbered and how structure parameters are passed. This will be very useful later and will help you avoid a later subtle bug. The function call example in 2.3 neglects to add 12 to esp following the call instruction, andso the three parameters are never thus popped.
 
== Problem: CFLAGS ==
Line 46:
== Problem: Missing functions ==
 
The gccGCC documentation mentions that the <tt>memset</tt>, <tt>memcpy</tt>, <tt>memmove</tt> and <tt>memcmp</tt> functions must always be present. The compiler uses these automatically for certain optimization purposes and even code that doesn't use them can automatically generate calls to them. You should add them at your earliest convenience.
 
== Problem: Interrupt handlers corrupt interrupted state ==
 
This article previously told you to know the ABI. If you do you will see a huge problem in the interrupt.s suggested by the tutorial: It breaks the ABI for structure passing! It creates an instance of the <tt>struct registers</tt> on the stack and then passes it by value to the <tt>isr_handler</tt> function and then assumes the structure is intact afterwards. However, the function parameters on the stack belongs to the function and it is allowed to trash these values as it sees fit (if you need to know whether the compiler actually does this, you are thinking the wrong way, but it actually does).
There are two ways around this. The most practical method is to pass the structure as a pointer instead, which allows you to explicitly edit the register state when needed - very useful for system calls, without having the compiler randomly doing it for you. The compiler can still edit the pointer on the stack when it's not specifically needed. The second option is to make another copy the structure and pass that.
 
== Problem: ISR 17 and 21 have error codes ==
 
The interrupt handling code in the downloadable code have a bug where it handles ISR 17 and 21 by pushing a fake error code, but the CPU does push error codes here.
 
== Problem: struct registers::esp is useless ==
Line 59 ⟶ 63:
== Problem: __attribute__((packed)) ==
 
This attribute packs the associated structure. This is useful in a few cases, such as the idtIDT and gdtGDT code (actually just the idtIDT, gdtGDT and tssTSS pointers). However, the tutorial tends to randomly attach it to every struct parameter, even where it isn't even needed. It is only needed where you badly want aligned structure members, it doesn't do anything if all the structure members were already naturally aligned. Otherwise, the compiler will automatically insert gaps between structure members so each begins at its own natural alignment.
 
== Problem: cli and sti in interrupt handlers ==
Line 70 ⟶ 74:
 
The <tt>kmalloc</tt> function in 6.4.1 only 1-byte aligns or page-aligns its memory address. This means you can only reliably use it allocate memories for chars (size 1), but not any larger types unless you use page-alignment. A proper malloc implementation returns pointers that are aligned such that they are suitable for all the common types, for instance it could be 64-bit (8-byte) aligned. You'll also want to modify the parameters such that it uses <tt>size_t</tt> appropriately rather than <tt>u32int</tt>.
 
Additionally the check if the address is page aligned is wrong.
<nowiki>
if (align == 1 && (placement_address & 0xFFFFF000)) // If the address is not already page-aligned</nowiki>
should be
<nowiki>
if (align == 1 && (placement_address & 0x00000FFF)) // If the address is not already page-aligned</nowiki>
 
== Problem: Paging Code ==
 
The paging code isn't terribly good and it is worth it to fully understand paging and design it all yourself. Paging code tends to be quite ugly, but it'll probably be decent after your fifth design revision. There is no need to always re-enable paging in <tt>switch_page_directory</tt>, it is likely best to have a special function the first time paging is enabled. The inlineInline assemblyAssembly in 6.4.5. doesn't need to be volatile as it is simply reading a memory value, which has no side-effects and it is acceptable if the compiler optimizes it away if the value is never used.
 
== Problem: Heap Code ==
 
It is probably best that you write your own heap implementation.
 
There is an operator precedence bug in <tt>find_smallest_hole()</tt> that will cause bad allocations and memory overwrites if attempting to fork in user mode later on.
 
To fix the problem, change this:
<nowiki>
if ((location+sizeof(header_t)) & 0xFFFFF000 != 0)</nowiki>
 
to this:
<nowiki>
if ((location+sizeof(header_t) & 0xFFFFF000) != 0)</nowiki>
 
See the section on user mode below for more details.
 
== Problem: VFS Code ==
Line 85 ⟶ 108:
== Problem: multiboot.h ==
 
It's advisable to get a copy of <tt>multiboot.h</tt> from the GRUB source code rather than copied from the tutorial. Beware, the copy in the GRUB documentation is out of date, use one from an official release.
 
 
== Problem: Multitasking ==
 
It is strongly recommended that you write your own implementation of this and disregard the tutorial. The tutorial attempts to implement forking kernel threads by searching for magic values on the stack, which is insanity. If you wish to create a new kernel thread, simply decide which registers it should have and point its stack pointer at its freshly allocated stack. It will then start executing at your desired entry point. The part where it disables paging is bad and you should just map the source and destination physical frames at appropriate virtual addresses and memcpy with paging on at all times. Section 9.3 in particular is insanity and has blown up at least one well-established hobby operating system.
 
 
=== Inline Assembly optimiser problem with GCC 4.8 ===
As mentioned above, writing Inline Assembly can be tricky. The original Inline Assembly is this:
 
<nowiki>
asm volatile(" \
cli; \
mov %0, %%ecx; \
mov %1, %%esp; \
mov %2, %%ebp; \
mov %3, %%cr3; \
mov $0x12345, %%eax; \
sti; \
jmp *%%ecx "
: : "r"(eip), "r"(esp), "r"(ebp), "r"(current_directory->physicalAddr)); </nowiki>
 
Everything works fine when using gcc-4.2.4. However, the gcc-4.8.4 optimizer produces the following assembly (produced with '''objdump -d src/kernel'''):
<nowiki>
10387c: fa cli
10387d: 89 c1 mov %eax,%ecx
10387f: 89 d4 mov %edx,%esp
103881: 89 cd mov %ecx,%ebp
103883: 0f 22 db mov %ebx,%cr3
103886: b8 45 23 01 00 mov $0x12345,%eax
10388b: fb sti
10388c: ff e1 jmp *%ecx</nowiki>
 
Note how the EAX register is assigned to the ECX register. However, later on the ECX register is assigned to EBP register. The reason for this is that the optimizer was using the EAX register to store the EIP variable and the ECX register to store the EBP variable. This results in the EIP variable being assigned to the ECX register ''as well as'' the EBP register. This leads to a subsequent '''ret''' statement sending the CPU to some invalid memory location.
 
A way to fix this is to remove the Inline Assembly by, for example adding this to '''process.s''':
<nowiki>
; Here we:
; * Stop interrupts so we don't get interrupted.
; * Temporarily put the new EIP location in ECX.
; * Temporarily put the new page directory's physical address in EAX.
; * Set the base and stack pointers
; * Set the page directory
; * Put a dummy value (0x12345) in EAX so that above we can recognize that we've just
; switched task.
; * Restart interrupts. The STI instruction has a delay - it doesn't take effect until after
; the next instruction.
; * Jump to the location in ECX (remember we put the new EIP in there).
 
[GLOBAL perform_task_switch]
perform_task_switch:
cli;
mov ecx, [esp+4] ; EIP
mov eax, [esp+8] ; physical address of current directory
mov ebp, [esp+12] ; EBP
mov esp, [esp+16] ; ESP
mov cr3, eax ; set the page directory
mov eax, 0x12345 ; magic number to detect a task switch
sti;
jmp ecx</nowiki>
 
Then edit '''task.c''' and add this to the top of the file:
<nowiki>
extern void perform_task_switch(u32int, u32int, u32int, u32int);</nowiki>
and replace the Inline Assembly with:
<nowiki>
perform_task_switch(eip, current_directory->physicalAddr, ebp, esp);</nowiki>
 
== Problem: User mode ==
 
There are several problems. The downloadable code has everything fixed except a <tt>find_smallest_hole()</tt> page-aligned heap allocation bug.
 
=== Problem 1: nasm byte keyword causing <tt>0x80</tt> to become <tt>0xffffff80</tt> ===
 
This code is at fault:
<nowiki>
%macro ISR_NOERRCODE 1 ; define a macro, taking one parameter
[GLOBAL isr%1] ; %1 accesses the first parameter.
isr%1:
cli
push byte 0
push byte %1
jmp isr_common_stub
%endmacro</nowiki>
 
It should be:
 
<nowiki>
%macro ISR_NOERRCODE 1 ; define a macro, taking one parameter
[GLOBAL isr%1] ; %1 accesses the first parameter.
isr%1:
cli
push byte 0
push %1
jmp isr_common_stub
%endmacro</nowiki>
 
=== Problem 2: Don't forget to allow interrupts in user mode in idt_set_gate ===
 
Find this comment in chapter 4 and uncomment the code:
<nowiki>
// We must uncomment the OR below when we get to using user-mode.
// It sets the interrupt gate's privilege level to 3.
idt_entries[num].flags = flags /* | 0x60 */;</nowiki>
 
=== Problem 3: regs var must called by reference instead of by value in the irq and isr handlers ===
Various changes are needed.
 
Change this in <tt>page_fault()</tt>:
<nowiki>
void page_fault(registers_t regs)
// to
void page_fault(registers_t *regs)</nowiki>
 
And in
<tt>isr_handler()</tt> and <tt>irq_handler()</tt>, change
 
<nowiki>
handler(regs);</nowiki>
to
<nowiki>
handler(&regs);</nowiki>
 
This fixes a problem where the syscall hander won't get called.
 
=== Problem 4: Missing documentation around set_kernel_stack ===
 
<tt>KERNEL_STACK_SIZE</tt> needs defining in <tt>task.h</tt>:
<nowiki>
#define KERNEL_STACK_SIZE 2048 // Use a 2kb kernel stack.</nowiki>
 
Also, some code needs to be added to four sections in <tt>task.c</tt>
 
In <tt>initialise_tasking()</tt>
<nowiki>
current_task->kernel_stack = kmalloc_a(KERNEL_STACK_SIZE);</nowiki>
 
In <tt>fork()</tt>
<nowiki>
current_task->kernel_stack = kmalloc_a(KERNEL_STACK_SIZE);</nowiki>
 
In <tt>switch_task()</tt>
<nowiki>
set_kernel_stack(current_task->kernel_stack+KERNEL_STACK_SIZE);</nowiki>
 
In <tt>switch_to_user_mode()</tt>
<nowiki>
set_kernel_stack(current_task->kernel_stack+KERNEL_STACK_SIZE);</nowiki>
 
=== Problem 5: find_smallest_hole() bug in heap code causing fork() to page fault ===
 
This bug from the heap chapter may not hit you until now. The bug results in the newly allocated <tt>kernel_stack</tt> messing up the page directory, causing <tt>clone_directory()</tt> to fail in <tt>fork()</tt>. See the heap section above for the details.
 
== Conclusion ==
73

edits