Java Primer: Difference between revisions

11,910 bytes added ,  26 days ago
m
Bot: Replace deprecated source tag with syntaxhighlight
[unchecked revision][unchecked revision]
(Update compiler with some flavour text)
m (Bot: Replace deprecated source tag with syntaxhighlight)
 
(5 intermediate revisions by 3 users not shown)
Line 1:
{{Rating|3}}
Time to create an overview of how you can make managed languages work on bare hardware
 
This is a demonstration tutorial of how you could write an OS using Java. The methods described in here however are quite generic - you can use similar approaches for other bytecoded languages, such as .NET. In addition, this article contains a simple compiler that can be a source of inspiration for any custom languages.
Steps:
 
# define native ABI for Java
Being a demonstration, the method posted here is exceedingly simple and only does the necessary work to make the bundled Java code run. It can not deal with most things one would expect from Java. It does not do garbage collection in any forms, let alone support strings. The code provided here is purposefully limited, so you can play with it a bit to get a feeling, but it is impossible to extend this code to support more key components of the language.
# create bytecode-to-native compiler (uses objectweb asm)
 
# compile compiler
Consider this a throwaway prototype - '''it's deliberately written in such a fashion that you're forced to rewrite a new one from scratch.'''
# compile managed os to bytecode (uses regular javac)
 
# compile bytecode to native assembly
== Supporting a non-native language ==
# create runtime for things that have to be non-native
 
# assemble os and runtime (uses yasm)
Since you have much less to leech from existing compilers, you essentially are in charge of the entire toolchain. There are a number of rough steps:
# package the final kernel binary. (uses binutils)
* '''Define your native ABI'''
:: Java and .NET come in bytecode, which are designed for machines that are fundamentally different than your typical register-based hardware. You'll need to understand assembly for your platform, and you'll have to think about how the language structures should be converted to native ones. One of the things this example does is to adhere to the stdcall convention, because it's closer to the virtual java machine than the cdecl that's the default in most compilers.
* '''Create bytecode-to-native compiler'''
:: Java comes in bytecode form, and we need a method of converting it to native. Because we are writing an OS at the same time, everything must be compiled ahead of time, and not rely on an existing runtime that expects the existing presence of an OS. Therefore, we write our own compiler. As part of the compiler, we include the org.objectweb.asm that can read .class files and save us from writing significant portions of code.
* '''Compile the compiler'''
:: The compiler is of course written in Java as well. We don't need additional languages to add to the bootstrap problem later. Of course this compiler is overly simplistic, and will never be able to compile itself.
* '''Compile managed os to bytecode'''
:: Javac is written in Java, so that's obviously the best choice for later porting.
* '''Compile bytecode to native assembly'''
:: Now that we have the compiler, we can run it on all our OS classes to generate native code.
* '''Create runtime for things that have to be non-native'''
:: Java is memory-safe, which prevents it from implementing a lot of things. A minimal set of startup code is done in native assembly. Everything architecture-specific has to be done in assembly. Some parts you'll need later, such as memory management and exception handling are probably easier to have pure native parts for as well.
* '''Assemble os and runtime'''
:: We have everything in machine language form now. We're still reuseing the parts that we can, and in this example we use yasm. You're free to write an assembler in Java later on.
* '''package the final kernel binary'''
:: This is done using binutils. Note that there is not a single C compiler in the chain, and again, you can build your own linker in your language - it doesn't really have to do that much.
 
 
Line 19 ⟶ 35:
 
The entirety of a language is still a whole lot to deal with, but for our example it suffices to deal with just integers. That's right, no objects yet!
 
 
The Java bytecode uses a stack for operations, and a list of locals. These need not be in the same place, but as the x86 only has one hardware stack, we'll be using it as both local stack, call stack, and operation stack. A few tricks are used to make this compiler easier, and in turn, make it difficult to interface with C. Locals in Java terms include the function arguments, and as a result the locals would be split around return addresses. Since the caller doesn't know the storage needed - it doesn't even get the number of arguments for free - the called function should fix this. We also can't put things past the top of the stack because that'll be a big issue with interrupts later. Basically we copy all arguments to the other side of the return address and then we make some room for locals so that they can be indexed by <tt>EBP - 4 * slot_number</tt> where for instance 0 and 1 would be arguments and 2+ would be true locals.
Line 25 ⟶ 40:
The fact that getting the number of arguments is convoluted to perform on the caller's side, we do callee-cleanup using <tt>RET imm</tt> instead of the regular <tt>RET</tt>. Locals is a convoluted issue as well, so we just reserve room for 8 because you're not meant to copy this code anyway.
 
<sourcesyntaxhighlight lang="java">
// File: compiler/nl/combuster/minijava/Compiler.java
 
package nl.combuster.minijava;
 
Line 284 ⟶ 299:
}
 
</syntaxhighlight>
</source>
 
Yes, that's a compiler in a magic 256 lines of code. The result is Intel syntax assembly, because there already are decent tools for assembly out there that deal with object formats out there. Of course you can write your own later as well.
 
== HAL and Runtime ==
Almost no programming language has standard constructs for all the things a processor can do, in particular not languages that were designed to be portable and memory-safe. For that reason we are required to add support for that outside of the language. Fortunately, Java does come with one construct specifically design to wrap the language to platform specifics: <tt>native</tt>
 
<syntaxhighlight lang="java">
// File: os/nl/combuster/minijavaos/Hal.java
package nl.combuster.minijavaos;
 
public class Hal
{
public static native void poke(int address, byte data);
}
</syntaxhighlight>
At some point the OS needs to access memory unsafely, and this is the method we'll use. The native keyword also implies that there is no implementation in java and that it'll come from elsewhere.
 
There are more things that might be difficult to do in straight Java: this example omits memory management in its entirety, but there is still a quirk: Every object subclasses from java.lang.Object. To make matters more complicated, the compiler does not allow us to write java.lang classes in Java. In this case it's a particular chicken-and-egg problem: how would we even compile a class that extends itself? We are in some way forced to treat java.lang.Object special, and basically use its constructor as a terminator for a constructor chain. At a later stage, we might even want to do something more in there, but for now we don't need to. The first piece of native support code deals with these problems:
 
<syntaxhighlight lang="asm">
; File: os/hal.asm
section .text
 
global java_lang_Object___init_
global nl_combuster_minijavaos_Hal__poke
 
java_lang_Object___init_:
ret
 
nl_combuster_minijavaos_Hal__poke:
mov eax, [esp+8]
mov dl, [esp+4]
mov [eax], dl
ret 2
 
</syntaxhighlight>
 
== A primitive kernel ==
"Hello world!" is a very overrated thing. Or is it? Strings are a full-blown object in Java, and passing objects safely is one of the things skipped in the compiler itself. Instead, we can count our numbers to show what we have actually works:
 
<syntaxhighlight lang="java">
package nl.combuster.minijavaos;
 
public class Kernel
{
public Kernel(int arg1)
{
}
 
public static void kmain()
{
for (int i = 0; i < 10; i++)
{
Hal.poke(0xB8000 + 2 * i, (byte) ('0' + i));
Hal.poke(0xB8000 + 2 * i + 1, (byte) 0x1F);
}
}
}
</syntaxhighlight>
This is little more than some simple VGA code, but it demonstrates that we can call Java code, do some calculations, and then call our hardware abstraction layer to complete the little bits Java just couldn't do.
 
== Wrapping it all together ==
The last thing we need to do is to glue all the pieces together and stick a bootloader on top of it. [[Bare Bones]] has a more extensive description of the process. You might have noticed the full filenames in the source files above, basically we have two projects fairly tightly intertwined with each other, residing in the compiler and os subfolders that serve as the Java source root at the same time. A build folder is added so that you can simply delete that folder to start fresh, without having to worry about finding all your intermediates in between the real sources. In particular, we wouldn't be able to tell which .asm file is original and which one is compiler output without it!
 
==== Multiboot ====
We use GRUB as a bootloader, so the mechanics are the same. The only real difference is that we have to cope with the name mangling and be unable to call something kmain.
<syntaxhighlight lang="asm">
; File: os/multiboot.asm
 
; Declare constants used for creating a multiboot header.
MBALIGN equ 1<<0 ; align loaded modules on page boundaries
MEMINFO equ 1<<1 ; provide memory map
FLAGS equ MBALIGN | MEMINFO ; this is the Multiboot 'flag' field
MAGIC equ 0x1BADB002 ; 'magic number' lets bootloader find the header
CHECKSUM equ -(MAGIC + FLAGS) ; checksum of above, to prove we are multiboot
; Declare a header as in the Multiboot Standard.
section .multiboot
align 4
dd MAGIC
dd FLAGS
dd CHECKSUM
; We'll need a stack
section .bootstrap_stack, nobits
align 4
stack_bottom:
resb 16384
stack_top:
; Kernel entry point
section .text
global _start
_start:
mov esp, stack_top
 
extern nl_combuster_minijavaos_Kernel__kmain
call nl_combuster_minijavaos_Kernel__kmain
cli
.hang:
;hlt
jmp .hang
</syntaxhighlight>
 
====Linker script====
Linking also happens with the same script as [[Bare Bones]]
 
<syntaxhighlight lang="c">
/* File: os/linker.ld */
/* The bootloader will look at this image and start execution at the symbol
designated as the entry point. */
ENTRY(_start)
 
/* Tell where the various sections of the object files will be put in the final
kernel image. */
SECTIONS
{
/* Begin putting sections at 1 MiB, a conventional place for kernels to be
loaded at by the bootloader. */
. = 1M;
 
/* First put the multiboot header, as it is required to be put very early
early in the image or the bootloader won't recognize the file format.
Next we'll put the .text section. */
.text BLOCK(4K) : ALIGN(4K)
{
*(.multiboot)
*(.text)
}
 
/* Read-only data. */
.rodata BLOCK(4K) : ALIGN(4K)
{
*(.rodata)
}
 
/* Read-write data (initialized) */
.data BLOCK(4K) : ALIGN(4K)
{
*(.data)
}
 
/* Read-write data (uninitialized) and stack */
.bss BLOCK(4K) : ALIGN(4K)
{
*(COMMON)
*(.bss)
*(.bootstrap_stack)
}
 
/* The compiler may produce other sections, by default it will put them in
a segment with the same name. Simply add stuff here as needed. */
}
</syntaxhighlight>
 
==== GRUB ====
Some configuration files for GRUB legacy. Doing this with a CD is the easiest way, really:
<syntaxhighlight lang="bash">
#
# File: os/grub.cfg
#
default 0
timeout 30
 
# For booting GNU/Linux
title Java OS
root (cd)
kernel (cd)/kernel
</syntaxhighlight>
 
==== Build instructions ====
To simplify things, here's the Makefile that glues all the steps together. This one is a bit more elaborate as it covers for pretty much all the intermediate steps needed. Compile the compiler, make an executable jar file, compile the java files to class files, to assembly files, to object files, to an ELF binary. Include the runtime halfway into the process, then build a CD image with GRUB.
 
In this example, GRUB is assumed to be preinstalled in /boot where it resides by default on a linux system that has it installed. We also need mkisofs and the binutils step from the [[GCC Cross-Compiler]]. Of course, at some point in time you can decide to write these tools in Java as well
<syntaxhighlight lang="make">
 
COMPILER_J=$(shell find -L compiler -iname '*.java')
COMPILER_C=$(addprefix build/,$(patsubst %.java,%.class,$(COMPILER_J)))
 
OS_J:=$(shell find -L os -iname '*.java')
OS_C:=$(addprefix build/,$(patsubst %.java,%.class,$(OS_J)))
OS_A:=$(patsubst %.class,%.asm,$(OS_C))
RT_A:=$(shell find -L os -iname '*.asm')
OS_O:=$(patsubst %.asm,%.o,$(OS_A))
RT_O:=$(addprefix build/,$(patsubst %.asm,%.o,$(RT_A)))
 
default: build/compiler.jar build/kernel build/image.iso
echo done compiling
 
.PHONY: default
 
build/.dir:
mkdir -p build/compiler
mkdir -p build/os
touch build/.dir
 
build/compiler.jar: $(COMPILER_C) compiler/compiler.manifest
jar cvfm $@ compiler/compiler.manifest -C build/compiler .
 
build/compiler/%.class: compiler/%.java build/.dir
javac -d build/compiler -sourcepath compiler $<
 
build/os/%.class: os/%.java build/.dir
javac -d build/os -sourcepath os $<
 
%.asm:%.class build/compiler.jar
java -jar build/compiler.jar $< $@
 
%.o:%.asm
yasm -f elf -o $@ $<
 
build/os/%.o: os/%.asm build/.dir
yasm -f elf -o $@ $<
 
build/kernel: $(OS_O) $(RT_O)
i586-elf-ld -o $@ -T os/linker.ld $(OS_O) $(RT_O)
 
build/image.iso: build/kernel os/grub.cfg Makefile
-rm -rf build/iso
mkdir -p build/iso/boot/grub
cp build/kernel build/iso/
cp os/grub.cfg build/iso/boot/grub/menu.lst
cp /boot/grub/stage2_eltorito build/iso/boot/grub/
mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -o $@ build/iso
 
</syntaxhighlight>
 
If you paid attention, you'll notice that the JAR step requires a manifest at <tt>compiler/compiler.manifest</tt>. It only really marks it as runnable so that we can use it easily:
<syntaxhighlight lang="bash">
Main-Class: nl.combuster.minijava.Compiler
</syntaxhighlight>
 
Lastly, the [http://forge.ow2.org/project/download.php?group_id=23&file_id=20549 one library] our compiler uses to do most of its magic. You can just copy the source into the compiler folder and the build script will pick it up.
 
[[Category:Tutorials]]
[[Category:Compilers]]
[[Category:Languages]]
[[Category:Java]]