Java Primer

Time to create an overview of how you can make managed languages work on bare hardware

Steps:

1. define native ABI for Java
2. create bytecode-to-native compiler (uses objectweb asm)
3. compile compiler
4. compile managed os to bytecode (uses regular javac)
5. compile bytecode to native assembly
6. create runtime for things that have to be non-native
7. assemble os and runtime (uses yasm)
8. package the final kernel binary. (uses binutils)

Compiler

In bytecoded languages there are several steps before code can be run. Typically you have "the" compiler, which converts your source files into some portable binary, and you have an interpreter that reads those binaries and runs instructions from them. Modern interpreters turn bytecode into native code, as to avoid the if(instruction = ...) that takes several cycles while the instruction you'd actually want to execute would otherwise cost you just one CPU instruction.

In the case of an OS, we need to take this a step further. We could run an interpreter, but that's slow. We could compile into native on boot but that needs just as much OS as we actually want to run. Instead, the appropriate solution is to compile to native in advance, so we can just run the code directly from the start.

The entirety of a language is still a whole lot to deal with, but for our example it suffices to deal with just integers. That's right, no objects yet!

The Java bytecode uses a stack for operations, and a list of locals. These need not be in the same place, but as the x86 only has one hardware stack, we'll be using it as both local stack, call stack, and operation stack. A few tricks are used to make this compiler easier, and in turn, make it difficult to interface with C. Locals in Java terms include the function arguments, and as a result the locals would be split around return addresses. Since the caller doesn't know the storage needed - it doesn't even get the number of arguments for free - the called function should fix this. We also can't put things past the top of the stack because that'll be a big issue with interrupts later. Basically we copy all arguments to the other side of the return address and then we make some room for locals so that they can be indexed by EBP - 4 * slot_number where for instance 0 and 1 would be arguments and 2+ would be true locals.

The fact that getting the number of arguments is convoluted to perform on the caller's side, we do callee-cleanup using RET imm instead of the regular RET. Locals is a convoluted issue as well, so we just reserve room for 8 because you're not meant to copy this code anyway.

package nl.combuster.minijava;

import org.objectweb.asm.*;
import org.objectweb.asm.tree.*;
import java.util.*;
import java.io.*;

public class Compiler
{
    public static byte[] readEntireFile(String filename)
    {
        File file = new File(filename);
        try 
        {
            FileInputStream input = new FileInputStream(file);
            byte bytes[] = new byte[(int)file.length()];
            input.read(bytes);
            return bytes;
        }
        catch (IOException e)
        {
            throw new RuntimeException("Unable to read file " + file, e);
        }
    }

    public static void writeOutput(String filename, List<String> lines)
    {
        try 
        {
            BufferedWriter writer = new BufferedWriter(new FileWriter(new File(filename)));
            for (String string : lines)
            {
                writer.write(string);
                writer.newLine();
            }
            writer.close();
        }
        catch (IOException e)
        {
            throw new RuntimeException("Unable to write output file " + filename, e);
        }
    }

    public static String decorate(String classname, String method, String signature)
    {
        // make all these assembly-friendly names. Note that the 
        // constructor is for instance called <init>
        return classname.replace("/","_") + "__" + method.replace("<","_").replace(">","_");
    }
    
    public static void jumpgroup(List<String> output, String jumpcode, LabelNode dest)
    {
    	output.add("pop edx");
    	output.add("pop ecx");
    	output.add("cmp ecx, edx");
    	output.add(jumpcode + " .l" + dest.getLabel());
    }

    public static void main(String args[])
    {
        if (args.length != 2) throw new RuntimeException("Usage: compiler input-file output-file");

        ClassNode node = new ClassNode();
        ClassReader reader = new ClassReader(readEntireFile(args[0]));
        reader.accept(node, 0);

        List<String> outputdata = new LinkedList<String>();

        outputdata.add("section .text");

        for (MethodNode method : node.methods)
        {
            method.visitCode();
            String methodname = decorate(node.name, method.name, method.signature);

            if ((method.access & Opcodes.ACC_NATIVE) != 0) continue;

            // prologue
            System.out.println("; attributes: " + method.attrs);
            outputdata.add("global " + methodname);
            outputdata.add(methodname + ":");
            outputdata.add("push ebp");
            outputdata.add("mov ebp, esp");

            int locals = (method.localVariables == null) ? 0 : method.localVariables.size();
            outputdata.add("; locals: + " + locals);
			int arguments = (method.parameters == null) ? 0 : method.parameters.size();
            if ((method.access & Opcodes.ACC_STATIC) == 0) arguments++; // hidden "this"
            outputdata.add("; params: + " + arguments);
            // copy params so that they correspond with java indexing and join with the local numbering
			for (int i = 0; i < arguments; i++)
            {
                outputdata.add("push dword [ebp + " + (8 + 4 * i) + "]");
            }
            // do some frame checking for many variables, locals is not of much use...
            outputdata.add("sub esp, 32");

            Iterator<AbstractInsnNode> iterator = method.instructions.iterator();
            while (iterator.hasNext())
            {     
                AbstractInsnNode insn = iterator.next();
                int opcode = insn.getOpcode() & 0xff;
                outputdata.add("    ; " + opcode + " = " + insn.getClass().getSimpleName());
                //outputdata.add("mov byte [" + (0xb8000 + 156) + "], '0' + " + (opcode % 10));
                //outputdata.add("mov byte [" + (0xb8000 + 154) + "], '0' + " + (opcode / 10)%10);
                //outputdata.add("mov byte [" + (0xb8000 + 152) + "], '0' + " + (opcode / 100));
                
                if (insn instanceof LabelNode)
                {
                    LabelNode labelinsn = (LabelNode)insn;
                    outputdata.add(".l" + labelinsn.getLabel());
                }
                else if (insn instanceof LineNumberNode)
                {
                    // ignore these
                }
                else if (insn instanceof VarInsnNode)
                {
                    // copy a variable
                    VarInsnNode varinsn = (VarInsnNode) insn;
                    switch(varinsn.getOpcode())
                    {
                        case Opcodes.ILOAD:
                        case Opcodes.ALOAD:
                            // todo: verify offset
                            outputdata.add("push dword [ebp - " + (4 + 4 * varinsn.var) + "]");
                            break;
                            
                        case Opcodes.ISTORE:
                        case Opcodes.ASTORE:
                            outputdata.add("pop dword [ebp - " + (4 + 4 * varinsn.var) + "]");
                            break;

                        default:
                            outputdata.add("Can't deal with varinsnnode: " + varinsn.getOpcode());
                    }
                }
                else if (insn instanceof MethodInsnNode)
                {
                    MethodInsnNode methodinsn = (MethodInsnNode) insn;
                    String calledmethod = decorate(methodinsn.owner, methodinsn.name, methodinsn.desc);
                    outputdata.add("extern " + calledmethod);
                    outputdata.add("call " + calledmethod);
                    if (!methodinsn.desc.endsWith("V"))
                    {
                        // not a void return value
                        outputdata.add("push eax");
                    }
                    /*switch (methodinsn.getOpcode())
                    {
                        default:
                            outputdata.add("Can't deal with methodcall: " + methodinsn.getOpcode());
                    }*/
                }
                else if (insn instanceof IntInsnNode)
                {
                	IntInsnNode intinsn = (IntInsnNode)insn;
                    switch(intinsn.getOpcode())
                    {
                        case Opcodes.BIPUSH:
                            outputdata.add("push " + intinsn.operand);
                            break;

                        default:
                            outputdata.add("Can't deal with intinsnnode: " + intinsn.getOpcode());
                    }
                	
                }
                else if (insn instanceof JumpInsnNode)
                {
                	JumpInsnNode jmpinsn = (JumpInsnNode)insn;
                	
                    switch(jmpinsn.getOpcode())
                    {
                    	case Opcodes.IF_ICMPEQ:		jumpgroup(outputdata, "je", jmpinsn.label); 	break; // 159
                    	case Opcodes.IF_ICMPNE:		jumpgroup(outputdata, "jne", jmpinsn.label); 	break; // 160                    	
                    	case Opcodes.IF_ICMPLT:		jumpgroup(outputdata, "jb", jmpinsn.label); 	break; // 161
                    	case Opcodes.IF_ICMPGE:		jumpgroup(outputdata, "jae", jmpinsn.label); 	break; // 162
                    	case Opcodes.IF_ICMPGT:		jumpgroup(outputdata, "ja", jmpinsn.label); 	break; // 163
                    	case Opcodes.IF_ICMPLE:		jumpgroup(outputdata, "jbe", jmpinsn.label);	break; // 164
                    
                        case Opcodes.GOTO: // 167
                            outputdata.add("jmp .l" + jmpinsn.label.getLabel());
                            break;

                        default:
                            outputdata.add("Can't deal with jumpinsnnode: " + jmpinsn.getOpcode());
                    }
                }   
                else if (insn instanceof LdcInsnNode)
                {
                	LdcInsnNode ldcinsn = (LdcInsnNode) insn;
                	if (ldcinsn.cst instanceof Integer)
                	{
                		outputdata.add("push " + ldcinsn.cst.toString());
                	}
                	else
                	{
	                	outputdata.add("Can't deal with data in ldcinsnnode (" + ldcinsn.getOpcode() +"): " + ldcinsn.cst.getClass().getSimpleName());
                	}
                }             
                else if (insn instanceof IincInsnNode)
                {
                	IincInsnNode incinsn = (IincInsnNode) insn;
                	switch (incinsn.getOpcode())
                	{	
                	    case Opcodes.IINC:
                	    	outputdata.add("add dword [ebp - " + (4 + 4 * incinsn.var) + "], " + incinsn.incr);
                	    	break;
                		default:
                			outputdata.add("Can't deal with iincinsnnode: " + incinsn.getOpcode());                			
                	}
                	
                }
                else if (insn.getOpcode() >= Opcodes.ICONST_M1 && insn.getOpcode() <= Opcodes.ICONST_5) // 2...8
                {
                    outputdata.add("push " + (insn.getOpcode() - Opcodes.ICONST_M1 - 1));
                }
                else if (insn.getOpcode() == Opcodes.IADD) // 96
                {
                	outputdata.add("pop edx");
                	outputdata.add("add [esp], edx");
                }
                else if (insn.getOpcode() == Opcodes.IMUL) // 104
                {
                	outputdata.add("pop eax");
                	outputdata.add("pop ecx");
                	outputdata.add("imul ecx"); // eax:edx = eax * ecx
                	outputdata.add("push eax");
                }                
                else if (insn.getOpcode() == Opcodes.I2B) // 145
                {
                	outputdata.add("and dword [esp], 0xff");
                }
                else if (insn.getOpcode() == Opcodes.RETURN)	// 177
                {
                	outputdata.add("xor eax, eax");
                }
                else if (insn instanceof FrameNode)
                {
                    outputdata.add("; framenode");
                }
                else
                {
                    outputdata.add("Can't deal with " + insn.getClass().getSimpleName() + ", fix it (" + insn.getOpcode() + ")");
                }
            }

            // epilogue, stdcall to save complexity on decoding methodinsns
            outputdata.add("leave");
            outputdata.add("ret " + arguments);
        }

        writeOutput(args[1], outputdata);
    }
}

Yes, that's a compiler in a magic 256 lines of code. The result is Intel syntax assembly, because there already are decent tools for assembly out there that deal with object formats out there. Of course you can write your own later as well.