PE: Difference between revisions

From OSDev.wiki
Jump to navigation Jump to search
[unchecked revision][unchecked revision]
Content added Content deleted
(Added overview section)
Line 19: Line 19:


== PE header ==
== PE header ==
The PE header contains information that concerns the entire file rather than individual pieces that will be coming up later. The bare minimum header contains a 4-byte signature (0x000011C6), the machine type/architecture of the executable code inside, a time stamp, a pointer to symbols, as well as various flags (is the file an executable, DLL, can the application handle addresses above 2GB, does the file needed be copy to the swap file if ran from a removable device, etc). Unless you're using a really stripped down statically linked PE file to save memory with a hard coded entry point and no resources, then the PE header alone isn't enough.
The PE header contains information that concerns the entire file rather than individual pieces that will be coming up later. The bare minimum header contains a 4-byte signature (0x00004550), the machine type/architecture of the executable code inside, a time stamp, a pointer to symbols, as well as various flags (is the file an executable, DLL, can the application handle addresses above 2GB, does the file needed be copy to the swap file if ran from a removable device, etc). Unless you're using a really stripped down statically linked PE file to save memory with a hard coded entry point and no resources, then the PE header alone isn't enough.

<source lang="c">
// 1 byte aligned
struct PeHeader {
uint32 mMagic; // PE\0\0 or 0x00004550
uint16 mMachine;
uint16 mNumberOfSections;
uint32 mTimeDateStamp;
uint32 mPointerToSymbolTable;
uint32 mNumberOfSymbols;
uint16 mSizeOfOptionalHeader;
uint16 mCharacteristics;
};
</source>


=== Optional header ===
=== Optional header ===

Revision as of 15:24, 13 October 2012

Executable Formats
Microsoft

16 bit:
COM
MZ
NE
Mixed (16/32 bit):
LE
32/64 bit:
PE
COFF

*nix
Apple

With Windows 95/NT, a new executable file type was required. Thus was born the "PE" Portable Executable, which is still in use. Unlike its predecessors, WIN-PE is a true 32bit file format, supporting relocatable code. It does distinguish between TEXT, DATA, and BSS. It is, in fact, a bastardized version of the COFF format.

If you did set up a Cygwin environment on your Windows machine, "PE" is the target format for your Cygwin GCC toolchain, which causes the unaware some headache when trying to link parts build under Cygwin with parts build under Linux or BSD (which use the ELF target by default). (Hint: You have to build a GCC Cross-Compiler)

The PE format is used by Windows 95 and higher, Windows NT 3.1 and higher, the Mobius and ReactOS. The PE format is also used as a container for .NET assemblies by both the Microsoft .Net CLI and Mono. In which case it doesn't actually store executable data but .Net metadata with IL attached to it.

Inside the PE file

Below will attempt to explain the various concepts and parts that make up a PE file rather than the exact data structures inside of them since that would certainly take up too much space. The reasoning is that most resources on PE files tend to just dump a bunch of data structures in your face without fully explaining what they are for. So by reading the following and knowing what a PE file consists of you will have a better understanding of what to expect and how to utilize PE file resources.

Overview

The PE file consists of several parts. They are briefly sumarized below, then more detail is entered on each part.

An overview of the format


DOS Stub

The PE format begins with a MS-DOS stub (a header plus executable code) which makes it a valid MS-DOS executable. The MS-DOS header begins with the magic code 0x5A4D and is 64 bytes long, followed by real-mode executable code. The standard stub used almost universally is 128-bytes long (including header and executable code) and simply outputs "This program cannot be run in DOS mode." Despite many utilities that with PE files are hard coded to expect the PE header to start at exactly 128 bytes in, this is incorrect since in some linkers, including Microsoft's own Link, it is possible to replace the MS-DOS stub with one of your own choosing, and many older programs did this to allow the developer to bundle a MS-DOS and Windows version into a single file. The correct way is to read a formerly reserved 4-byte address inside the MS-DOS header located at 0x3C which contains the address at which the PE header begins.

PE header

The PE header contains information that concerns the entire file rather than individual pieces that will be coming up later. The bare minimum header contains a 4-byte signature (0x00004550), the machine type/architecture of the executable code inside, a time stamp, a pointer to symbols, as well as various flags (is the file an executable, DLL, can the application handle addresses above 2GB, does the file needed be copy to the swap file if ran from a removable device, etc). Unless you're using a really stripped down statically linked PE file to save memory with a hard coded entry point and no resources, then the PE header alone isn't enough.

// 1 byte aligned
struct PeHeader {
	uint32 mMagic; // PE\0\0 or 0x00004550
	uint16 mMachine;
	uint16 mNumberOfSections;
	uint32 mTimeDateStamp;
	uint32 mPointerToSymbolTable;
	uint32 mNumberOfSymbols;
	uint16 mSizeOfOptionalHeader;
	uint16 mCharacteristics;
};

Optional header

The optional PE header follows directly after the standard PE header. It's size is specified in the PE header which you can also use to tell if the optional header exists. The optional PE header begins with a 2-byte magic code representing the architecture (0x010B for PE32, 0x020B for PE64, 0x0107 ROM). This can be used in conjunction with the machine type to see in the PE header to detect if the PE file is running on a compatible system. There are a few other useful memory-related variables including the size and virtual base of the code and data, as well as the application's version number (completely user specified, some update utilities use this to detect if a newer version is available), entry point, and how many directories there are (see below).

Part of the optional header is NT-specific. This include the subsystem (console, driver, or GUI application), how much stack and heap space to reserve, and the minimum required Operating System, subsystem and Windows version. You can use your own values for all of these depending on the needs of your OS.

Data Directories

While technically part of the optional header and follows directly after it is a list of entries pointing to data directories. Because the optional header can vary in size, you only need to pay attention to the directories that exist and you expect, since it is likely that new data directories will be added to the PE specification in the future (.Net is an example of one that was recently added). Each data directory is referenced as an 8-byte entry in the optional header. The first 4 bytes is the Relative Virtual Address, or RVA (see Sections below), of the directory, and the last 4 bytes is the size of the directory.

Each data directory that the entries point to have their own format. Data directories are used to describe import tables for dynamic linking, a table of resources that are embedded inside of the PE file, debug information (line numbers and break points), the CLI .Net header.

Sections

A PE file is made up of sections which consist of a name, offset within the file, virtual address to copy to, as well as the size of the section in the file and in virtual memory (which may differ, in which case the difference should be cleared 0s), and associated flags. The sections usually follow universal naming (".text", ".rsrc", etc), but this can also vary between linker and in some cases can be user-defined, so it is better to depend on the flags to tell if a section is executable or writable. However in saying that, if you have custom data that you wish to embed inside of the executable, then placing it inside of a section and identifying it by the section's name can be a good idea since you won't be changing the PE format and your executable will remain compatible with PE tools.

The Relative Virtual Base is a phrase that comes up a lot in PE documentation. The RVA is the address at where something exists once it's loaded into memory, rather than an offset into the file. To calculate the file's address from an RVA without actually loading the sections into memory, you can use the table of section entries. By using the virtual address and size of each section you can find which section the RVA belongs to, then subtract the difference between the section's virtual address and file offset.

In asm linkage

if in nasm you declare a block of code like this:

segment .code
aAsmFunction:
;Do whatever
mov BYTE[aData], 0
ret
segment .data
aData: db 0xFF

The segments will apear as sections. Using this it is possible to keep C and Asm seperate, as a linker will not automatically merge .code and .text, which is the normal output by C compilers.

Position Independent Code

If each section specifies which virtual address to load it in to, you may be wondering how multiple DLLs can exist in one virtual address space without conflict. It is true that most code you'll find in a PE file (DLL or otherwise) is position dependent and linked to a specific address. However to resolve this issue there exist a structure called a Relocation Table that is attached to each section entry. The table is basically a HUGE long list of every address stored in that section so you can offset it to the location where you loaded the section.

Because addresses can point across section borders, relocations should be done after each section is loaded into memory. Then reiterate over each section, iterate through each address in the Relocation Table, find out what section that RVA exists in and add/subtract the offset between that section's linked virtual address and the section's virtual address you loaded it into.

CLI / .Net

CLI works alongside the PE format. Rather than being an extension to the format, it really exists as its own format inside of a format with a completely different way of storing tables and values. All the .Net data and headers exist inside of sections that are loaded into memory (they are loaded into memory since CLI involves heavy language reflection requiring the metadata without thrashing the disk). The second reason that the .Net metadata exists inside of the sections rather than the PE headers is because the PE loader actually has no concept of .Net at all. (Exception: There is a data directory entry pointing to the RVA of the CLI header so tools can easily access the .Net data without loading it into virtual memory.) In fact, I highly doubt the Windows kernel has any sort of concept of .Net at all. The way .Net works is by dynamically linking against the .Net runtime (mscoree.dll), and setting the entry point to a symbol (_CorExeMain) that resolves to a location inside of mscoree.dll instead of the local executable. This means that Windows CE, WINE, and ReactOS can all load .Net assemblies once the .Net framework can be installed without any specific code.

Loading a PE file

To load a PE file is quite simple;

1. Extract from the header the entry point, heap and stack sizes.

2. Iterate through each section and copy it from the file into virtual memory (although not required, it is good to clear the difference between the section size in memory and in the file to 0).

3. Find the address of the entry point by finding the correct entry in the symbol table.

4. Create a new thread at that address and begin executing!

To load a PE file that requires a dynamic DLL you can do the same, but check the Import Table (referred to by the data directory) to find what symbols and PE files are required, the Export Table (also referred to by the data directory) inside of that PE file to see where those symbols are and match them up once you've loaded that PE's sections into memory (and relocated them!) And lastly, beware that you'll have to recursively resolve each DLL's Import Tables a swell, and some DLLs can use tricks to reference a symbol in the DLL loading it so make sure you don't get your loader stuck in a loop! Registering symbols loaded and making them global might be a good solution.

It may also be a good idea to check the Machine and Magic fields for validity, not just the PE signature. This way your loader won't try loading a 64 bit binary into 32 bit mode (this would be certain to cause an exception).

64 bit PE

64 bit PE's are extremely similar to normal PE's, but the machine type, if AMD64, is 0x8664, not 0x14c. This field is directly after the PE signature. The magic number also changes from 0x10b to 0x20b. The magic field is at the beginning of the optional header.

See Also