Virtual 8086 Mode: Difference between revisions

The only way to set the VM flag is to use the iret instruction. This instruction is normally used to return from an interrupt. When executing an iret, the CPU pops eip, cs, eflags, esp, ss from the stack and continues executing at the new eip.

; you should declare this function as :

; extern void enter_v86(uint32_t ss, uint32_t esp, uint32_t cs, uint32_t eip);

push dword [ebp+16] ; eip

iret

==V86 Problem==

EFLAGS.VM is NEVER pushed onto the stack if the V86 task uses PUSHFD. You should check if CR0.PE=1 and then assume it's V86 if that bit is set.

detect_v86:

smsw ax

and eax,1 ;CR0.PE bit

ret

VM mode detection is mainly useful when writing DOS extenders or other programs that could be started either in plain real mode or in virtual mode from a protected mode system. An 'ordinary' bootloader shouldn't worry about this since the BIOS will not set up VM86 to read the bootsector ;)