Timekeeping in virtual machines

This page is a work in progress.

This page may thus be incomplete. Its content may be changed in the near future.

There are several ways to keep track of time in a VM, but they're either very slow (e.g. HPET) or do not work correctly if the VM is migrated (e.g. TSC).

To work around this, VMs such as QEMU/KVM provide several ways keep track of time whilst sacrificing little performance.

KVM_HC_CLOCK_PAIRING

This hypercall is used to get the parameters to calculate a host's clock (KVM_CLOCK_PAIRING_WALLCLOCK for CLOCK_REALTIME).

The host copies the following structure to a physical address given by the guest:

struct kvm_clock_pairing {
    s64 sec;
    s64 nsec;
    u64 tsc;
    u32 flags;
    u32 pad[9];
};

A hypercall is performed with the `vmcall` instruction. On KVM, RBX, RCX, RDX and RSI are used for arguments, RAX as the hypercall number and as the return value. No other registers are clobbered (unless explicitly noted).

For example, calling KVM_HC_CLOCK_PAIRING can be done as follows on x86_64:

; rdi: physical address to copy structure to
; rsi: clock type (KVM_CLOCK_PAIRING_WALLCLOCK = 0)
kvm_hc_clock_pairing:
    mov eax, 9   ; KVM_HC_CLOCK_PAIRING
    mov rbx, rdi
    mov rcx, rsi
    vmcall
    ret

pvclock

pvclock is a simple protocol and the fastest way to properly track system time in a VM.

To use it, write a 64-bit 4-byte aligned physical address with bit 0 set to 1 to MSR_KVM_SYSTEM_TIME_NEW (0x4b564d01). The presence of this MSR is indicated by bit 3 in EAX from leaf 0x4000001 of CPUID.

The host will write the following structure to this address:

struct pvclock_vcpu_time_info {
    u32 version;
    u32 pad0;
    u64 tsc_timestamp;
    u64 system_time;
    u32 tsc_to_system_mul;
    s8 tsc_shift;
    u8 flags;
    u8 pad[2];
};

The host will automatically update this structure when necessary (e.g. when finishing a migration).

The system time in nanoseconds is calculated as such:

time = rdtsc() - tsc_timestamp
if (tsc_shift >= 0)
    time <<= tsc_shift;
else
    time >>= -tsc_shift;
time = (time * tsc_to_system_mul) >> 32
time = time + system_time

The version field is used to detect when the structure has been / is being updated. If the version is odd an update is in progress and the guest must not read the other fields yet.

Hyper-V TSC page

struct ms_hyperv_tsc_page {
    volatile u32 tsc_sequence;
    u32 reserved1;
    volatile u64 tsc_scale;
    volatile s64 tsc_offset;
    u64 reserved2[509];
};

See also

References

• Linux KVM Hypercall
• KVM-specific MSRs
• An introduction to timekeeping in Linux VMs