Preface

This article mainly records my study and research on Intel x86 CPU VMX hardware virtualization technology.

The reference list is as follows:

Barriers to x86 Virtualization

Definition of sensitive instructions:

Gerald J. Popek and Robert P. Goldberg pointed out that instructions that modify system resources, or behave differently in different modes, are sensitive instructions. In a virtualization scenario, the VMM needs to monitor these sensitive instructions.

Definition of privileged instructions:

In a virtualizable architecture, every sensitive instruction is also privileged, meaning that executing such instructions at a non-privileged level causes the CPU to raise an exception and enter the VMM’s exception handler, thereby allowing control over the VM’s access to sensitive resources.

However, x86 does not satisfy this rule. Not all sensitive instructions on x86 are privileged instructions; some sensitive instructions do not raise an exception when executed in non-privileged mode, so the VMM cannot intercept and handle the VM’s behavior.

We can use changing the IF (Interrupt Flag) bit in the FLAGS register as an example:

  1. First, use the pushf instruction to push the FLAGS register onto the stack, then clear IF at the top of the stack, and finally use popf to restore FLAGS from the stack.
  2. If the guest kernel is not running at ring 0, the x86 CPU will not raise an exception; instead, it will silently ignore popf, so the VM’s attempt to clear IF does not take effect.

Some people proposed paravirtualization, which requires modifying guest code, but that violates the transparency principle of virtualization.

Later, binary translation was proposed, including static translation and dynamic translation:

  • Static translation: scan the entire executable before runtime, translate sensitive instructions, and produce a new file. However, static translation must be done ahead of time, and for some instructions whose side effects only appear at runtime, it cannot handle them statically.

  • Dynamic translation: dynamically modify binary code at runtime in code-block units. Dynamic translation is used in many VMMs and performs very well.

Introduction to Intel VMX

Intel developed VT technology to support virtualization and added Virtual-Machine Extensions, or VMX, to the CPU.

Two operating modes:

  • VMX root operation (host mode)
  • VMX non-root operation (guest mode)

Each mode supports ring 0 through ring 3.

As shown below:

┌──────────────────┐            ┌──────────────────┐
│ring3             │            │ring3             │
│                  │            │                  │
│  ┌─────────────┐ │            │ ┌──────────────┐ │
│  │  Host App   │ │  VM Entry  │ │   Guest App  │ │
│  └─────────────┘ │ ─────────► │ └──────────────┘ │
│ ──────────────── │            │ ─────────────────┤
│  ┌─────────────┐ │            │ ┌──────────────┐ │
│  │ Host Kernel │ │  VM Exit   │ │ Guest Kernel │ │
│  └─────────────┘ │ ◄───────── │ └──────────────┘ │
│                  │            │                  │
│ring0             │            │ring0             │
└──────────────────┘            └──────────────────┘
   VMX Root Mode                  VMX non-Root Mode

Mode switches:

  • VM Entry: switch from VMX root mode to VMX non-root mode
  • VM Exit: switch from VMX non-root mode to VMX root mode

One data structure:

  • Virtual-Machine Control Structure (VMCS)