1# Memory Layout 2 3## x86-64 guest physical memory map 4 5This is a survey of the existing memory layout for crosvm on x86-64 when booting a Linux kernel. Some of these values are different when booting a BIOS image or when compiled with features=direct (ManaTEE); see the source. All addresses are in hexadecimal. 6 7| Name/source link | Address | End (exclusive) | Size | Notes | 8| ---------------------------- | ------------- | --------------- | --------- | ---------------------------------------------------------------------------------------- | 9| | `0000` | `7000` | | RAM (may start at 0x1000 for crosvm-direct) | 10| [`ZERO_PAGE_OFFSET`] | `7000` | | | Linux boot_params structure | 11| [`BOOT_STACK_POINTER`] | `8000` | | | Boot SP value | 12| [`boot_pml4_addr`] | `9000` | | | Boot page table | 13| [`boot_pdpte_addr`] | `A000` | | | Boot page table | 14| [`boot_pde_addr`] | `B000` | | | Boot page table | 15| [`CMDLINE_OFFSET`] | `2_0000` | `2_0800` | 2 KiB | Linux kernel command line | 16| [`SETUP_DATA_START`] | `2_0800` | `E_0000` | 766 KiB | Linux kernel `setup_data` linked list | 17| [`ACPI_HI_RSDP_WINDOW_BASE`] | `E_0000` | | | ACPI tables | 18| [`KERNEL_START_OFFSET`] | `20_0000` | | | Linux kernel image load address | 19| [`initrd_start`] | after kernel | | | Initial RAM disk for Linux kernel (optional) | 20| [`END_ADDR_BEFORE_32BITS`] | after initrd | `D000_0000` | ~3.24 GiB | RAM (\<4G) | 21| [`END_ADDR_BEFORE_32BITS`] | `D000_0000` | `F400_0000` | 576 MiB | Low (\<4G) MMIO allocation area | 22| [`PCIE_CFG_MMIO_START`] | `F400_0000` | `F800_0000` | 64 MiB | PCIe enhanced config (ECAM) | 23| [`RESERVED_MEM_SIZE`] | `F800_0000` | `1_0000_0000` | 128 MiB | LAPIC/IOAPIC/HPET/… | 24| [`TSS_ADDR`] | `FFFB_D000` | | | Boot task state segment | 25| | `1_0000_0000` | | | RAM (>4G) | 26| | (end of RAM) | | | High (>4G) MMIO allocation area | 27 28[`zero_page_offset`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=338?q=ZERO_PAGE_OFFSET 29[`boot_stack_pointer`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=332?q=BOOT_STACK_POINTER 30[`boot_pml4_addr`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/regs.rs;l=299?q=boot_pml4_addr 31[`boot_pdpte_addr`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/regs.rs;l=300?q=boot_pdpte_addr 32[`boot_pde_addr`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/regs.rs;l=301?q=boot_pde_addr 33[`cmdline_offset`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=342?q=CMDLINE_OFFSET 34[`setup_data_start`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=344?q=SETUP_DATA_START 35[`acpi_hi_rsdp_window_base`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=357?q=ACPI_HI_RSDP_WINDOW_BASE 36[`kernel_start_offset`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=341?q=KERNEL_START_OFFSET 37[`initrd_start`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=1633?q=initrd_start 38[`end_addr_before_32bits`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=230?q=END_ADDR_BEFORE_32BITS 39[`pcie_cfg_mmio_start`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=400?q=PCIE_CFG_MMIO_START 40[`reserved_mem_size`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=395?q=RESERVED_MEM_SIZE 41[`tss_addr`]: https://crsrc.org/o/src/platform/crosvm/x86_64/src/lib.rs;l=339?q=TSS_ADDR 42 43## aarch64 guest physical memory map 44 45All addresses are IPA in hexadecimal. 46 47### Common layout 48 49These apply for all boot modes. 50 51| Name/source link | Address | End (exclusive) | Size | Notes | 52| --------------------------------- | --------------- | --------------- | ---------- | ------------------------------------------------------------- | 53| [`SERIAL_ADDR[3]`][serial_addr] | `2e8` | `2f0` | 8 bytes | Serial port MMIO | 54| [`SERIAL_ADDR[1]`][serial_addr] | `2f8` | `300` | 8 bytes | Serial port MMIO | 55| [`SERIAL_ADDR[2]`][serial_addr] | `3e8` | `3f0` | 8 bytes | Serial port MMIO | 56| [`SERIAL_ADDR[0]`][serial_addr] | `3f8` | `400` | 8 bytes | Serial port MMIO | 57| [`AARCH64_RTC_ADDR`] | `2000` | `3000` | 4 KiB | Real-time clock | 58| [`AARCH64_VMWDT_ADDR`] | `3000` | `4000` | 4 KiB | Watchdog device | 59| [`AARCH64_PCI_CFG_BASE`] | `1_0000` | `2_0000` | 64 KiB | PCI configuration (CAM) | 60| [`AARCH64_PVTIME_IPA_START`] | `1f0_0000` | `200_0000` | 64 KiB | Paravirtualized time | 61| [`AARCH64_MMIO_BASE`] | `200_0000` | `400_0000` | 32 MiB | Low MMIO allocation area | 62| [`AARCH64_GIC_CPUI_BASE`] | `3ffd_0000` | `3fff_0000` | 128 KiB | vGIC | 63| [`AARCH64_GIC_DIST_BASE`] | `3fff_0000` | `4000_0000` | 64 KiB | vGIC | 64| [`AARCH64_AXI_BASE`] | `4000_0000` | | | Seemingly unused? Is this hard-coded somewhere in the kernel? | 65| [`AARCH64_PROTECTED_VM_FW_START`] | `7fc0_0000` | `8000_0000` | 4 MiB | pVM firmware (if running a protected VM) | 66| [`AARCH64_PHYS_MEM_START`] | `8000_0000` | | --mem size | RAM (starts at IPA = 2 GiB) | 67| [`get_swiotlb_addr`] | after RAM | | --swiotlb size | Only present for hypervisors requiring static swiotlb alloc | 68| [`plat_mmio_base`] | after swiotlb | +0x800000 | 8 MiB | Platform device MMIO region | 69| [`high_mmio_base`] | after plat_mmio | max phys addr | | High MMIO allocation area | 70 71### Layout when booting a kernel 72 73These apply when no bootloader is passed, so crosvm boots a kernel directly. 74 75| Name/source link | Address | End (exclusive) | Size | Notes | 76| ------------------------- | ----------------- | --------------- | ----- | ---------------------------- | 77| [`AARCH64_KERNEL_OFFSET`] | `8000_0000` | | | Kernel load location in RAM | 78| [`initrd_addr`] | after kernel | | | Linux initrd location in RAM | 79| [`fdt_address`] | before end of RAM | | 2 MiB | Flattened device tree in RAM | 80 81### Layout when booting a bootloader 82 83These apply when a bootloader is passed with `--bios`. 84 85| Name/source link | Address | End (exclusive) | Size | Notes | 86| ----------------------------------- | ----------- | --------------- | ----- | ---------------------------- | 87| [`AARCH64_FDT_OFFSET_IN_BIOS_MODE`] | `8000_0000` | `8020_0000` | 2 MiB | Flattened device tree in RAM | 88| [`AARCH64_BIOS_OFFSET`] | `8020_0000` | | | Bootloader image in RAM | 89 90[serial_addr]: https://crsrc.org/o/src/platform/crosvm/arch/src/serial.rs;l=78?q=SERIAL_ADDR 91[`aarch64_rtc_addr`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=177?q=AARCH64_RTC_ADDR 92[`aarch64_vmwdt_addr`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=187?q=AARCH64_VMWDT_ADDR 93[`aarch64_pci_cfg_base`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=192?q=AARCH64_PCI_CFG_BASE 94[`aarch64_mmio_base`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=196?q=AARCH64_MMIO_BASE 95[`aarch64_gic_cpui_base`]: https://crsrc.org/o/src/platform/crosvm/devices/src/irqchip/kvm/aarch64.rs;l=106?q=AARCH64_GIC_CPUI_BASE 96[`aarch64_gic_dist_base`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=105?q=AARCH64_GIC_DIST_BASE 97[`aarch64_axi_base`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=86?q=AARCH64_AXI_BASE 98[`aarch64_pvtime_ipa_start`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=100?q=AARCH64_PVTIME_IPA_START 99[`aarch64_protected_vm_fw_start`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=96?q=AARCH64_PROTECTED_VM_FW_START 100[`aarch64_phys_mem_start`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=85?q=AARCH64_PHYS_MEM_START 101[`get_swiotlb_addr`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs?q=get_swiotlb_addr 102[`plat_mmio_base`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=835?q=plat_mmio_base 103[`high_mmio_base`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=838?q=high_mmio_base 104[`aarch64_kernel_offset`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=76?q=AARCH64_KERNEL_OFFSET 105[`initrd_addr`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=409?q=initrd_addr 106[`fdt_address`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=301?q=fdt_address 107[`aarch64_fdt_offset_in_bios_mode`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=90?q=AARCH64_FDT_OFFSET_IN_BIOS_MODE 108[`aarch64_bios_offset`]: https://crsrc.org/o/src/platform/crosvm/aarch64/src/lib.rs;l=92?q=AARCH64_BIOS_OFFSET 109