1The PPC KVM paravirtual interface 2================================= 3 4The basic execution principle by which KVM on PowerPC works is to run all kernel 5space code in PR=1 which is user space. This way we trap all privileged 6instructions and can emulate them accordingly. 7 8Unfortunately that is also the downfall. There are quite some privileged 9instructions that needlessly return us to the hypervisor even though they 10could be handled differently. 11 12This is what the PPC PV interface helps with. It takes privileged instructions 13and transforms them into unprivileged ones with some help from the hypervisor. 14This cuts down virtualization costs by about 50% on some of my benchmarks. 15 16The code for that interface can be found in arch/powerpc/kernel/kvm* 17 18Querying for existence 19====================== 20 21To find out if we're running on KVM or not, we leverage the device tree. When 22Linux is running on KVM, a node /hypervisor exists. That node contains a 23compatible property with the value "linux,kvm". 24 25Once you determined you're running under a PV capable KVM, you can now use 26hypercalls as described below. 27 28KVM hypercalls 29============== 30 31Inside the device tree's /hypervisor node there's a property called 32'hypercall-instructions'. This property contains at most 4 opcodes that make 33up the hypercall. To call a hypercall, just call these instructions. 34 35The parameters are as follows: 36 37 Register IN OUT 38 39 r0 - volatile 40 r3 1st parameter Return code 41 r4 2nd parameter 1st output value 42 r5 3rd parameter 2nd output value 43 r6 4th parameter 3rd output value 44 r7 5th parameter 4th output value 45 r8 6th parameter 5th output value 46 r9 7th parameter 6th output value 47 r10 8th parameter 7th output value 48 r11 hypercall number 8th output value 49 r12 - volatile 50 51Hypercall definitions are shared in generic code, so the same hypercall numbers 52apply for x86 and powerpc alike with the exception that each KVM hypercall 53also needs to be ORed with the KVM vendor code which is (42 << 16). 54 55Return codes can be as follows: 56 57 Code Meaning 58 59 0 Success 60 12 Hypercall not implemented 61 <0 Error 62 63The magic page 64============== 65 66To enable communication between the hypervisor and guest there is a new shared 67page that contains parts of supervisor visible register state. The guest can 68map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE. 69 70With this hypercall issued the guest always gets the magic page mapped at the 71desired location. The first parameter indicates the effective address when the 72MMU is enabled. The second parameter indicates the address in real mode, if 73applicable to the target. For now, we always map the page to -4096. This way we 74can access it using absolute load and store functions. The following 75instruction reads the first field of the magic page: 76 77 ld rX, -4096(0) 78 79The interface is designed to be extensible should there be need later to add 80additional registers to the magic page. If you add fields to the magic page, 81also define a new hypercall feature to indicate that the host can give you more 82registers. Only if the host supports the additional features, make use of them. 83 84The magic page layout is described by struct kvm_vcpu_arch_shared 85in arch/powerpc/include/asm/kvm_para.h. 86 87Magic page features 88=================== 89 90When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE, 91a second return value is passed to the guest. This second return value contains 92a bitmap of available features inside the magic page. 93 94The following enhancements to the magic page are currently available: 95 96 KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page 97 98For enhanced features in the magic page, please check for the existence of the 99feature before using them! 100 101MSR bits 102======== 103 104The MSR contains bits that require hypervisor intervention and bits that do 105not require direct hypervisor intervention because they only get interpreted 106when entering the guest or don't have any impact on the hypervisor's behavior. 107 108The following bits are safe to be set inside the guest: 109 110 MSR_EE 111 MSR_RI 112 MSR_CR 113 MSR_ME 114 115If any other bit changes in the MSR, please still use mtmsr(d). 116 117Patched instructions 118==================== 119 120The "ld" and "std" instructions are transormed to "lwz" and "stw" instructions 121respectively on 32 bit systems with an added offset of 4 to accommodate for big 122endianness. 123 124The following is a list of mapping the Linux kernel performs when running as 125guest. Implementing any of those mappings is optional, as the instruction traps 126also act on the shared page. So calling privileged instructions still works as 127before. 128 129From To 130==== == 131 132mfmsr rX ld rX, magic_page->msr 133mfsprg rX, 0 ld rX, magic_page->sprg0 134mfsprg rX, 1 ld rX, magic_page->sprg1 135mfsprg rX, 2 ld rX, magic_page->sprg2 136mfsprg rX, 3 ld rX, magic_page->sprg3 137mfsrr0 rX ld rX, magic_page->srr0 138mfsrr1 rX ld rX, magic_page->srr1 139mfdar rX ld rX, magic_page->dar 140mfdsisr rX lwz rX, magic_page->dsisr 141 142mtmsr rX std rX, magic_page->msr 143mtsprg 0, rX std rX, magic_page->sprg0 144mtsprg 1, rX std rX, magic_page->sprg1 145mtsprg 2, rX std rX, magic_page->sprg2 146mtsprg 3, rX std rX, magic_page->sprg3 147mtsrr0 rX std rX, magic_page->srr0 148mtsrr1 rX std rX, magic_page->srr1 149mtdar rX std rX, magic_page->dar 150mtdsisr rX stw rX, magic_page->dsisr 151 152tlbsync nop 153 154mtmsrd rX, 0 b <special mtmsr section> 155mtmsr rX b <special mtmsr section> 156 157mtmsrd rX, 1 b <special mtmsrd section> 158 159[Book3S only] 160mtsrin rX, rY b <special mtsrin section> 161 162[BookE only] 163wrteei [0|1] b <special wrteei section> 164 165 166Some instructions require more logic to determine what's going on than a load 167or store instruction can deliver. To enable patching of those, we keep some 168RAM around where we can live translate instructions to. What happens is the 169following: 170 171 1) copy emulation code to memory 172 2) patch that code to fit the emulated instruction 173 3) patch that code to return to the original pc + 4 174 4) patch the original instruction to branch to the new code 175 176That way we can inject an arbitrary amount of code as replacement for a single 177instruction. This allows us to check for pending interrupts when setting EE=1 178for example. 179