1KVM Lock Overview 2================= 3 41. Acquisition Orders 5--------------------- 6 7The acquisition orders for mutexes are as follows: 8 9- kvm->lock is taken outside vcpu->mutex 10 11- kvm->lock is taken outside kvm->slots_lock and kvm->irq_lock 12 13- kvm->slots_lock is taken outside kvm->irq_lock, though acquiring 14 them together is quite rare. 15 16For spinlocks, kvm_lock is taken outside kvm->mmu_lock. Everything 17else is a leaf: no other lock is taken inside the critical sections. 18 192: Exception 20------------ 21 22Fast page fault: 23 24Fast page fault is the fast path which fixes the guest page fault out of 25the mmu-lock on x86. Currently, the page fault can be fast only if the 26shadow page table is present and it is caused by write-protect, that means 27we just need change the W bit of the spte. 28 29What we use to avoid all the race is the SPTE_HOST_WRITEABLE bit and 30SPTE_MMU_WRITEABLE bit on the spte: 31- SPTE_HOST_WRITEABLE means the gfn is writable on host. 32- SPTE_MMU_WRITEABLE means the gfn is writable on mmu. The bit is set when 33 the gfn is writable on guest mmu and it is not write-protected by shadow 34 page write-protection. 35 36On fast page fault path, we will use cmpxchg to atomically set the spte W 37bit if spte.SPTE_HOST_WRITEABLE = 1 and spte.SPTE_WRITE_PROTECT = 1, this 38is safe because whenever changing these bits can be detected by cmpxchg. 39 40But we need carefully check these cases: 411): The mapping from gfn to pfn 42The mapping from gfn to pfn may be changed since we can only ensure the pfn 43is not changed during cmpxchg. This is a ABA problem, for example, below case 44will happen: 45 46At the beginning: 47gpte = gfn1 48gfn1 is mapped to pfn1 on host 49spte is the shadow page table entry corresponding with gpte and 50spte = pfn1 51 52 VCPU 0 VCPU0 53on fast page fault path: 54 55 old_spte = *spte; 56 pfn1 is swapped out: 57 spte = 0; 58 59 pfn1 is re-alloced for gfn2. 60 61 gpte is changed to point to 62 gfn2 by the guest: 63 spte = pfn1; 64 65 if (cmpxchg(spte, old_spte, old_spte+W) 66 mark_page_dirty(vcpu->kvm, gfn1) 67 OOPS!!! 68 69We dirty-log for gfn1, that means gfn2 is lost in dirty-bitmap. 70 71For direct sp, we can easily avoid it since the spte of direct sp is fixed 72to gfn. For indirect sp, before we do cmpxchg, we call gfn_to_pfn_atomic() 73to pin gfn to pfn, because after gfn_to_pfn_atomic(): 74- We have held the refcount of pfn that means the pfn can not be freed and 75 be reused for another gfn. 76- The pfn is writable that means it can not be shared between different gfns 77 by KSM. 78 79Then, we can ensure the dirty bitmaps is correctly set for a gfn. 80 81Currently, to simplify the whole things, we disable fast page fault for 82indirect shadow page. 83 842): Dirty bit tracking 85In the origin code, the spte can be fast updated (non-atomically) if the 86spte is read-only and the Accessed bit has already been set since the 87Accessed bit and Dirty bit can not be lost. 88 89But it is not true after fast page fault since the spte can be marked 90writable between reading spte and updating spte. Like below case: 91 92At the beginning: 93spte.W = 0 94spte.Accessed = 1 95 96 VCPU 0 VCPU0 97In mmu_spte_clear_track_bits(): 98 99 old_spte = *spte; 100 101 /* 'if' condition is satisfied. */ 102 if (old_spte.Accessed == 1 && 103 old_spte.W == 0) 104 spte = 0ull; 105 on fast page fault path: 106 spte.W = 1 107 memory write on the spte: 108 spte.Dirty = 1 109 110 111 else 112 old_spte = xchg(spte, 0ull) 113 114 115 if (old_spte.Accessed == 1) 116 kvm_set_pfn_accessed(spte.pfn); 117 if (old_spte.Dirty == 1) 118 kvm_set_pfn_dirty(spte.pfn); 119 OOPS!!! 120 121The Dirty bit is lost in this case. 122 123In order to avoid this kind of issue, we always treat the spte as "volatile" 124if it can be updated out of mmu-lock, see spte_has_volatile_bits(), it means, 125the spte is always atomically updated in this case. 126 1273): flush tlbs due to spte updated 128If the spte is updated from writable to readonly, we should flush all TLBs, 129otherwise rmap_write_protect will find a read-only spte, even though the 130writable spte might be cached on a CPU's TLB. 131 132As mentioned before, the spte can be updated to writable out of mmu-lock on 133fast page fault path, in order to easily audit the path, we see if TLBs need 134be flushed caused by this reason in mmu_spte_update() since this is a common 135function to update spte (present -> present). 136 137Since the spte is "volatile" if it can be updated out of mmu-lock, we always 138atomically update the spte, the race caused by fast page fault can be avoided, 139See the comments in spte_has_volatile_bits() and mmu_spte_update(). 140 1413. Reference 142------------ 143 144Name: kvm_lock 145Type: spinlock_t 146Arch: any 147Protects: - vm_list 148 149Name: kvm_count_lock 150Type: raw_spinlock_t 151Arch: any 152Protects: - hardware virtualization enable/disable 153Comment: 'raw' because hardware enabling/disabling must be atomic /wrt 154 migration. 155 156Name: kvm_arch::tsc_write_lock 157Type: raw_spinlock 158Arch: x86 159Protects: - kvm_arch::{last_tsc_write,last_tsc_nsec,last_tsc_offset} 160 - tsc offset in vmcb 161Comment: 'raw' because updating the tsc offsets must not be preempted. 162 163Name: kvm->mmu_lock 164Type: spinlock_t 165Arch: any 166Protects: -shadow page/shadow tlb entry 167Comment: it is a spinlock since it is used in mmu notifier. 168 169Name: kvm->srcu 170Type: srcu lock 171Arch: any 172Protects: - kvm->memslots 173 - kvm->buses 174Comment: The srcu read lock must be held while accessing memslots (e.g. 175 when using gfn_to_* functions) and while accessing in-kernel 176 MMIO/PIO address->device structure mapping (kvm->buses). 177 The srcu index can be stored in kvm_vcpu->srcu_idx per vcpu 178 if it is needed by multiple functions. 179 180Name: blocked_vcpu_on_cpu_lock 181Type: spinlock_t 182Arch: x86 183Protects: blocked_vcpu_on_cpu 184Comment: This is a per-CPU lock and it is used for VT-d posted-interrupts. 185 When VT-d posted-interrupts is supported and the VM has assigned 186 devices, we put the blocked vCPU on the list blocked_vcpu_on_cpu 187 protected by blocked_vcpu_on_cpu_lock, when VT-d hardware issues 188 wakeup notification event since external interrupts from the 189 assigned devices happens, we will find the vCPU on the list to 190 wakeup. 191