1x86 Topology 2============ 3 4This documents and clarifies the main aspects of x86 topology modelling and 5representation in the kernel. Update/change when doing changes to the 6respective code. 7 8The architecture-agnostic topology definitions are in 9Documentation/cputopology.txt. This file holds x86-specific 10differences/specialities which must not necessarily apply to the generic 11definitions. Thus, the way to read up on Linux topology on x86 is to start 12with the generic one and look at this one in parallel for the x86 specifics. 13 14Needless to say, code should use the generic functions - this file is *only* 15here to *document* the inner workings of x86 topology. 16 17Started by Thomas Gleixner <tglx@linutronix.de> and Borislav Petkov <bp@alien8.de>. 18 19The main aim of the topology facilities is to present adequate interfaces to 20code which needs to know/query/use the structure of the running system wrt 21threads, cores, packages, etc. 22 23The kernel does not care about the concept of physical sockets because a 24socket has no relevance to software. It's an electromechanical component. In 25the past a socket always contained a single package (see below), but with the 26advent of Multi Chip Modules (MCM) a socket can hold more than one package. So 27there might be still references to sockets in the code, but they are of 28historical nature and should be cleaned up. 29 30The topology of a system is described in the units of: 31 32 - packages 33 - cores 34 - threads 35 36* Package: 37 38 Packages contain a number of cores plus shared resources, e.g. DRAM 39 controller, shared caches etc. 40 41 AMD nomenclature for package is 'Node'. 42 43 Package-related topology information in the kernel: 44 45 - cpuinfo_x86.x86_max_cores: 46 47 The number of cores in a package. This information is retrieved via CPUID. 48 49 - cpuinfo_x86.phys_proc_id: 50 51 The physical ID of the package. This information is retrieved via CPUID 52 and deduced from the APIC IDs of the cores in the package. 53 54 - cpuinfo_x86.logical_id: 55 56 The logical ID of the package. As we do not trust BIOSes to enumerate the 57 packages in a consistent way, we introduced the concept of logical package 58 ID so we can sanely calculate the number of maximum possible packages in 59 the system and have the packages enumerated linearly. 60 61 - topology_max_packages(): 62 63 The maximum possible number of packages in the system. Helpful for per 64 package facilities to preallocate per package information. 65 66 67* Cores: 68 69 A core consists of 1 or more threads. It does not matter whether the threads 70 are SMT- or CMT-type threads. 71 72 AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses 73 "core". 74 75 Core-related topology information in the kernel: 76 77 - smp_num_siblings: 78 79 The number of threads in a core. The number of threads in a package can be 80 calculated by: 81 82 threads_per_package = cpuinfo_x86.x86_max_cores * smp_num_siblings 83 84 85* Threads: 86 87 A thread is a single scheduling unit. It's the equivalent to a logical Linux 88 CPU. 89 90 AMDs nomenclature for CMT threads is "Compute Unit Core". The kernel always 91 uses "thread". 92 93 Thread-related topology information in the kernel: 94 95 - topology_core_cpumask(): 96 97 The cpumask contains all online threads in the package to which a thread 98 belongs. 99 100 The number of online threads is also printed in /proc/cpuinfo "siblings." 101 102 - topology_sibling_mask(): 103 104 The cpumask contains all online threads in the core to which a thread 105 belongs. 106 107 - topology_logical_package_id(): 108 109 The logical package ID to which a thread belongs. 110 111 - topology_physical_package_id(): 112 113 The physical package ID to which a thread belongs. 114 115 - topology_core_id(); 116 117 The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo 118 "core_id." 119 120 121 122System topology examples 123 124Note: 125 126The alternative Linux CPU enumeration depends on how the BIOS enumerates the 127threads. Many BIOSes enumerate all threads 0 first and then all threads 1. 128That has the "advantage" that the logical Linux CPU numbers of threads 0 stay 129the same whether threads are enabled or not. That's merely an implementation 130detail and has no practical impact. 131 1321) Single Package, Single Core 133 134 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 135 1362) Single Package, Dual Core 137 138 a) One thread per core 139 140 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 141 -> [core 1] -> [thread 0] -> Linux CPU 1 142 143 b) Two threads per core 144 145 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 146 -> [thread 1] -> Linux CPU 1 147 -> [core 1] -> [thread 0] -> Linux CPU 2 148 -> [thread 1] -> Linux CPU 3 149 150 Alternative enumeration: 151 152 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 153 -> [thread 1] -> Linux CPU 2 154 -> [core 1] -> [thread 0] -> Linux CPU 1 155 -> [thread 1] -> Linux CPU 3 156 157 AMD nomenclature for CMT systems: 158 159 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0 160 -> [Compute Unit Core 1] -> Linux CPU 1 161 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2 162 -> [Compute Unit Core 1] -> Linux CPU 3 163 1644) Dual Package, Dual Core 165 166 a) One thread per core 167 168 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 169 -> [core 1] -> [thread 0] -> Linux CPU 1 170 171 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2 172 -> [core 1] -> [thread 0] -> Linux CPU 3 173 174 b) Two threads per core 175 176 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 177 -> [thread 1] -> Linux CPU 1 178 -> [core 1] -> [thread 0] -> Linux CPU 2 179 -> [thread 1] -> Linux CPU 3 180 181 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 4 182 -> [thread 1] -> Linux CPU 5 183 -> [core 1] -> [thread 0] -> Linux CPU 6 184 -> [thread 1] -> Linux CPU 7 185 186 Alternative enumeration: 187 188 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 189 -> [thread 1] -> Linux CPU 4 190 -> [core 1] -> [thread 0] -> Linux CPU 1 191 -> [thread 1] -> Linux CPU 5 192 193 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2 194 -> [thread 1] -> Linux CPU 6 195 -> [core 1] -> [thread 0] -> Linux CPU 3 196 -> [thread 1] -> Linux CPU 7 197 198 AMD nomenclature for CMT systems: 199 200 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0 201 -> [Compute Unit Core 1] -> Linux CPU 1 202 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2 203 -> [Compute Unit Core 1] -> Linux CPU 3 204 205 [node 1] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 4 206 -> [Compute Unit Core 1] -> Linux CPU 5 207 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 6 208 -> [Compute Unit Core 1] -> Linux CPU 7 209