1<html> 2<head> 3<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 4<title>NUMA</title> 5<link rel="stylesheet" href="../../../../../doc/src/boostbook.css" type="text/css"> 6<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 7<link rel="home" href="../index.html" title="Chapter 1. Fiber"> 8<link rel="up" href="../index.html" title="Chapter 1. Fiber"> 9<link rel="prev" href="speculation.html" title="Specualtive execution"> 10<link rel="next" href="gpu_computing.html" title="GPU computing"> 11</head> 12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 13<table cellpadding="2" width="100%"><tr> 14<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../boost.png"></td> 15<td align="center"><a href="../../../../../index.html">Home</a></td> 16<td align="center"><a href="../../../../../libs/libraries.htm">Libraries</a></td> 17<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 18<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 19<td align="center"><a href="../../../../../more/index.htm">More</a></td> 20</tr></table> 21<hr> 22<div class="spirit-nav"> 23<a accesskey="p" href="speculation.html"><img src="../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../index.html"><img src="../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="gpu_computing.html"><img src="../../../../../doc/src/images/next.png" alt="Next"></a> 24</div> 25<div class="section"> 26<div class="titlepage"><div><div><h2 class="title" style="clear: both"> 27<a name="fiber.numa"></a><a name="numa"></a><a class="link" href="numa.html" title="NUMA">NUMA</a> 28</h2></div></div></div> 29<p> 30 Modern micro-processors contain integrated memory controllers that are connected 31 via channels to the memory. Accessing the memory can be organized in two kinds:<br> 32 Uniform Memory Access (UMA) and Non-Uniform Memory Access (NUMA). 33 </p> 34<p> 35 In contrast to UMA, that provides a centralized pool of memory (and thus does 36 not scale after a certain number of processors), a NUMA architecture divides 37 the memory into local and remote memory relative to the micro-processor.<br> 38 Local memory is directly attached to the processor's integrated memory controller. 39 Memory connected to the memory controller of another micro-processor (multi-socket 40 systems) is considered as remote memory. If a memory controller access remote 41 memory it has to traverse the interconnect<a href="#ftn.fiber.numa.f0" class="footnote" name="fiber.numa.f0"><sup class="footnote">[8]</sup></a> and connect to the remote memory controller.<br> Thus accessing 42 remote memory adds additional latency overhead to local memory access. Because 43 of the different memory locations, a NUMA-system experiences <span class="emphasis"><em>non-uniform</em></span> 44 memory access time.<br> As a consequence the best performance is achieved 45 by keeping the memory access local. 46 </p> 47<p> 48 <span class="inlinemediaobject"><img src="../../../../../libs/fiber/doc/NUMA.png" align="middle" alt="NUMA"></span> 49 </p> 50<h4> 51<a name="fiber.numa.h0"></a> 52 <span class="phrase"><a name="fiber.numa.numa_support_in_boost_fiber"></a></span><a class="link" href="numa.html#fiber.numa.numa_support_in_boost_fiber">NUMA 53 support in Boost.Fiber</a> 54 </h4> 55<p> 56 Because only a subset of the NUMA-functionality is exposed by several operating 57 systems, Boost.Fiber provides only a minimalistic NUMA API. 58 </p> 59<div class="important"><table border="0" summary="Important"> 60<tr> 61<td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="../../../../../doc/src/images/important.png"></td> 62<th align="left">Important</th> 63</tr> 64<tr><td align="left" valign="top"><p> 65 In order to enable NUMA support, b2 property <code class="computeroutput"><span class="identifier">numa</span><span class="special">=</span><span class="identifier">on</span></code> must 66 be specified and linked against additional library <code class="computeroutput"><span class="identifier">libboost_fiber_numa</span><span class="special">.</span><span class="identifier">so</span></code>. 67 </p></td></tr> 68</table></div> 69<div class="important"><table border="0" summary="Important"> 70<tr> 71<td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="../../../../../doc/src/images/important.png"></td> 72<th align="left">Important</th> 73</tr> 74<tr><td align="left" valign="top"><p> 75 MinGW using pthread implementation is not supported on Windows. 76 </p></td></tr> 77</table></div> 78<div class="table"> 79<a name="fiber.numa.supported_functionality_operating_systems"></a><p class="title"><b>Table 1.1. Supported functionality/operating systems</b></p> 80<div class="table-contents"><table class="table" summary="Supported functionality/operating systems"> 81<colgroup> 82<col> 83<col> 84<col> 85<col> 86<col> 87<col> 88<col> 89</colgroup> 90<thead><tr> 91<th> 92 </th> 93<th> 94 <p> 95 AIX 96 </p> 97 </th> 98<th> 99 <p> 100 FreeBSD 101 </p> 102 </th> 103<th> 104 <p> 105 HP/UX 106 </p> 107 </th> 108<th> 109 <p> 110 Linux 111 </p> 112 </th> 113<th> 114 <p> 115 Solaris 116 </p> 117 </th> 118<th> 119 <p> 120 Windows 121 </p> 122 </th> 123</tr></thead> 124<tbody> 125<tr> 126<td> 127 <p> 128 pin thread 129 </p> 130 </td> 131<td> 132 <p> 133 + 134 </p> 135 </td> 136<td> 137 <p> 138 + 139 </p> 140 </td> 141<td> 142 <p> 143 + 144 </p> 145 </td> 146<td> 147 <p> 148 + 149 </p> 150 </td> 151<td> 152 <p> 153 + 154 </p> 155 </td> 156<td> 157 <p> 158 + 159 </p> 160 </td> 161</tr> 162<tr> 163<td> 164 <p> 165 logical CPUs/NUMA nodes 166 </p> 167 </td> 168<td> 169 <p> 170 + 171 </p> 172 </td> 173<td> 174 <p> 175 + 176 </p> 177 </td> 178<td> 179 <p> 180 + 181 </p> 182 </td> 183<td> 184 <p> 185 + 186 </p> 187 </td> 188<td> 189 <p> 190 + 191 </p> 192 </td> 193<td> 194 <p> 195 +<a href="#ftn.fiber.numa.f1" class="footnote" name="fiber.numa.f1"><sup class="footnote">[a]</sup></a> 196 </p> 197 </td> 198</tr> 199<tr> 200<td> 201 <p> 202 NUMA node distance 203 </p> 204 </td> 205<td> 206 <p> 207 - 208 </p> 209 </td> 210<td> 211 <p> 212 - 213 </p> 214 </td> 215<td> 216 <p> 217 - 218 </p> 219 </td> 220<td> 221 <p> 222 + 223 </p> 224 </td> 225<td> 226 <p> 227 - 228 </p> 229 </td> 230<td> 231 <p> 232 - 233 </p> 234 </td> 235</tr> 236<tr> 237<td> 238 <p> 239 tested on 240 </p> 241 </td> 242<td> 243 <p> 244 AIX 7.2 245 </p> 246 </td> 247<td> 248 <p> 249 FreeBSD 11 250 </p> 251 </td> 252<td> 253 <p> 254 - 255 </p> 256 </td> 257<td> 258 <p> 259 Arch Linux (4.10.13) 260 </p> 261 </td> 262<td> 263 <p> 264 OpenIndiana HIPSTER 265 </p> 266 </td> 267<td> 268 <p> 269 Windows 10 270 </p> 271 </td> 272</tr> 273</tbody> 274<tbody class="footnotes"><tr><td colspan="7"><div id="ftn.fiber.numa.f1" class="footnote"><p><a href="#fiber.numa.f1" class="para"><sup class="para">[a] </sup></a> 275 Windows organizes logical cpus in groups of 64; boost.fiber maps 276 {group-id,cpud-id} to a scalar equivalent to cpu ID of Linux (64 277 * group ID + cpu ID). 278 </p></div></td></tr></tbody> 279</table></div> 280</div> 281<br class="table-break"><p> 282 In order to keep the memory access local as possible, the NUMA topology must 283 be evaluated. 284 </p> 285<pre class="programlisting"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">node</span> <span class="special">></span> <span class="identifier">topo</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">topology</span><span class="special">();</span> 286<span class="keyword">for</span> <span class="special">(</span> <span class="keyword">auto</span> <span class="identifier">n</span> <span class="special">:</span> <span class="identifier">topo</span><span class="special">)</span> <span class="special">{</span> 287 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"node: "</span> <span class="special"><<</span> <span class="identifier">n</span><span class="special">.</span><span class="identifier">id</span> <span class="special"><<</span> <span class="string">" | "</span><span class="special">;</span> 288 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"cpus: "</span><span class="special">;</span> 289 <span class="keyword">for</span> <span class="special">(</span> <span class="keyword">auto</span> <span class="identifier">cpu_id</span> <span class="special">:</span> <span class="identifier">n</span><span class="special">.</span><span class="identifier">logical_cpus</span><span class="special">)</span> <span class="special">{</span> 290 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">cpu_id</span> <span class="special"><<</span> <span class="string">" "</span><span class="special">;</span> 291 <span class="special">}</span> 292 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"| distance: "</span><span class="special">;</span> 293 <span class="keyword">for</span> <span class="special">(</span> <span class="keyword">auto</span> <span class="identifier">d</span> <span class="special">:</span> <span class="identifier">n</span><span class="special">.</span><span class="identifier">distance</span><span class="special">)</span> <span class="special">{</span> 294 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">d</span> <span class="special"><<</span> <span class="string">" "</span><span class="special">;</span> 295 <span class="special">}</span> 296 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 297<span class="special">}</span> 298<span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"done"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 299 300<span class="identifier">output</span><span class="special">:</span> 301 <span class="identifier">node</span><span class="special">:</span> <span class="number">0</span> <span class="special">|</span> <span class="identifier">cpus</span><span class="special">:</span> <span class="number">0</span> <span class="number">1</span> <span class="number">2</span> <span class="number">3</span> <span class="number">4</span> <span class="number">5</span> <span class="number">6</span> <span class="number">7</span> <span class="number">16</span> <span class="number">17</span> <span class="number">18</span> <span class="number">19</span> <span class="number">20</span> <span class="number">21</span> <span class="number">22</span> <span class="number">23</span> <span class="special">|</span> <span class="identifier">distance</span><span class="special">:</span> <span class="number">10</span> <span class="number">21</span> 302 <span class="identifier">node</span><span class="special">:</span> <span class="number">1</span> <span class="special">|</span> <span class="identifier">cpus</span><span class="special">:</span> <span class="number">8</span> <span class="number">9</span> <span class="number">10</span> <span class="number">11</span> <span class="number">12</span> <span class="number">13</span> <span class="number">14</span> <span class="number">15</span> <span class="number">24</span> <span class="number">25</span> <span class="number">26</span> <span class="number">27</span> <span class="number">28</span> <span class="number">29</span> <span class="number">30</span> <span class="number">31</span> <span class="special">|</span> <span class="identifier">distance</span><span class="special">:</span> <span class="number">21</span> <span class="number">10</span> 303 <span class="identifier">done</span> 304</pre> 305<p> 306 The example shows that the systems consits out of 2 NUMA-nodes, to each NUMA-node 307 belong 16 logical cpus. The distance measures the costs to access the memory 308 of another NUMA-node. A NUMA-node has always a distance <code class="computeroutput"><span class="number">10</span></code> 309 to itself (lowest possible value).<br> The position in the array corresponds 310 with the NUMA-node ID. 311 </p> 312<p> 313 Some work-loads benefit from pinning threads to a logical cpus. For instance 314 scheduling algorithm <a class="link" href="numa.html#class_numa_work_stealing"><code class="computeroutput">numa::work_stealing</code></a> pins the thread 315 that runs the fiber scheduler to a logical cpu. This prevents the operating 316 system scheduler to move the thread to another logical cpu that might run other 317 fiber scheduler(s) or migrating the thread to a logical cpu part of another 318 NUMA-node. 319 </p> 320<pre class="programlisting"><span class="keyword">void</span> <span class="identifier">thread</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">cpu_id</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">node_id</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">node</span> <span class="special">></span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">topo</span><span class="special">)</span> <span class="special">{</span> 321 <span class="comment">// thread registers itself at work-stealing scheduler</span> 322 <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">use_scheduling_algorithm</span><span class="special"><</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">algo</span><span class="special">::</span><span class="identifier">work_stealing</span> <span class="special">>(</span> <span class="identifier">cpu_id</span><span class="special">,</span> <span class="identifier">node_id</span><span class="special">,</span> <span class="identifier">topo</span><span class="special">);</span> 323 <span class="special">...</span> 324<span class="special">}</span> 325 326<span class="comment">// evaluate the NUMA topology</span> 327<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">node</span> <span class="special">></span> <span class="identifier">topo</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">topology</span><span class="special">();</span> 328<span class="comment">// start-thread runs on NUMA-node `0`</span> 329<span class="keyword">auto</span> <span class="identifier">node</span> <span class="special">=</span> <span class="identifier">topo</span><span class="special">[</span><span class="number">0</span><span class="special">];</span> 330<span class="comment">// start-thread is pinnded to first cpu ID in the list of logical cpus of NUMA-node `0`</span> 331<span class="keyword">auto</span> <span class="identifier">start_cpu_id</span> <span class="special">=</span> <span class="special">*</span> <span class="identifier">node</span><span class="special">.</span><span class="identifier">logical_cpus</span><span class="special">.</span><span class="identifier">begin</span><span class="special">();</span> 332<span class="comment">// start worker-threads first</span> 333<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">thread</span> <span class="special">></span> <span class="identifier">threads</span><span class="special">;</span> 334<span class="keyword">for</span> <span class="special">(</span> <span class="keyword">auto</span> <span class="special">&</span> <span class="identifier">node</span> <span class="special">:</span> <span class="identifier">topo</span><span class="special">)</span> <span class="special">{</span> 335 <span class="keyword">for</span> <span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">cpu_id</span> <span class="special">:</span> <span class="identifier">node</span><span class="special">.</span><span class="identifier">logical_cpus</span><span class="special">)</span> <span class="special">{</span> 336 <span class="comment">// exclude start-thread</span> 337 <span class="keyword">if</span> <span class="special">(</span> <span class="identifier">start_cpu_id</span> <span class="special">!=</span> <span class="identifier">cpu_id</span><span class="special">)</span> <span class="special">{</span> 338 <span class="comment">// spawn thread</span> 339 <span class="identifier">threads</span><span class="special">.</span><span class="identifier">emplace_back</span><span class="special">(</span> <span class="identifier">thread</span><span class="special">,</span> <span class="identifier">cpu_id</span><span class="special">,</span> <span class="identifier">node</span><span class="special">.</span><span class="identifier">id</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">cref</span><span class="special">(</span> <span class="identifier">topo</span><span class="special">)</span> <span class="special">);</span> 340 <span class="special">}</span> 341 <span class="special">}</span> 342<span class="special">}</span> 343<span class="comment">// start-thread registers itself on work-stealing scheduler</span> 344<span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">use_scheduling_algorithm</span><span class="special"><</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">algo</span><span class="special">::</span><span class="identifier">work_stealing</span> <span class="special">>(</span> <span class="identifier">start_cpu_id</span><span class="special">,</span> <span class="identifier">node</span><span class="special">.</span><span class="identifier">id</span><span class="special">,</span> <span class="identifier">topo</span><span class="special">);</span> 345<span class="special">...</span> 346</pre> 347<p> 348 The example evaluates the NUMA topology with <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">topology</span><span class="special">()</span></code> 349 and spawns for each logical cpu a thread. Each spawned thread installs the 350 NUMA-aware work-stealing scheduler. The scheduler pins the thread to the logical 351 cpu that was specified at construction.<br> If the local queue of one thread 352 runs out of ready fibers, the thread tries to steal a ready fiber from another 353 thread running at logical cpu that belong to the same NUMA-node (local memory 354 access). If no fiber could be stolen, the thread tries to steal fibers from 355 logical cpus part of other NUMA-nodes (remote memory access). 356 </p> 357<h4> 358<a name="fiber.numa.h1"></a> 359 <span class="phrase"><a name="fiber.numa.synopsis"></a></span><a class="link" href="numa.html#fiber.numa.synopsis">Synopsis</a> 360 </h4> 361<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">fiber</span><span class="special">/</span><span class="identifier">numa</span><span class="special">/</span><span class="identifier">pin_thread</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 362<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">fiber</span><span class="special">/</span><span class="identifier">numa</span><span class="special">/</span><span class="identifier">topology</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 363 364<span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> 365<span class="keyword">namespace</span> <span class="identifier">fibers</span> <span class="special">{</span> 366<span class="keyword">namespace</span> <span class="identifier">numa</span> <span class="special">{</span> 367 368<span class="keyword">struct</span> <span class="identifier">node</span> <span class="special">{</span> 369 <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">id</span><span class="special">;</span> 370 <span class="identifier">std</span><span class="special">::</span><span class="identifier">set</span><span class="special"><</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="special">></span> <span class="identifier">logical_cpus</span><span class="special">;</span> 371 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="special">></span> <span class="identifier">distance</span><span class="special">;</span> 372<span class="special">};</span> 373<span class="keyword">bool</span> <span class="keyword">operator</span><span class="special"><(</span> <span class="identifier">node</span> <span class="keyword">const</span><span class="special">&,</span> <span class="identifier">node</span> <span class="keyword">const</span><span class="special">&)</span> <span class="keyword">noexcept</span><span class="special">;</span> 374 375<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">node</span> <span class="special">></span> <span class="identifier">topology</span><span class="special">();</span> 376 377<span class="keyword">void</span> <span class="identifier">pin_thread</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span><span class="special">);</span> 378<span class="keyword">void</span> <span class="identifier">pin_thread</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">thread</span><span class="special">::</span><span class="identifier">native_handle_type</span><span class="special">);</span> 379 380<span class="special">}}}</span> 381 382<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">fiber</span><span class="special">/</span><span class="identifier">numa</span><span class="special">/</span><span class="identifier">algo</span><span class="special">/</span><span class="identifier">work_stealing</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 383 384<span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> 385<span class="keyword">namespace</span> <span class="identifier">fibers</span> <span class="special">{</span> 386<span class="keyword">namespace</span> <span class="identifier">numa</span> <span class="special">{</span> 387<span class="keyword">namespace</span> <span class="identifier">algo</span> <span class="special">{</span> 388 389<span class="keyword">class</span> <span class="identifier">work_stealing</span><span class="special">;</span> 390 391<span class="special">}}}</span> 392</pre> 393<p> 394 </p> 395<h5> 396<a name="class_numa_node_bridgehead"></a> 397 <span class="phrase"><a name="class_numa_node"></a></span> 398 <a class="link" href="numa.html#class_numa_node">Class <code class="computeroutput">numa::node</code></a> 399</h5> 400<p> 401 </p> 402<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">fiber</span><span class="special">/</span><span class="identifier">numa</span><span class="special">/</span><span class="identifier">topology</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 403 404<span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> 405<span class="keyword">namespace</span> <span class="identifier">fibers</span> <span class="special">{</span> 406<span class="keyword">namespace</span> <span class="identifier">numa</span> <span class="special">{</span> 407 408<span class="keyword">struct</span> <span class="identifier">node</span> <span class="special">{</span> 409 <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">id</span><span class="special">;</span> 410 <span class="identifier">std</span><span class="special">::</span><span class="identifier">set</span><span class="special"><</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="special">></span> <span class="identifier">logical_cpus</span><span class="special">;</span> 411 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="special">></span> <span class="identifier">distance</span><span class="special">;</span> 412<span class="special">};</span> 413<span class="keyword">bool</span> <span class="keyword">operator</span><span class="special"><(</span> <span class="identifier">node</span> <span class="keyword">const</span><span class="special">&,</span> <span class="identifier">node</span> <span class="keyword">const</span><span class="special">&)</span> <span class="keyword">noexcept</span><span class="special">;</span> 414 415<span class="special">}}}</span> 416</pre> 417<p> 418 </p> 419<h5> 420<a name="numa_node_id_bridgehead"></a> 421 <span class="phrase"><a name="numa_node_id"></a></span> 422 <a class="link" href="numa.html#numa_node_id">Data member <code class="computeroutput">id</code></a> 423</h5> 424<p> 425 </p> 426<pre class="programlisting"><span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">id</span><span class="special">;</span> 427</pre> 428<div class="variablelist"> 429<p class="title"><b></b></p> 430<dl class="variablelist"> 431<dt><span class="term">Effects:</span></dt> 432<dd><p> 433 ID of the NUMA-node 434 </p></dd> 435</dl> 436</div> 437<p> 438 </p> 439<h5> 440<a name="numa_node_logical_cpus_bridgehead"></a> 441 <span class="phrase"><a name="numa_node_logical_cpus"></a></span> 442 <a class="link" href="numa.html#numa_node_logical_cpus">Data 443 member <code class="computeroutput">logical_cpus</code></a> 444</h5> 445<p> 446 </p> 447<pre class="programlisting"><span class="identifier">std</span><span class="special">::</span><span class="identifier">set</span><span class="special"><</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="special">></span> <span class="identifier">logical_cpus</span><span class="special">;</span> 448</pre> 449<div class="variablelist"> 450<p class="title"><b></b></p> 451<dl class="variablelist"> 452<dt><span class="term">Effects:</span></dt> 453<dd><p> 454 set of logical cpu IDs belonging to the NUMA-node 455 </p></dd> 456</dl> 457</div> 458<p> 459 </p> 460<h5> 461<a name="numa_node_distance_bridgehead"></a> 462 <span class="phrase"><a name="numa_node_distance"></a></span> 463 <a class="link" href="numa.html#numa_node_distance">Data member 464 <code class="computeroutput">distance</code></a> 465</h5> 466<p> 467 </p> 468<pre class="programlisting"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="special">></span> <span class="identifier">distance</span><span class="special">;</span> 469</pre> 470<div class="variablelist"> 471<p class="title"><b></b></p> 472<dl class="variablelist"> 473<dt><span class="term">Effects:</span></dt> 474<dd><p> 475 The distance between NUMA-nodes describe the cots of accessing the remote 476 memory. 477 </p></dd> 478<dt><span class="term">Note:</span></dt> 479<dd><p> 480 A NUMA-node has a distance of <code class="computeroutput"><span class="number">10</span></code> 481 to itself, remote NUMA-nodes have a distance > <code class="computeroutput"><span class="number">10</span></code>. 482 The index in the array corresponds to the ID <code class="computeroutput"><span class="identifier">id</span></code> 483 of the NUMA-node. At the moment only Linux returns the correct distances, 484 for all other operating systems remote NUMA-nodes get a default value 485 of <code class="computeroutput"><span class="number">20</span></code>. 486 </p></dd> 487</dl> 488</div> 489<p> 490 </p> 491<h5> 492<a name="numa_node_operator_less_bridgehead"></a> 493 <span class="phrase"><a name="numa_node_operator_less"></a></span> 494 <a class="link" href="numa.html#numa_node_operator_less">Member 495 function <code class="computeroutput">operator<</code>()</a> 496</h5> 497<p> 498 </p> 499<pre class="programlisting"><span class="keyword">bool</span> <span class="keyword">operator</span><span class="special"><(</span> <span class="identifier">node</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">lhs</span><span class="special">,</span> <span class="identifier">node</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">rhs</span><span class="special">)</span> <span class="keyword">const</span> <span class="keyword">noexcept</span><span class="special">;</span> 500</pre> 501<div class="variablelist"> 502<p class="title"><b></b></p> 503<dl class="variablelist"> 504<dt><span class="term">Returns:</span></dt> 505<dd><p> 506 <code class="computeroutput"><span class="keyword">true</span></code> if <code class="computeroutput"><span class="identifier">lhs</span> 507 <span class="special">!=</span> <span class="identifier">rhs</span></code> 508 is true and the implementation-defined total order of <code class="computeroutput"><span class="identifier">node</span><span class="special">::</span><span class="identifier">id</span></code> 509 values places <code class="computeroutput"><span class="identifier">lhs</span></code> before 510 <code class="computeroutput"><span class="identifier">rhs</span></code>, false otherwise. 511 </p></dd> 512<dt><span class="term">Throws:</span></dt> 513<dd><p> 514 Nothing. 515 </p></dd> 516</dl> 517</div> 518<p> 519 </p> 520<h5> 521<a name="numa_topology_bridgehead"></a> 522 <span class="phrase"><a name="numa_topology"></a></span> 523 <a class="link" href="numa.html#numa_topology">Non-member function <code class="computeroutput">numa::topology()</code></a> 524</h5> 525<p> 526 </p> 527<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">fiber</span><span class="special">/</span><span class="identifier">numa</span><span class="special">/</span><span class="identifier">topology</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 528 529<span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> 530<span class="keyword">namespace</span> <span class="identifier">fibers</span> <span class="special">{</span> 531<span class="keyword">namespace</span> <span class="identifier">numa</span> <span class="special">{</span> 532 533<span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">node</span> <span class="special">></span> <span class="identifier">topology</span><span class="special">();</span> 534 535<span class="special">}}}</span> 536</pre> 537<div class="variablelist"> 538<p class="title"><b></b></p> 539<dl class="variablelist"> 540<dt><span class="term">Effects:</span></dt> 541<dd><p> 542 Evaluates the NUMA topology. 543 </p></dd> 544<dt><span class="term">Returns:</span></dt> 545<dd><p> 546 a vector of NUMA-nodes describing the NUMA architecture of the system 547 (each element represents a NUMA-node). 548 </p></dd> 549<dt><span class="term">Throws:</span></dt> 550<dd><p> 551 <code class="computeroutput"><span class="identifier">system_error</span></code> 552 </p></dd> 553</dl> 554</div> 555<p> 556 </p> 557<h5> 558<a name="numa_pin_thread_bridgehead"></a> 559 <span class="phrase"><a name="numa_pin_thread"></a></span> 560 <a class="link" href="numa.html#numa_pin_thread">Non-member function 561 <code class="computeroutput">numa::pin_thread()</code></a> 562</h5> 563<p> 564 </p> 565<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">fiber</span><span class="special">/</span><span class="identifier">numa</span><span class="special">/</span><span class="identifier">pin_thread</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 566 567<span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> 568<span class="keyword">namespace</span> <span class="identifier">fibers</span> <span class="special">{</span> 569<span class="keyword">namespace</span> <span class="identifier">numa</span> <span class="special">{</span> 570 571<span class="keyword">void</span> <span class="identifier">pin_thread</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">cpu_id</span><span class="special">);</span> 572<span class="keyword">void</span> <span class="identifier">pin_thread</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">cpu_id</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">thread</span><span class="special">::</span><span class="identifier">native_handle_type</span> <span class="identifier">h</span><span class="special">);</span> 573 574<span class="special">}}}</span> 575</pre> 576<div class="variablelist"> 577<p class="title"><b></b></p> 578<dl class="variablelist"> 579<dt><span class="term">Effects:</span></dt> 580<dd><p> 581 First version pins <code class="computeroutput"><span class="keyword">this</span> <span class="identifier">thread</span></code> to the logical cpu with ID 582 <code class="computeroutput"><span class="identifier">cpu_id</span></code>, e.g. the operating 583 system scheduler will not migrate the thread to another logical cpu. 584 The second variant pins the thread with the native ID <code class="computeroutput"><span class="identifier">h</span></code> 585 to logical cpu with ID <code class="computeroutput"><span class="identifier">cpu_id</span></code>. 586 </p></dd> 587<dt><span class="term">Throws:</span></dt> 588<dd><p> 589 <code class="computeroutput"><span class="identifier">system_error</span></code> 590 </p></dd> 591</dl> 592</div> 593<p> 594 </p> 595<h5> 596<a name="class_numa_work_stealing_bridgehead"></a> 597 <span class="phrase"><a name="class_numa_work_stealing"></a></span> 598 <a class="link" href="numa.html#class_numa_work_stealing">Class 599 <code class="computeroutput">numa::work_stealing</code></a> 600</h5> 601<p> 602 </p> 603<p> 604 This class implements <a class="link" href="scheduling.html#class_algorithm"><code class="computeroutput">algorithm</code></a>; the thread running this scheduler 605 is pinned to the given logical cpu. If the local ready-queue runs out of ready 606 fibers, ready fibers are stolen from other schedulers that run on logical cpus 607 that belong to the same NUMA-node (local memory access).<br> If no ready 608 fibers can be stolen from the local NUMA-node, the algorithm selects schedulers 609 running on other NUMA-nodes (remote memory access).<br> The victim scheduler 610 (from which a ready fiber is stolen) is selected at random. 611 </p> 612<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">fiber</span><span class="special">/</span><span class="identifier">numa</span><span class="special">/</span><span class="identifier">algo</span><span class="special">/</span><span class="identifier">work_stealing</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 613 614<span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> 615<span class="keyword">namespace</span> <span class="identifier">fibers</span> <span class="special">{</span> 616<span class="keyword">namespace</span> <span class="identifier">numa</span> <span class="special">{</span> 617<span class="keyword">namespace</span> <span class="identifier">algo</span> <span class="special">{</span> 618 619<span class="keyword">class</span> <span class="identifier">work_stealing</span> <span class="special">:</span> <span class="keyword">public</span> <span class="identifier">algorithm</span> <span class="special">{</span> 620<span class="keyword">public</span><span class="special">:</span> 621 <span class="identifier">work_stealing</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">cpu_id</span><span class="special">,</span> 622 <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">node_id</span><span class="special">,</span> 623 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">node</span> <span class="special">></span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">topo</span><span class="special">,</span> 624 <span class="keyword">bool</span> <span class="identifier">suspend</span> <span class="special">=</span> <span class="keyword">false</span><span class="special">);</span> 625 626 <span class="identifier">work_stealing</span><span class="special">(</span> <span class="identifier">work_stealing</span> <span class="keyword">const</span><span class="special">&)</span> <span class="special">=</span> <span class="keyword">delete</span><span class="special">;</span> 627 <span class="identifier">work_stealing</span><span class="special">(</span> <span class="identifier">work_stealing</span> <span class="special">&&)</span> <span class="special">=</span> <span class="keyword">delete</span><span class="special">;</span> 628 629 <span class="identifier">work_stealing</span> <span class="special">&</span> <span class="keyword">operator</span><span class="special">=(</span> <span class="identifier">work_stealing</span> <span class="keyword">const</span><span class="special">&)</span> <span class="special">=</span> <span class="keyword">delete</span><span class="special">;</span> 630 <span class="identifier">work_stealing</span> <span class="special">&</span> <span class="keyword">operator</span><span class="special">=(</span> <span class="identifier">work_stealing</span> <span class="special">&&)</span> <span class="special">=</span> <span class="keyword">delete</span><span class="special">;</span> 631 632 <span class="keyword">virtual</span> <span class="keyword">void</span> <span class="identifier">awakened</span><span class="special">(</span> <span class="identifier">context</span> <span class="special">*)</span> <span class="keyword">noexcept</span><span class="special">;</span> 633 634 <span class="keyword">virtual</span> <span class="identifier">context</span> <span class="special">*</span> <span class="identifier">pick_next</span><span class="special">()</span> <span class="keyword">noexcept</span><span class="special">;</span> 635 636 <span class="keyword">virtual</span> <span class="keyword">bool</span> <span class="identifier">has_ready_fibers</span><span class="special">()</span> <span class="keyword">const</span> <span class="keyword">noexcept</span><span class="special">;</span> 637 638 <span class="keyword">virtual</span> <span class="keyword">void</span> <span class="identifier">suspend_until</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">chrono</span><span class="special">::</span><span class="identifier">steady_clock</span><span class="special">::</span><span class="identifier">time_point</span> <span class="keyword">const</span><span class="special">&)</span> <span class="keyword">noexcept</span><span class="special">;</span> 639 640 <span class="keyword">virtual</span> <span class="keyword">void</span> <span class="identifier">notify</span><span class="special">()</span> <span class="keyword">noexcept</span><span class="special">;</span> 641<span class="special">};</span> 642 643<span class="special">}}}}</span> 644</pre> 645<h4> 646<a name="fiber.numa.h2"></a> 647 <span class="phrase"><a name="fiber.numa.constructor"></a></span><a class="link" href="numa.html#fiber.numa.constructor">Constructor</a> 648 </h4> 649<pre class="programlisting"><span class="identifier">work_stealing</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">cpu_id</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">uint32_t</span> <span class="identifier">node_id</span><span class="special">,</span> 650 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">fibers</span><span class="special">::</span><span class="identifier">numa</span><span class="special">::</span><span class="identifier">node</span> <span class="special">></span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">topo</span><span class="special">,</span> 651 <span class="keyword">bool</span> <span class="identifier">suspend</span> <span class="special">=</span> <span class="keyword">false</span><span class="special">);</span> 652</pre> 653<div class="variablelist"> 654<p class="title"><b></b></p> 655<dl class="variablelist"> 656<dt><span class="term">Effects:</span></dt> 657<dd><p> 658 Constructs work-stealing scheduling algorithm. The thread is pinned to 659 logical cpu with ID <code class="computeroutput"><span class="identifier">cpu_id</span></code>. 660 If local ready-queue runs out of ready fibers, ready fibers are stolen 661 from other schedulers using <code class="computeroutput"><span class="identifier">topology</span></code> 662 (represents the NUMA-topology of the system). 663 </p></dd> 664<dt><span class="term">Throws:</span></dt> 665<dd><p> 666 <code class="computeroutput"><span class="identifier">system_error</span></code> 667 </p></dd> 668<dt><span class="term">Note:</span></dt> 669<dd><p> 670 If <code class="computeroutput"><span class="identifier">suspend</span></code> is set to 671 <code class="computeroutput"><span class="keyword">true</span></code>, then the scheduler 672 suspends if no ready fiber could be stolen. The scheduler will by woken 673 up if a sleeping fiber times out or it was notified from remote (other 674 thread or fiber scheduler). 675 </p></dd> 676</dl> 677</div> 678<p> 679 </p> 680<h5> 681<a name="numa_work_stealing_awakened_bridgehead"></a> 682 <span class="phrase"><a name="numa_work_stealing_awakened"></a></span> 683 <a class="link" href="numa.html#numa_work_stealing_awakened">Member 684 function <code class="computeroutput">awakened</code>()</a> 685</h5> 686<p> 687 </p> 688<pre class="programlisting"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="identifier">awakened</span><span class="special">(</span> <span class="identifier">context</span> <span class="special">*</span> <span class="identifier">f</span><span class="special">)</span> <span class="keyword">noexcept</span><span class="special">;</span> 689</pre> 690<div class="variablelist"> 691<p class="title"><b></b></p> 692<dl class="variablelist"> 693<dt><span class="term">Effects:</span></dt> 694<dd><p> 695 Enqueues fiber <code class="computeroutput"><span class="identifier">f</span></code> onto 696 the shared ready queue. 697 </p></dd> 698<dt><span class="term">Throws:</span></dt> 699<dd><p> 700 Nothing. 701 </p></dd> 702</dl> 703</div> 704<p> 705 </p> 706<h5> 707<a name="numa_work_stealing_pick_next_bridgehead"></a> 708 <span class="phrase"><a name="numa_work_stealing_pick_next"></a></span> 709 <a class="link" href="numa.html#numa_work_stealing_pick_next">Member 710 function <code class="computeroutput">pick_next</code>()</a> 711</h5> 712<p> 713 </p> 714<pre class="programlisting"><span class="keyword">virtual</span> <span class="identifier">context</span> <span class="special">*</span> <span class="identifier">pick_next</span><span class="special">()</span> <span class="keyword">noexcept</span><span class="special">;</span> 715</pre> 716<div class="variablelist"> 717<p class="title"><b></b></p> 718<dl class="variablelist"> 719<dt><span class="term">Returns:</span></dt> 720<dd><p> 721 the fiber at the head of the ready queue, or <code class="computeroutput"><span class="keyword">nullptr</span></code> 722 if the queue is empty. 723 </p></dd> 724<dt><span class="term">Throws:</span></dt> 725<dd><p> 726 Nothing. 727 </p></dd> 728<dt><span class="term">Note:</span></dt> 729<dd><p> 730 Placing ready fibers onto the tail of the sahred queue, and returning 731 them from the head of that queue, shares the thread between ready fibers 732 in round-robin fashion. 733 </p></dd> 734</dl> 735</div> 736<p> 737 </p> 738<h5> 739<a name="numa_work_stealing_has_ready_fibers_bridgehead"></a> 740 <span class="phrase"><a name="numa_work_stealing_has_ready_fibers"></a></span> 741 <a class="link" href="numa.html#numa_work_stealing_has_ready_fibers">Member 742 function <code class="computeroutput">has_ready_fibers</code>()</a> 743</h5> 744<p> 745 </p> 746<pre class="programlisting"><span class="keyword">virtual</span> <span class="keyword">bool</span> <span class="identifier">has_ready_fibers</span><span class="special">()</span> <span class="keyword">const</span> <span class="keyword">noexcept</span><span class="special">;</span> 747</pre> 748<div class="variablelist"> 749<p class="title"><b></b></p> 750<dl class="variablelist"> 751<dt><span class="term">Returns:</span></dt> 752<dd><p> 753 <code class="computeroutput"><span class="keyword">true</span></code> if scheduler has fibers 754 ready to run. 755 </p></dd> 756<dt><span class="term">Throws:</span></dt> 757<dd><p> 758 Nothing. 759 </p></dd> 760</dl> 761</div> 762<p> 763 </p> 764<h5> 765<a name="numa_work_stealing_suspend_until_bridgehead"></a> 766 <span class="phrase"><a name="numa_work_stealing_suspend_until"></a></span> 767 <a class="link" href="numa.html#numa_work_stealing_suspend_until">Member 768 function <code class="computeroutput">suspend_until</code>()</a> 769</h5> 770<p> 771 </p> 772<pre class="programlisting"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="identifier">suspend_until</span><span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">chrono</span><span class="special">::</span><span class="identifier">steady_clock</span><span class="special">::</span><span class="identifier">time_point</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">abs_time</span><span class="special">)</span> <span class="keyword">noexcept</span><span class="special">;</span> 773</pre> 774<div class="variablelist"> 775<p class="title"><b></b></p> 776<dl class="variablelist"> 777<dt><span class="term">Effects:</span></dt> 778<dd><p> 779 Informs <code class="computeroutput"><span class="identifier">work_stealing</span></code> 780 that no ready fiber will be available until time-point <code class="computeroutput"><span class="identifier">abs_time</span></code>. This implementation blocks 781 in <a href="http://en.cppreference.com/w/cpp/thread/condition_variable/wait_until" target="_top"><code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">condition_variable</span><span class="special">::</span><span class="identifier">wait_until</span><span class="special">()</span></code></a>. 782 </p></dd> 783<dt><span class="term">Throws:</span></dt> 784<dd><p> 785 Nothing. 786 </p></dd> 787</dl> 788</div> 789<p> 790 </p> 791<h5> 792<a name="numa_work_stealing_notify_bridgehead"></a> 793 <span class="phrase"><a name="numa_work_stealing_notify"></a></span> 794 <a class="link" href="numa.html#numa_work_stealing_notify">Member 795 function <code class="computeroutput">notify</code>()</a> 796</h5> 797<p> 798 </p> 799<pre class="programlisting"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="identifier">notify</span><span class="special">()</span> <span class="keyword">noexcept</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> 800</pre> 801<div class="variablelist"> 802<p class="title"><b></b></p> 803<dl class="variablelist"> 804<dt><span class="term">Effects:</span></dt> 805<dd><p> 806 Wake up a pending call to <a class="link" href="scheduling.html#work_stealing_suspend_until"><code class="computeroutput">work_stealing::suspend_until()</code></a>, 807 some fibers might be ready. This implementation wakes <code class="computeroutput"><span class="identifier">suspend_until</span><span class="special">()</span></code> via <a href="http://en.cppreference.com/w/cpp/thread/condition_variable/notify_all" target="_top"><code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">condition_variable</span><span class="special">::</span><span class="identifier">notify_all</span><span class="special">()</span></code></a>. 808 </p></dd> 809<dt><span class="term">Throws:</span></dt> 810<dd><p> 811 Nothing. 812 </p></dd> 813</dl> 814</div> 815<div class="footnotes"> 816<br><hr style="width:100; text-align:left;margin-left: 0"> 817<div id="ftn.fiber.numa.f0" class="footnote"><p><a href="#fiber.numa.f0" class="para"><sup class="para">[8] </sup></a> 818 On x86 the interconnection is implemented by Intel's Quick Path Interconnect 819 (QPI) and AMD's HyperTransport. 820 </p></div> 821</div> 822</div> 823<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 824<td align="left"></td> 825<td align="right"><div class="copyright-footer">Copyright © 2013 Oliver Kowalke<p> 826 Distributed under the Boost Software License, Version 1.0. (See accompanying 827 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 828 </p> 829</div></td> 830</tr></table> 831<hr> 832<div class="spirit-nav"> 833<a accesskey="p" href="speculation.html"><img src="../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../index.html"><img src="../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="gpu_computing.html"><img src="../../../../../doc/src/images/next.png" alt="Next"></a> 834</div> 835</body> 836</html> 837
