1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 2<html> 3<head> 4<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 5<title>Thread coordination using Boost.Atomic</title> 6<link rel="stylesheet" href="../../../doc/src/boostbook.css" type="text/css"> 7<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 8<link rel="home" href="../index.html" title="The Boost C++ Libraries BoostBook Documentation Subset"> 9<link rel="up" href="../atomic.html" title="Chapter 6. Boost.Atomic"> 10<link rel="prev" href="../atomic.html" title="Chapter 6. Boost.Atomic"> 11<link rel="next" href="interface.html" title="Programming interfaces"> 12</head> 13<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 14<table cellpadding="2" width="100%"><tr> 15<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../boost.png"></td> 16<td align="center"><a href="../../../index.html">Home</a></td> 17<td align="center"><a href="../../../libs/libraries.htm">Libraries</a></td> 18<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 19<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 20<td align="center"><a href="../../../more/index.htm">More</a></td> 21</tr></table> 22<hr> 23<div class="spirit-nav"> 24<a accesskey="p" href="../atomic.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../atomic.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="interface.html"><img src="../../../doc/src/images/next.png" alt="Next"></a> 25</div> 26<div class="section"> 27<div class="titlepage"><div><div><h2 class="title" style="clear: both"> 28<a name="atomic.thread_coordination"></a><a class="link" href="thread_coordination.html" title="Thread coordination using Boost.Atomic">Thread coordination using Boost.Atomic</a> 29</h2></div></div></div> 30<div class="toc"><dl class="toc"> 31<dt><span class="section"><a href="thread_coordination.html#atomic.thread_coordination.mutex">Enforcing <span class="emphasis"><em>happens-before</em></span> 32 through mutual exclusion</a></span></dt> 33<dt><span class="section"><a href="thread_coordination.html#atomic.thread_coordination.release_acquire"><span class="emphasis"><em>happens-before</em></span> 34 through <code class="literal">release</code> and <code class="literal">acquire</code></a></span></dt> 35<dt><span class="section"><a href="thread_coordination.html#atomic.thread_coordination.fences">Fences</a></span></dt> 36<dt><span class="section"><a href="thread_coordination.html#atomic.thread_coordination.release_consume"><span class="emphasis"><em>happens-before</em></span> 37 through <code class="literal">release</code> and <code class="literal">consume</code></a></span></dt> 38<dt><span class="section"><a href="thread_coordination.html#atomic.thread_coordination.seq_cst">Sequential consistency</a></span></dt> 39</dl></div> 40<p> 41 The most common use of <span class="bold"><strong>Boost.Atomic</strong></span> is to 42 realize custom thread synchronization protocols: The goal is to coordinate 43 accesses of threads to shared variables in order to avoid "conflicts". 44 The programmer must be aware of the fact that compilers, CPUs and the cache 45 hierarchies may generally reorder memory references at will. As a consequence 46 a program such as: 47 </p> 48<pre class="programlisting"><span class="keyword">int</span> <span class="identifier">x</span> <span class="special">=</span> <span class="number">0</span><span class="special">,</span> <span class="keyword">int</span> <span class="identifier">y</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> 49 50<span class="identifier">thread1</span><span class="special">:</span> 51 <span class="identifier">x</span> <span class="special">=</span> <span class="number">1</span><span class="special">;</span> 52 <span class="identifier">y</span> <span class="special">=</span> <span class="number">1</span><span class="special">;</span> 53 54<span class="identifier">thread2</span><span class="special">:</span> 55 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">y</span> <span class="special">==</span> <span class="number">1</span><span class="special">)</span> <span class="special">{</span> 56 <span class="identifier">assert</span><span class="special">(</span><span class="identifier">x</span> <span class="special">==</span> <span class="number">1</span><span class="special">);</span> 57 <span class="special">}</span> 58</pre> 59<p> 60 might indeed fail as there is no guarantee that the read of <code class="computeroutput"><span class="identifier">x</span></code> 61 by thread2 "sees" the write by thread1. 62 </p> 63<p> 64 <span class="bold"><strong>Boost.Atomic</strong></span> uses a synchronisation concept 65 based on the <span class="emphasis"><em>happens-before</em></span> relation to describe the guarantees 66 under which situations such as the above one cannot occur. 67 </p> 68<p> 69 The remainder of this section will discuss <span class="emphasis"><em>happens-before</em></span> 70 in a "hands-on" way instead of giving a fully formalized definition. 71 The reader is encouraged to additionally have a look at the discussion of the 72 correctness of a few of the <a class="link" href="usage_examples.html" title="Usage examples">examples</a> 73 afterwards. 74 </p> 75<div class="section"> 76<div class="titlepage"><div><div><h3 class="title"> 77<a name="atomic.thread_coordination.mutex"></a><a class="link" href="thread_coordination.html#atomic.thread_coordination.mutex" title="Enforcing happens-before through mutual exclusion">Enforcing <span class="emphasis"><em>happens-before</em></span> 78 through mutual exclusion</a> 79</h3></div></div></div> 80<p> 81 As an introductory example to understand how arguing using <span class="emphasis"><em>happens-before</em></span> 82 works, consider two threads synchronizing using a common mutex: 83 </p> 84<pre class="programlisting"><span class="identifier">mutex</span> <span class="identifier">m</span><span class="special">;</span> 85 86<span class="identifier">thread1</span><span class="special">:</span> 87 <span class="identifier">m</span><span class="special">.</span><span class="identifier">lock</span><span class="special">();</span> 88 <span class="special">...</span> <span class="comment">/* A */</span> 89 <span class="identifier">m</span><span class="special">.</span><span class="identifier">unlock</span><span class="special">();</span> 90 91<span class="identifier">thread2</span><span class="special">:</span> 92 <span class="identifier">m</span><span class="special">.</span><span class="identifier">lock</span><span class="special">();</span> 93 <span class="special">...</span> <span class="comment">/* B */</span> 94 <span class="identifier">m</span><span class="special">.</span><span class="identifier">unlock</span><span class="special">();</span> 95</pre> 96<p> 97 The "lockset-based intuition" would be to argue that A and B cannot 98 be executed concurrently as the code paths require a common lock to be held. 99 </p> 100<p> 101 One can however also arrive at the same conclusion using <span class="emphasis"><em>happens-before</em></span>: 102 Either thread1 or thread2 will succeed first at <code class="literal">m.lock()</code>. 103 If this is be thread1, then as a consequence, thread2 cannot succeed at 104 <code class="literal">m.lock()</code> before thread1 has executed <code class="literal">m.unlock()</code>, 105 consequently A <span class="emphasis"><em>happens-before</em></span> B in this case. By symmetry, 106 if thread2 succeeds at <code class="literal">m.lock()</code> first, we can conclude 107 B <span class="emphasis"><em>happens-before</em></span> A. 108 </p> 109<p> 110 Since this already exhausts all options, we can conclude that either A <span class="emphasis"><em>happens-before</em></span> 111 B or B <span class="emphasis"><em>happens-before</em></span> A must always hold. Obviously 112 cannot state <span class="emphasis"><em>which</em></span> of the two relationships holds, but 113 either one is sufficient to conclude that A and B cannot conflict. 114 </p> 115<p> 116 Compare the <a class="link" href="usage_examples.html#boost_atomic.usage_examples.example_spinlock" title="Spinlock">spinlock</a> 117 implementation to see how the mutual exclusion concept can be mapped to 118 <span class="bold"><strong>Boost.Atomic</strong></span>. 119 </p> 120</div> 121<div class="section"> 122<div class="titlepage"><div><div><h3 class="title"> 123<a name="atomic.thread_coordination.release_acquire"></a><a class="link" href="thread_coordination.html#atomic.thread_coordination.release_acquire" title="happens-before through release and acquire"><span class="emphasis"><em>happens-before</em></span> 124 through <code class="literal">release</code> and <code class="literal">acquire</code></a> 125</h3></div></div></div> 126<p> 127 The most basic pattern for coordinating threads via <span class="bold"><strong>Boost.Atomic</strong></span> 128 uses <code class="literal">release</code> and <code class="literal">acquire</code> on an atomic 129 variable for coordination: If ... 130 </p> 131<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 132<li class="listitem"> 133 ... thread1 performs an operation A, 134 </li> 135<li class="listitem"> 136 ... thread1 subsequently writes (or atomically modifies) an atomic variable 137 with <code class="literal">release</code> semantic, 138 </li> 139<li class="listitem"> 140 ... thread2 reads (or atomically reads-and-modifies) the value this value 141 from the same atomic variable with <code class="literal">acquire</code> semantic 142 and 143 </li> 144<li class="listitem"> 145 ... thread2 subsequently performs an operation B, 146 </li> 147</ul></div> 148<p> 149 ... then A <span class="emphasis"><em>happens-before</em></span> B. 150 </p> 151<p> 152 Consider the following example 153 </p> 154<pre class="programlisting"><span class="identifier">atomic</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">a</span><span class="special">(</span><span class="number">0</span><span class="special">);</span> 155 156<span class="identifier">thread1</span><span class="special">:</span> 157 <span class="special">...</span> <span class="comment">/* A */</span> 158 <span class="identifier">a</span><span class="special">.</span><span class="identifier">fetch_add</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="identifier">memory_order_release</span><span class="special">);</span> 159 160<span class="identifier">thread2</span><span class="special">:</span> 161 <span class="keyword">int</span> <span class="identifier">tmp</span> <span class="special">=</span> <span class="identifier">a</span><span class="special">.</span><span class="identifier">load</span><span class="special">(</span><span class="identifier">memory_order_acquire</span><span class="special">);</span> 162 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">tmp</span> <span class="special">==</span> <span class="number">1</span><span class="special">)</span> <span class="special">{</span> 163 <span class="special">...</span> <span class="comment">/* B */</span> 164 <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span> 165 <span class="special">...</span> <span class="comment">/* C */</span> 166 <span class="special">}</span> 167</pre> 168<p> 169 In this example, two avenues for execution are possible: 170 </p> 171<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 172<li class="listitem"> 173 The <code class="literal">store</code> operation by thread1 precedes the <code class="literal">load</code> 174 by thread2: In this case thread2 will execute B and "A <span class="emphasis"><em>happens-before</em></span> 175 B" holds as all of the criteria above are satisfied. 176 </li> 177<li class="listitem"> 178 The <code class="literal">load</code> operation by thread2 precedes the <code class="literal">store</code> 179 by thread1: In this case, thread2 will execute C, but "A <span class="emphasis"><em>happens-before</em></span> 180 C" does <span class="emphasis"><em>not</em></span> hold: thread2 does not read the 181 value written by thread1 through <code class="literal">a</code>. 182 </li> 183</ul></div> 184<p> 185 Therefore, A and B cannot conflict, but A and C <span class="emphasis"><em>can</em></span> 186 conflict. 187 </p> 188</div> 189<div class="section"> 190<div class="titlepage"><div><div><h3 class="title"> 191<a name="atomic.thread_coordination.fences"></a><a class="link" href="thread_coordination.html#atomic.thread_coordination.fences" title="Fences">Fences</a> 192</h3></div></div></div> 193<p> 194 Ordering constraints are generally specified together with an access to an 195 atomic variable. It is however also possible to issue "fence" operations 196 in isolation, in this case the fence operates in conjunction with preceding 197 (for <code class="computeroutput"><span class="identifier">acquire</span></code>, <code class="computeroutput"><span class="identifier">consume</span></code> or <code class="computeroutput"><span class="identifier">seq_cst</span></code> 198 operations) or succeeding (for <code class="computeroutput"><span class="identifier">release</span></code> 199 or <code class="computeroutput"><span class="identifier">seq_cst</span></code>) atomic operations. 200 </p> 201<p> 202 The example from the previous section could also be written in the following 203 way: 204 </p> 205<pre class="programlisting"><span class="identifier">atomic</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">a</span><span class="special">(</span><span class="number">0</span><span class="special">);</span> 206 207<span class="identifier">thread1</span><span class="special">:</span> 208 <span class="special">...</span> <span class="comment">/* A */</span> 209 <span class="identifier">atomic_thread_fence</span><span class="special">(</span><span class="identifier">memory_order_release</span><span class="special">);</span> 210 <span class="identifier">a</span><span class="special">.</span><span class="identifier">fetch_add</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="identifier">memory_order_relaxed</span><span class="special">);</span> 211 212<span class="identifier">thread2</span><span class="special">:</span> 213 <span class="keyword">int</span> <span class="identifier">tmp</span> <span class="special">=</span> <span class="identifier">a</span><span class="special">.</span><span class="identifier">load</span><span class="special">(</span><span class="identifier">memory_order_relaxed</span><span class="special">);</span> 214 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">tmp</span> <span class="special">==</span> <span class="number">1</span><span class="special">)</span> <span class="special">{</span> 215 <span class="identifier">atomic_thread_fence</span><span class="special">(</span><span class="identifier">memory_order_acquire</span><span class="special">);</span> 216 <span class="special">...</span> <span class="comment">/* B */</span> 217 <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span> 218 <span class="special">...</span> <span class="comment">/* C */</span> 219 <span class="special">}</span> 220</pre> 221<p> 222 This provides the same ordering guarantees as previously, but elides a (possibly 223 expensive) memory ordering operation in the case C is executed. 224 </p> 225<div class="note"><table border="0" summary="Note"> 226<tr> 227<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../doc/src/images/note.png"></td> 228<th align="left">Note</th> 229</tr> 230<tr><td align="left" valign="top"><p> 231 Atomic fences are only indended to constraint ordering of regular and atomic 232 loads and stores for the purpose of thread synchronization. <code class="computeroutput"><span class="identifier">atomic_thread_fence</span></code> is not intended to 233 be used to order some architecture-specific memory accesses, such as non-temporal 234 loads and stores on x86 or write combining memory accesses. Use specialized 235 instructions for these purposes. 236 </p></td></tr> 237</table></div> 238</div> 239<div class="section"> 240<div class="titlepage"><div><div><h3 class="title"> 241<a name="atomic.thread_coordination.release_consume"></a><a class="link" href="thread_coordination.html#atomic.thread_coordination.release_consume" title="happens-before through release and consume"><span class="emphasis"><em>happens-before</em></span> 242 through <code class="literal">release</code> and <code class="literal">consume</code></a> 243</h3></div></div></div> 244<p> 245 The second pattern for coordinating threads via <span class="bold"><strong>Boost.Atomic</strong></span> 246 uses <code class="literal">release</code> and <code class="literal">consume</code> on an atomic 247 variable for coordination: If ... 248 </p> 249<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 250<li class="listitem"> 251 ... thread1 performs an operation A, 252 </li> 253<li class="listitem"> 254 ... thread1 subsequently writes (or atomically modifies) an atomic variable 255 with <code class="literal">release</code> semantic, 256 </li> 257<li class="listitem"> 258 ... thread2 reads (or atomically reads-and-modifies) the value this value 259 from the same atomic variable with <code class="literal">consume</code> semantic 260 and 261 </li> 262<li class="listitem"> 263 ... thread2 subsequently performs an operation B that is <span class="emphasis"><em>computationally 264 dependent on the value of the atomic variable</em></span>, 265 </li> 266</ul></div> 267<p> 268 ... then A <span class="emphasis"><em>happens-before</em></span> B. 269 </p> 270<p> 271 Consider the following example 272 </p> 273<pre class="programlisting"><span class="identifier">atomic</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">a</span><span class="special">(</span><span class="number">0</span><span class="special">);</span> 274<span class="identifier">complex_data_structure</span> <span class="identifier">data</span><span class="special">[</span><span class="number">2</span><span class="special">];</span> 275 276<span class="identifier">thread1</span><span class="special">:</span> 277 <span class="identifier">data</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="special">...;</span> <span class="comment">/* A */</span> 278 <span class="identifier">a</span><span class="special">.</span><span class="identifier">store</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="identifier">memory_order_release</span><span class="special">);</span> 279 280<span class="identifier">thread2</span><span class="special">:</span> 281 <span class="keyword">int</span> <span class="identifier">index</span> <span class="special">=</span> <span class="identifier">a</span><span class="special">.</span><span class="identifier">load</span><span class="special">(</span><span class="identifier">memory_order_consume</span><span class="special">);</span> 282 <span class="identifier">complex_data_structure</span> <span class="identifier">tmp</span> <span class="special">=</span> <span class="identifier">data</span><span class="special">[</span><span class="identifier">index</span><span class="special">];</span> <span class="comment">/* B */</span> 283</pre> 284<p> 285 In this example, two avenues for execution are possible: 286 </p> 287<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 288<li class="listitem"> 289 The <code class="literal">store</code> operation by thread1 precedes the <code class="literal">load</code> 290 by thread2: In this case thread2 will read <code class="literal">data[1]</code> 291 and "A <span class="emphasis"><em>happens-before</em></span> B" holds as all 292 of the criteria above are satisfied. 293 </li> 294<li class="listitem"> 295 The <code class="literal">load</code> operation by thread2 precedes the <code class="literal">store</code> 296 by thread1: In this case thread2 will read <code class="literal">data[0]</code> 297 and "A <span class="emphasis"><em>happens-before</em></span> B" does <span class="emphasis"><em>not</em></span> 298 hold: thread2 does not read the value written by thread1 through <code class="literal">a</code>. 299 </li> 300</ul></div> 301<p> 302 Here, the <span class="emphasis"><em>happens-before</em></span> relationship helps ensure that 303 any accesses (presumable writes) to <code class="literal">data[1]</code> by thread1 304 happen before before the accesses (presumably reads) to <code class="literal">data[1]</code> 305 by thread2: Lacking this relationship, thread2 might see stale/inconsistent 306 data. 307 </p> 308<p> 309 Note that in this example, the fact that operation B is computationally dependent 310 on the atomic variable, therefore the following program would be erroneous: 311 </p> 312<pre class="programlisting"><span class="identifier">atomic</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">a</span><span class="special">(</span><span class="number">0</span><span class="special">);</span> 313<span class="identifier">complex_data_structure</span> <span class="identifier">data</span><span class="special">[</span><span class="number">2</span><span class="special">];</span> 314 315<span class="identifier">thread1</span><span class="special">:</span> 316 <span class="identifier">data</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="special">...;</span> <span class="comment">/* A */</span> 317 <span class="identifier">a</span><span class="special">.</span><span class="identifier">store</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="identifier">memory_order_release</span><span class="special">);</span> 318 319<span class="identifier">thread2</span><span class="special">:</span> 320 <span class="keyword">int</span> <span class="identifier">index</span> <span class="special">=</span> <span class="identifier">a</span><span class="special">.</span><span class="identifier">load</span><span class="special">(</span><span class="identifier">memory_order_consume</span><span class="special">);</span> 321 <span class="identifier">complex_data_structure</span> <span class="identifier">tmp</span><span class="special">;</span> 322 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">index</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> 323 <span class="identifier">tmp</span> <span class="special">=</span> <span class="identifier">data</span><span class="special">[</span><span class="number">0</span><span class="special">];</span> 324 <span class="keyword">else</span> 325 <span class="identifier">tmp</span> <span class="special">=</span> <span class="identifier">data</span><span class="special">[</span><span class="number">1</span><span class="special">];</span> 326</pre> 327<p> 328 <code class="literal">consume</code> is most commonly (and most safely! see <a class="link" href="limitations.html" title="Limitations">limitations</a>) 329 used with pointers, compare for example the <a class="link" href="usage_examples.html#boost_atomic.usage_examples.singleton" title="Singleton with double-checked locking pattern">singleton 330 with double-checked locking</a>. 331 </p> 332</div> 333<div class="section"> 334<div class="titlepage"><div><div><h3 class="title"> 335<a name="atomic.thread_coordination.seq_cst"></a><a class="link" href="thread_coordination.html#atomic.thread_coordination.seq_cst" title="Sequential consistency">Sequential consistency</a> 336</h3></div></div></div> 337<p> 338 The third pattern for coordinating threads via <span class="bold"><strong>Boost.Atomic</strong></span> 339 uses <code class="literal">seq_cst</code> for coordination: If ... 340 </p> 341<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 342<li class="listitem"> 343 ... thread1 performs an operation A, 344 </li> 345<li class="listitem"> 346 ... thread1 subsequently performs any operation with <code class="literal">seq_cst</code>, 347 </li> 348<li class="listitem"> 349 ... thread1 subsequently performs an operation B, 350 </li> 351<li class="listitem"> 352 ... thread2 performs an operation C, 353 </li> 354<li class="listitem"> 355 ... thread2 subsequently performs any operation with <code class="literal">seq_cst</code>, 356 </li> 357<li class="listitem"> 358 ... thread2 subsequently performs an operation D, 359 </li> 360</ul></div> 361<p> 362 then either "A <span class="emphasis"><em>happens-before</em></span> D" or "C 363 <span class="emphasis"><em>happens-before</em></span> B" holds. 364 </p> 365<p> 366 In this case it does not matter whether thread1 and thread2 operate on the 367 same or different atomic variables, or use a "stand-alone" <code class="literal">atomic_thread_fence</code> 368 operation. 369 </p> 370</div> 371</div> 372<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 373<td align="left"></td> 374<td align="right"><div class="copyright-footer">Copyright © 2011 Helge Bahmann<br>Copyright © 2012 Tim Blechmann<br>Copyright © 2013, 2017, 2018, 2020 Andrey 375 Semashev<p> 376 Distributed under the Boost Software License, Version 1.0. (See accompanying 377 file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) 378 </p> 379</div></td> 380</tr></table> 381<hr> 382<div class="spirit-nav"> 383<a accesskey="p" href="../atomic.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../atomic.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="interface.html"><img src="../../../doc/src/images/next.png" alt="Next"></a> 384</div> 385</body> 386</html> 387