1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 2<html> 3<head> 4<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 5<title>Tutorial</title> 6<link rel="stylesheet" href="../../../doc/src/boostbook.css" type="text/css"> 7<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> 8<link rel="home" href="../index.html" title="The Boost C++ Libraries BoostBook Documentation Subset"> 9<link rel="up" href="../mpi.html" title="Chapter 26. Boost.MPI"> 10<link rel="prev" href="getting_started.html" title="Getting started"> 11<link rel="next" href="c_mapping.html" title="Mapping from C MPI to Boost.MPI"> 12</head> 13<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> 14<table cellpadding="2" width="100%"><tr> 15<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../boost.png"></td> 16<td align="center"><a href="../../../index.html">Home</a></td> 17<td align="center"><a href="../../../libs/libraries.htm">Libraries</a></td> 18<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> 19<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> 20<td align="center"><a href="../../../more/index.htm">More</a></td> 21</tr></table> 22<hr> 23<div class="spirit-nav"> 24<a accesskey="p" href="getting_started.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="c_mapping.html"><img src="../../../doc/src/images/next.png" alt="Next"></a> 25</div> 26<div class="section"> 27<div class="titlepage"><div><div><h2 class="title" style="clear: both"> 28<a name="mpi.tutorial"></a><a class="link" href="tutorial.html" title="Tutorial">Tutorial</a> 29</h2></div></div></div> 30<div class="toc"><dl class="toc"> 31<dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point">Point-to-Point communication</a></span></dt> 32<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives">Collective operations</a></span></dt> 33<dt><span class="section"><a href="tutorial.html#mpi.tutorial.user_data_types">User-defined data types</a></span></dt> 34<dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators">Communicators</a></span></dt> 35<dt><span class="section"><a href="tutorial.html#mpi.tutorial.threading">Threads</a></span></dt> 36<dt><span class="section"><a href="tutorial.html#mpi.tutorial.skeleton_and_content">Separating structure 37 from content</a></span></dt> 38<dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations">Performance optimizations</a></span></dt> 39</dl></div> 40<p> 41 A Boost.MPI program consists of many cooperating processes (possibly running 42 on different computers) that communicate among themselves by passing messages. 43 Boost.MPI is a library (as is the lower-level MPI), not a language, so the 44 first step in a Boost.MPI is to create an <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code> 45 object that initializes the MPI environment and enables communication among 46 the processes. The <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code> 47 object is initialized with the program arguments (which it may modify) in your 48 main program. The creation of this object initializes MPI, and its destruction 49 will finalize MPI. In the vast majority of Boost.MPI programs, an instance 50 of <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code> will 51 be declared in <code class="computeroutput"><span class="identifier">main</span></code> at the 52 very beginning of the program. 53 </p> 54<div class="warning"><table border="0" summary="Warning"> 55<tr> 56<td rowspan="2" align="center" valign="top" width="25"><img alt="[Warning]" src="../../../doc/src/images/warning.png"></td> 57<th align="left">Warning</th> 58</tr> 59<tr><td align="left" valign="top"><p> 60 Declaring an <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code> 61 at global scope is undefined behavior. <a href="#ftn.mpi.tutorial.f0" class="footnote" name="mpi.tutorial.f0"><sup class="footnote">[11]</sup></a> 62 </p></td></tr> 63</table></div> 64<p> 65 Communication with MPI always occurs over a <span class="bold"><strong>communicator</strong></span>, 66 which can be created by simply default-constructing an object of type <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">mpi::communicator</a></code>. This communicator 67 can then be queried to determine how many processes are running (the "size" 68 of the communicator) and to give a unique number to each process, from zero 69 to the size of the communicator (i.e., the "rank" of the process): 70 </p> 71<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 72<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 73<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 74<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 75 76<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 77<span class="special">{</span> 78 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 79 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 80 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"I am process "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" of "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> 81 <span class="special"><<</span> <span class="string">"."</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 82 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 83<span class="special">}</span> 84</pre> 85<p> 86 If you run this program with 7 processes, for instance, you will receive output 87 such as: 88 </p> 89<pre class="programlisting">I am process 5 of 7. 90I am process 0 of 7. 91I am process 1 of 7. 92I am process 6 of 7. 93I am process 2 of 7. 94I am process 4 of 7. 95I am process 3 of 7. 96</pre> 97<p> 98 Of course, the processes can execute in a different order each time, so the 99 ranks might not be strictly increasing. More interestingly, the text could 100 come out completely garbled, because one process can start writing "I 101 am a process" before another process has finished writing "of 7.". 102 </p> 103<p> 104 If you should still have an MPI library supporting only MPI 1.1 you will need 105 to pass the command line arguments to the environment constructor as shown 106 in this example: 107 </p> 108<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 109<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 110<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 111<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 112 113<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span> 114<span class="special">{</span> 115 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span> 116 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 117 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"I am process "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" of "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> 118 <span class="special"><<</span> <span class="string">"."</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 119 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 120<span class="special">}</span> 121</pre> 122<div class="section"> 123<div class="titlepage"><div><div><h3 class="title"> 124<a name="mpi.tutorial.point_to_point"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point" title="Point-to-Point communication">Point-to-Point communication</a> 125</h3></div></div></div> 126<div class="toc"><dl class="toc"> 127<dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point.blocking">Blocking communication</a></span></dt> 128<dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point.nonblocking">Non-blocking 129 communication</a></span></dt> 130</dl></div> 131<div class="section"> 132<div class="titlepage"><div><div><h4 class="title"> 133<a name="mpi.tutorial.point_to_point.blocking"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point.blocking" title="Blocking communication">Blocking communication</a> 134</h4></div></div></div> 135<p> 136 As a message passing library, MPI's primary purpose is to routine messages 137 from one process to another, i.e., point-to-point. MPI contains routines 138 that can send messages, receive messages, and query whether messages are 139 available. Each message has a source process, a target process, a tag, 140 and a payload containing arbitrary data. The source and target processes 141 are the ranks of the sender and receiver of the message, respectively. 142 Tags are integers that allow the receiver to distinguish between different 143 messages coming from the same sender. 144 </p> 145<p> 146 The following program uses two MPI processes to write "Hello, world!" 147 to the screen (<code class="computeroutput"><span class="identifier">hello_world</span><span class="special">.</span><span class="identifier">cpp</span></code>): 148 </p> 149<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 150<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 151<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span> 152<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 153<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 154 155<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 156<span class="special">{</span> 157 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 158 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 159 160 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 161 <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"Hello"</span><span class="special">));</span> 162 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span> 163 <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span> 164 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">"!"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 165 <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span> 166 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span> 167 <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span> 168 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">", "</span><span class="special">;</span> 169 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span><span class="special">.</span><span class="identifier">flush</span><span class="special">();</span> 170 <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"world"</span><span class="special">));</span> 171 <span class="special">}</span> 172 173 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 174<span class="special">}</span> 175</pre> 176<p> 177 The first processor (rank 0) passes the message "Hello" to the 178 second processor (rank 1) using tag 0. The second processor prints the 179 string it receives, along with a comma, then passes the message "world" 180 back to processor 0 with a different tag. The first processor then writes 181 this message with the "!" and exits. All sends are accomplished 182 with the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">communicator::send</a></code> 183 method and all receives use a corresponding <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">communicator::recv</a></code> 184 call. 185 </p> 186</div> 187<div class="section"> 188<div class="titlepage"><div><div><h4 class="title"> 189<a name="mpi.tutorial.point_to_point.nonblocking"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point.nonblocking" title="Non-blocking communication">Non-blocking 190 communication</a> 191</h4></div></div></div> 192<p> 193 The default MPI communication operations--<code class="computeroutput"><span class="identifier">send</span></code> 194 and <code class="computeroutput"><span class="identifier">recv</span></code>--may have to wait 195 until the entire transmission is completed before they can return. Sometimes 196 this <span class="bold"><strong>blocking</strong></span> behavior has a negative 197 impact on performance, because the sender could be performing useful computation 198 while it is waiting for the transmission to occur. More important, however, 199 are the cases where several communication operations must occur simultaneously, 200 e.g., a process will both send and receive at the same time. 201 </p> 202<p> 203 Let's revisit our "Hello, world!" program from the previous 204 <a class="link" href="tutorial.html#mpi.tutorial.point_to_point.blocking" title="Blocking communication">section</a>. The 205 core of this program transmits two messages: 206 </p> 207<pre class="programlisting"><span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 208 <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"Hello"</span><span class="special">));</span> 209 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span> 210 <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span> 211 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">"!"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 212<span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span> 213 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span> 214 <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span> 215 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">", "</span><span class="special">;</span> 216 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span><span class="special">.</span><span class="identifier">flush</span><span class="special">();</span> 217 <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"world"</span><span class="special">));</span> 218<span class="special">}</span> 219</pre> 220<p> 221 The first process passes a message to the second process, then prepares 222 to receive a message. The second process does the send and receive in the 223 opposite order. However, this sequence of events is just that--a <span class="bold"><strong>sequence</strong></span>--meaning that there is essentially no parallelism. 224 We can use non-blocking communication to ensure that the two messages are 225 transmitted simultaneously (<code class="computeroutput"><span class="identifier">hello_world_nonblocking</span><span class="special">.</span><span class="identifier">cpp</span></code>): 226 </p> 227<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 228<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 229<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span> 230<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 231<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 232 233<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 234<span class="special">{</span> 235 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 236 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 237 238 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 239 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">request</span> <span class="identifier">reqs</span><span class="special">[</span><span class="number">2</span><span class="special">];</span> 240 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">,</span> <span class="identifier">out_msg</span> <span class="special">=</span> <span class="string">"Hello"</span><span class="special">;</span> 241 <span class="identifier">reqs</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">isend</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">out_msg</span><span class="special">);</span> 242 <span class="identifier">reqs</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">irecv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span> 243 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">wait_all</span><span class="special">(</span><span class="identifier">reqs</span><span class="special">,</span> <span class="identifier">reqs</span> <span class="special">+</span> <span class="number">2</span><span class="special">);</span> 244 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">"!"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 245 <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span> 246 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">request</span> <span class="identifier">reqs</span><span class="special">[</span><span class="number">2</span><span class="special">];</span> 247 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">,</span> <span class="identifier">out_msg</span> <span class="special">=</span> <span class="string">"world"</span><span class="special">;</span> 248 <span class="identifier">reqs</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">isend</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">out_msg</span><span class="special">);</span> 249 <span class="identifier">reqs</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">irecv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span> 250 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">wait_all</span><span class="special">(</span><span class="identifier">reqs</span><span class="special">,</span> <span class="identifier">reqs</span> <span class="special">+</span> <span class="number">2</span><span class="special">);</span> 251 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="identifier">msg</span> <span class="special"><<</span> <span class="string">", "</span><span class="special">;</span> 252 <span class="special">}</span> 253 254 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 255<span class="special">}</span> 256</pre> 257<p> 258 We have replaced calls to the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">communicator::send</a></code> 259 and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">communicator::recv</a></code> 260 members with similar calls to their non-blocking counterparts, <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_16-bb">communicator::isend</a></code> 261 and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_21-bb">communicator::irecv</a></code>. 262 The prefix <span class="bold"><strong>i</strong></span> indicates that the operations 263 return immediately with a <code class="computeroutput"><a class="link" href="../boost/mpi/request.html" title="Class request">mpi::request</a></code> 264 object, which allows one to query the status of a communication request 265 (see the <code class="computeroutput"><a class="link" href="../boost/mpi/request.html#id-1_3_27_7_21_2_1_1_7_2-bb">test</a></code> 266 method) or wait until it has completed (see the <code class="computeroutput"><a class="link" href="../boost/mpi/request.html#id-1_3_27_7_21_2_1_1_7_1-bb">wait</a></code> 267 method). Multiple requests can be completed at the same time with the 268 <code class="computeroutput"><a class="link" href="../boost/mpi/wait_all.html" title="Function wait_all">wait_all</a></code> operation. 269 </p> 270<div class="important"><table border="0" summary="Important"> 271<tr> 272<td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="../../../doc/src/images/important.png"></td> 273<th align="left">Important</th> 274</tr> 275<tr><td align="left" valign="top"><p> 276 Regarding communication completion/progress: The MPI standard requires 277 users to keep the request handle for a non-blocking communication, and 278 to call the "wait" operation (or successfully test for completion) 279 to complete the send or receive. Unlike most C MPI implementations, which 280 allow the user to discard the request for a non-blocking send, Boost.MPI 281 requires the user to call "wait" or "test", since 282 the request object might contain temporary buffers that have to be kept 283 until the send is completed. Moreover, the MPI standard does not guarantee 284 that the receive makes any progress before a call to "wait" 285 or "test", although most implementations of the C MPI do allow 286 receives to progress before the call to "wait" or "test". 287 Boost.MPI, on the other hand, generally requires "test" or 288 "wait" calls to make progress. More specifically, Boost.MPI 289 guarantee that calling "test" multiple time will eventually 290 complete the communication (this is due to the fact that serialized communication 291 are potentially a multi step operation.). 292 </p></td></tr> 293</table></div> 294<p> 295 If you run this program multiple times, you may see some strange results: 296 namely, some runs will produce: 297 </p> 298<pre class="programlisting"><span class="identifier">Hello</span><span class="special">,</span> <span class="identifier">world</span><span class="special">!</span> 299</pre> 300<p> 301 while others will produce: 302 </p> 303<pre class="programlisting"><span class="identifier">world</span><span class="special">!</span> 304<span class="identifier">Hello</span><span class="special">,</span> 305</pre> 306<p> 307 or even some garbled version of the letters in "Hello" and "world". 308 This indicates that there is some parallelism in the program, because after 309 both messages are (simultaneously) transmitted, both processes will concurrent 310 execute their print statements. For both performance and correctness, non-blocking 311 communication operations are critical to many parallel applications using 312 MPI. 313 </p> 314</div> 315</div> 316<div class="section"> 317<div class="titlepage"><div><div><h3 class="title"> 318<a name="mpi.tutorial.collectives"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives" title="Collective operations">Collective operations</a> 319</h3></div></div></div> 320<div class="toc"><dl class="toc"> 321<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.broadcast">Broadcast</a></span></dt> 322<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.gather">Gather</a></span></dt> 323<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.scatter">Scatter</a></span></dt> 324<dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.reduce">Reduce</a></span></dt> 325</dl></div> 326<p> 327 <a class="link" href="tutorial.html#mpi.tutorial.point_to_point" title="Point-to-Point communication">Point-to-point operations</a> 328 are the core message passing primitives in Boost.MPI. However, many message-passing 329 applications also require higher-level communication algorithms that combine 330 or summarize the data stored on many different processes. These algorithms 331 support many common tasks such as "broadcast this value to all processes", 332 "compute the sum of the values on all processors" or "find 333 the global minimum." 334 </p> 335<div class="section"> 336<div class="titlepage"><div><div><h4 class="title"> 337<a name="mpi.tutorial.collectives.broadcast"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.broadcast" title="Broadcast">Broadcast</a> 338</h4></div></div></div> 339<p> 340 The <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast</a></code> 341 algorithm is by far the simplest collective operation. It broadcasts a 342 value from a single process to all other processes within a <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">communicator</a></code>. For instance, 343 the following program broadcasts "Hello, World!" from process 344 0 to every other process. (<code class="computeroutput"><span class="identifier">hello_world_broadcast</span><span class="special">.</span><span class="identifier">cpp</span></code>) 345 </p> 346<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 347<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 348<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span> 349<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 350<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 351 352<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 353<span class="special">{</span> 354 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 355 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 356 357 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">value</span><span class="special">;</span> 358 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 359 <span class="identifier">value</span> <span class="special">=</span> <span class="string">"Hello, World!"</span><span class="special">;</span> 360 <span class="special">}</span> 361 362 <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">value</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 363 364 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"Process #"</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" says "</span> <span class="special"><<</span> <span class="identifier">value</span> 365 <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 366 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 367<span class="special">}</span> 368</pre> 369<p> 370 Running this program with seven processes will produce a result such as: 371 </p> 372<pre class="programlisting">Process #0 says Hello, World! 373Process #2 says Hello, World! 374Process #1 says Hello, World! 375Process #4 says Hello, World! 376Process #3 says Hello, World! 377Process #5 says Hello, World! 378Process #6 says Hello, World! 379</pre> 380</div> 381<div class="section"> 382<div class="titlepage"><div><div><h4 class="title"> 383<a name="mpi.tutorial.collectives.gather"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.gather" title="Gather">Gather</a> 384</h4></div></div></div> 385<p> 386 The <code class="computeroutput"><a class="link" href="../boost/mpi/gather.html" title="Function gather">gather</a></code> 387 collective gathers the values produced by every process in a communicator 388 into a vector of values on the "root" process (specified by an 389 argument to <code class="computeroutput"><span class="identifier">gather</span></code>). The 390 /i/th element in the vector will correspond to the value gathered from 391 the /i/th process. For instance, in the following program each process 392 computes its own random number. All of these random numbers are gathered 393 at process 0 (the "root" in this case), which prints out the 394 values that correspond to each processor. (<code class="computeroutput"><span class="identifier">random_gather</span><span class="special">.</span><span class="identifier">cpp</span></code>) 395 </p> 396<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 397<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 398<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">vector</span><span class="special">></span> 399<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span> 400<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 401 402<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 403<span class="special">{</span> 404 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 405 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 406 407 <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span> 408 <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span> 409 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 410 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">all_numbers</span><span class="special">;</span> 411 <span class="identifier">gather</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">all_numbers</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 412 <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">proc</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">proc</span> <span class="special"><</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">proc</span><span class="special">)</span> 413 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"Process #"</span> <span class="special"><<</span> <span class="identifier">proc</span> <span class="special"><<</span> <span class="string">" thought of "</span> 414 <span class="special"><<</span> <span class="identifier">all_numbers</span><span class="special">[</span><span class="identifier">proc</span><span class="special">]</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 415 <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span> 416 <span class="identifier">gather</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 417 <span class="special">}</span> 418 419 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 420<span class="special">}</span> 421</pre> 422<p> 423 Executing this program with seven processes will result in output such 424 as the following. Although the random values will change from one run to 425 the next, the order of the processes in the output will remain the same 426 because only process 0 writes to <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span></code>. 427 </p> 428<pre class="programlisting">Process #0 thought of 332199874 429Process #1 thought of 20145617 430Process #2 thought of 1862420122 431Process #3 thought of 480422940 432Process #4 thought of 1253380219 433Process #5 thought of 949458815 434Process #6 thought of 650073868 435</pre> 436<p> 437 The <code class="computeroutput"><span class="identifier">gather</span></code> operation collects 438 values from every process into a vector at one process. If instead the 439 values from every process need to be collected into identical vectors on 440 every process, use the <code class="computeroutput"><a class="link" href="../boost/mpi/all_gather.html" title="Function all_gather">all_gather</a></code> algorithm, 441 which is semantically equivalent to calling <code class="computeroutput"><span class="identifier">gather</span></code> 442 followed by a <code class="computeroutput"><span class="identifier">broadcast</span></code> 443 of the resulting vector. 444 </p> 445</div> 446<div class="section"> 447<div class="titlepage"><div><div><h4 class="title"> 448<a name="mpi.tutorial.collectives.scatter"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.scatter" title="Scatter">Scatter</a> 449</h4></div></div></div> 450<p> 451 The <code class="computeroutput"><a class="link" href="../boost/mpi/scatter.html" title="Function scatter">scatter</a></code> 452 collective scatters the values from a vector in the "root" process 453 in a communicator into values in all the processes of the communicator. 454 The /i/th element in the vector will correspond to the value received by 455 the /i/th process. For instance, in the following program, the root process 456 produces a vector of random nomber and send one value to each process that 457 will print it. (<code class="computeroutput"><span class="identifier">random_scatter</span><span class="special">.</span><span class="identifier">cpp</span></code>) 458 </p> 459<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 460<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">collectives</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 461<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 462<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span> 463<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">vector</span><span class="special">></span> 464 465<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 466 467<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span> 468<span class="special">{</span> 469 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span> 470 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 471 472 <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span> 473 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">all</span><span class="special">;</span> 474 <span class="keyword">int</span> <span class="identifier">mine</span> <span class="special">=</span> <span class="special">-</span><span class="number">1</span><span class="special">;</span> 475 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 476 <span class="identifier">all</span><span class="special">.</span><span class="identifier">resize</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">());</span> 477 <span class="identifier">std</span><span class="special">::</span><span class="identifier">generate</span><span class="special">(</span><span class="identifier">all</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">all</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">);</span> 478 <span class="special">}</span> 479 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">scatter</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">all</span><span class="special">,</span> <span class="identifier">mine</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 480 <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">r</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">r</span> <span class="special"><</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">r</span><span class="special">)</span> <span class="special">{</span> 481 <span class="identifier">world</span><span class="special">.</span><span class="identifier">barrier</span><span class="special">();</span> 482 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">r</span> <span class="special">==</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">())</span> <span class="special">{</span> 483 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"Rank "</span> <span class="special"><<</span> <span class="identifier">r</span> <span class="special"><<</span> <span class="string">" got "</span> <span class="special"><<</span> <span class="identifier">mine</span> <span class="special"><<</span> <span class="char">'\n'</span><span class="special">;</span> 484 <span class="special">}</span> 485 <span class="special">}</span> 486 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 487<span class="special">}</span> 488</pre> 489<p> 490 Executing this program with seven processes will result in output such 491 as the following. Although the random values will change from one run to 492 the next, the order of the processes in the output will remain the same 493 because of the barrier. 494 </p> 495<pre class="programlisting">Rank 0 got 1409381269 496Rank 1 got 17045268 497Rank 2 got 440120016 498Rank 3 got 936998224 499Rank 4 got 1827129182 500Rank 5 got 1951746047 501Rank 6 got 2117359639 502</pre> 503</div> 504<div class="section"> 505<div class="titlepage"><div><div><h4 class="title"> 506<a name="mpi.tutorial.collectives.reduce"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce" title="Reduce">Reduce</a> 507</h4></div></div></div> 508<p> 509 The <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce</a></code> 510 collective summarizes the values from each process into a single value 511 at the user-specified "root" process. The Boost.MPI <code class="computeroutput"><span class="identifier">reduce</span></code> operation is similar in spirit 512 to the STL <a href="http://www.sgi.com/tech/stl/accumulate.html" target="_top"><code class="computeroutput"><span class="identifier">accumulate</span></code></a> operation, because 513 it takes a sequence of values (one per process) and combines them via a 514 function object. For instance, we can randomly generate values in each 515 process and the compute the minimum value over all processes via a call 516 to <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce</a></code> 517 (<code class="computeroutput"><span class="identifier">random_min</span><span class="special">.</span><span class="identifier">cpp</span></code>): 518 </p> 519<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 520<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 521<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span> 522<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 523 524<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 525<span class="special">{</span> 526 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 527 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 528 529 <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span> 530 <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span> 531 532 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 533 <span class="keyword">int</span> <span class="identifier">minimum</span><span class="special">;</span> 534 <span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">minimum</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>(),</span> <span class="number">0</span><span class="special">);</span> 535 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The minimum value is "</span> <span class="special"><<</span> <span class="identifier">minimum</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 536 <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span> 537 <span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>(),</span> <span class="number">0</span><span class="special">);</span> 538 <span class="special">}</span> 539 540 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 541<span class="special">}</span> 542</pre> 543<p> 544 The use of <code class="computeroutput"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span></code> 545 indicates that the minimum value should be computed. <code class="computeroutput"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span></code> is a binary function object that compares 546 its two parameters via <code class="computeroutput"><span class="special"><</span></code> 547 and returns the smaller value. Any associative binary function or function 548 object will work provided it's stateless. For instance, to concatenate 549 strings with <code class="computeroutput"><span class="identifier">reduce</span></code> one 550 could use the function object <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">></span></code> (<code class="computeroutput"><span class="identifier">string_cat</span><span class="special">.</span><span class="identifier">cpp</span></code>): 551 </p> 552<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 553<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 554<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">string</span><span class="special">></span> 555<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">functional</span><span class="special">></span> 556<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 557<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 558 559<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 560<span class="special">{</span> 561 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 562 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 563 564 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">names</span><span class="special">[</span><span class="number">10</span><span class="special">]</span> <span class="special">=</span> <span class="special">{</span> <span class="string">"zero "</span><span class="special">,</span> <span class="string">"one "</span><span class="special">,</span> <span class="string">"two "</span><span class="special">,</span> <span class="string">"three "</span><span class="special">,</span> 565 <span class="string">"four "</span><span class="special">,</span> <span class="string">"five "</span><span class="special">,</span> <span class="string">"six "</span><span class="special">,</span> <span class="string">"seven "</span><span class="special">,</span> 566 <span class="string">"eight "</span><span class="special">,</span> <span class="string">"nine "</span> <span class="special">};</span> 567 568 <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">result</span><span class="special">;</span> 569 <span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> 570 <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><</span> <span class="number">10</span><span class="special">?</span> <span class="identifier">names</span><span class="special">[</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()]</span> 571 <span class="special">:</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"many "</span><span class="special">),</span> 572 <span class="identifier">result</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">>(),</span> <span class="number">0</span><span class="special">);</span> 573 574 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> 575 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The result is "</span> <span class="special"><<</span> <span class="identifier">result</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 576 577 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 578<span class="special">}</span> 579</pre> 580<p> 581 In this example, we compute a string for each process and then perform 582 a reduction that concatenates all of the strings together into one, long 583 string. Executing this program with seven processors yields the following 584 output: 585 </p> 586<pre class="programlisting">The result is zero one two three four five six 587</pre> 588<h5> 589<a name="mpi.tutorial.collectives.reduce.h0"></a> 590 <span class="phrase"><a name="mpi.tutorial.collectives.reduce.binary_operations_for_reduce"></a></span><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce.binary_operations_for_reduce">Binary 591 operations for reduce</a> 592 </h5> 593<p> 594 Any kind of binary function objects can be used with <code class="computeroutput"><span class="identifier">reduce</span></code>. 595 For instance, and there are many such function objects in the C++ standard 596 <code class="computeroutput"><span class="special"><</span><span class="identifier">functional</span><span class="special">></span></code> header and the Boost.MPI header <code class="computeroutput"><span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">operations</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span></code>. Or, you can create your own function 597 object. Function objects used with <code class="computeroutput"><span class="identifier">reduce</span></code> 598 must be associative, i.e. <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> 599 <span class="identifier">f</span><span class="special">(</span><span class="identifier">y</span><span class="special">,</span> <span class="identifier">z</span><span class="special">))</span></code> must be equivalent to <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">y</span><span class="special">),</span> <span class="identifier">z</span><span class="special">)</span></code>. If they are also commutative (i..e, 600 <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">y</span><span class="special">)</span> <span class="special">==</span> <span class="identifier">f</span><span class="special">(</span><span class="identifier">y</span><span class="special">,</span> 601 <span class="identifier">x</span><span class="special">)</span></code>), 602 Boost.MPI can use a more efficient implementation of <code class="computeroutput"><span class="identifier">reduce</span></code>. 603 To state that a function object is commutative, you will need to specialize 604 the class <code class="computeroutput"><a class="link" href="../boost/mpi/is_commutative.html" title="Struct template is_commutative">is_commutative</a></code>. 605 For instance, we could modify the previous example by telling Boost.MPI 606 that string concatenation is commutative: 607 </p> 608<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span> 609 610 <span class="keyword">template</span><span class="special"><></span> 611 <span class="keyword">struct</span> <span class="identifier">is_commutative</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special"><</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">>,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">></span> 612 <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span> 613 614<span class="special">}</span> <span class="special">}</span> <span class="comment">// end namespace boost::mpi</span> 615</pre> 616<p> 617 By adding this code prior to <code class="computeroutput"><span class="identifier">main</span><span class="special">()</span></code>, Boost.MPI will assume that string concatenation 618 is commutative and employ a different parallel algorithm for the <code class="computeroutput"><span class="identifier">reduce</span></code> operation. Using this algorithm, 619 the program outputs the following when run with seven processes: 620 </p> 621<pre class="programlisting">The result is zero one four five six two three 622</pre> 623<p> 624 Note how the numbers in the resulting string are in a different order: 625 this is a direct result of Boost.MPI reordering operations. The result 626 in this case differed from the non-commutative result because string concatenation 627 is not commutative: <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="string">"x"</span><span class="special">,</span> 628 <span class="string">"y"</span><span class="special">)</span></code> 629 is not the same as <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="string">"y"</span><span class="special">,</span> 630 <span class="string">"x"</span><span class="special">)</span></code>, 631 because argument order matters. For truly commutative operations (e.g., 632 integer addition), the more efficient commutative algorithm will produce 633 the same result as the non-commutative algorithm. Boost.MPI also performs 634 direct mappings from function objects in <code class="computeroutput"><span class="special"><</span><span class="identifier">functional</span><span class="special">></span></code> 635 to <code class="computeroutput"><span class="identifier">MPI_Op</span></code> values predefined 636 by MPI (e.g., <code class="computeroutput"><span class="identifier">MPI_SUM</span></code>, 637 <code class="computeroutput"><span class="identifier">MPI_MAX</span></code>); if you have your 638 own function objects that can take advantage of this mapping, see the class 639 template <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_op.html" title="Struct template is_mpi_op">is_mpi_op</a></code>. 640 </p> 641<div class="warning"><table border="0" summary="Warning"> 642<tr> 643<td rowspan="2" align="center" valign="top" width="25"><img alt="[Warning]" src="../../../doc/src/images/warning.png"></td> 644<th align="left">Warning</th> 645</tr> 646<tr><td align="left" valign="top"><p> 647 Due to the underlying MPI limitations, it is important to note that the 648 operation must be stateless. 649 </p></td></tr> 650</table></div> 651<h5> 652<a name="mpi.tutorial.collectives.reduce.h1"></a> 653 <span class="phrase"><a name="mpi.tutorial.collectives.reduce.all_process_variant"></a></span><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce.all_process_variant">All process 654 variant</a> 655 </h5> 656<p> 657 Like <a class="link" href="tutorial.html#mpi.tutorial.collectives.gather" title="Gather"><code class="computeroutput"><span class="identifier">gather</span></code></a>, 658 <code class="computeroutput"><span class="identifier">reduce</span></code> has an "all" 659 variant called <code class="computeroutput"><a class="link" href="../boost/mpi/all_reduce.html" title="Function all_reduce">all_reduce</a></code> that performs 660 the reduction operation and broadcasts the result to all processes. This 661 variant is useful, for instance, in establishing global minimum or maximum 662 values. 663 </p> 664<p> 665 The following code (<code class="computeroutput"><span class="identifier">global_min</span><span class="special">.</span><span class="identifier">cpp</span></code>) 666 shows a broadcasting version of the <code class="computeroutput"><span class="identifier">random_min</span><span class="special">.</span><span class="identifier">cpp</span></code> 667 example: 668 </p> 669<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 670<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 671<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span> 672<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 673 674<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span> 675<span class="special">{</span> 676 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span> 677 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 678 679 <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span> 680 <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span> 681 <span class="keyword">int</span> <span class="identifier">minimum</span><span class="special">;</span> 682 683 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">minimum</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>());</span> 684 685 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 686 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The minimum value is "</span> <span class="special"><<</span> <span class="identifier">minimum</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 687 <span class="special">}</span> 688 689 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 690<span class="special">}</span> 691</pre> 692<p> 693 In that example we provide both input and output values, requiring twice 694 as much space, which can be a problem depending on the size of the transmitted 695 data. If there is no need to preserve the input value, the output value 696 can be omitted. In that case the input value will be overridden with the 697 output value and Boost.MPI is able, in some situation, to implement the 698 operation with a more space efficient solution (using the <code class="computeroutput"><span class="identifier">MPI_IN_PLACE</span></code> flag of the MPI C mapping), 699 as in the following example (<code class="computeroutput"><span class="identifier">in_place_global_min</span><span class="special">.</span><span class="identifier">cpp</span></code>): 700 </p> 701<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 702<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 703<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span> 704<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 705 706<span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span> 707<span class="special">{</span> 708 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span> 709 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 710 711 <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span> 712 <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span> 713 714 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special"><</span><span class="keyword">int</span><span class="special">>());</span> 715 716 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span> 717 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"The minimum value is "</span> <span class="special"><<</span> <span class="identifier">my_number</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 718 <span class="special">}</span> 719 720 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 721<span class="special">}</span> 722</pre> 723</div> 724</div> 725<div class="section"> 726<div class="titlepage"><div><div><h3 class="title"> 727<a name="mpi.tutorial.user_data_types"></a><a class="link" href="tutorial.html#mpi.tutorial.user_data_types" title="User-defined data types">User-defined data types</a> 728</h3></div></div></div> 729<p> 730 The inclusion of <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span></code> in the previous examples is very important: 731 it makes values of type <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code> 732 serializable, so that they can be be transmitted using Boost.MPI. In general, 733 built-in C++ types (<code class="computeroutput"><span class="keyword">int</span></code>s, <code class="computeroutput"><span class="keyword">float</span></code>s, characters, etc.) can be transmitted 734 over MPI directly, while user-defined and library-defined types will need 735 to first be serialized (packed) into a format that is amenable to transmission. 736 Boost.MPI relies on the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a> 737 library to serialize and deserialize data types. 738 </p> 739<p> 740 For types defined by the standard library (such as <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code> 741 or <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span></code>) and some types in Boost (such as 742 <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">variant</span></code>), the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a> 743 library already contains all of the required serialization code. In these 744 cases, you need only include the appropriate header from the <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span></code> directory. 745 </p> 746<p> 747 For types that do not already have a serialization header, you will first 748 need to implement serialization code before the types can be transmitted 749 using Boost.MPI. Consider a simple class <a class="link" href="tutorial.html#gps_position"><code class="computeroutput"><span class="identifier">gps_position</span></code></a> that contains members 750 <code class="computeroutput"><span class="identifier">degrees</span></code>, <code class="computeroutput"><span class="identifier">minutes</span></code>, 751 and <code class="computeroutput"><span class="identifier">seconds</span></code>. This class is 752 made serializable by making it a friend of <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">serialization</span><span class="special">::</span><span class="identifier">access</span></code> 753 and introducing the templated <code class="computeroutput"><span class="identifier">serialize</span><span class="special">()</span></code> function, as follows:<a name="gps_position"></a> 754 </p> 755<pre class="programlisting"><span class="keyword">class</span> <span class="identifier">gps_position</span> 756<span class="special">{</span> 757<span class="keyword">private</span><span class="special">:</span> 758 <span class="keyword">friend</span> <span class="keyword">class</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">serialization</span><span class="special">::</span><span class="identifier">access</span><span class="special">;</span> 759 760 <span class="keyword">template</span><span class="special"><</span><span class="keyword">class</span> <span class="identifier">Archive</span><span class="special">></span> 761 <span class="keyword">void</span> <span class="identifier">serialize</span><span class="special">(</span><span class="identifier">Archive</span> <span class="special">&</span> <span class="identifier">ar</span><span class="special">,</span> <span class="keyword">const</span> <span class="keyword">unsigned</span> <span class="keyword">int</span> <span class="identifier">version</span><span class="special">)</span> 762 <span class="special">{</span> 763 <span class="identifier">ar</span> <span class="special">&</span> <span class="identifier">degrees</span><span class="special">;</span> 764 <span class="identifier">ar</span> <span class="special">&</span> <span class="identifier">minutes</span><span class="special">;</span> 765 <span class="identifier">ar</span> <span class="special">&</span> <span class="identifier">seconds</span><span class="special">;</span> 766 <span class="special">}</span> 767 768 <span class="keyword">int</span> <span class="identifier">degrees</span><span class="special">;</span> 769 <span class="keyword">int</span> <span class="identifier">minutes</span><span class="special">;</span> 770 <span class="keyword">float</span> <span class="identifier">seconds</span><span class="special">;</span> 771<span class="keyword">public</span><span class="special">:</span> 772 <span class="identifier">gps_position</span><span class="special">(){};</span> 773 <span class="identifier">gps_position</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">d</span><span class="special">,</span> <span class="keyword">int</span> <span class="identifier">m</span><span class="special">,</span> <span class="keyword">float</span> <span class="identifier">s</span><span class="special">)</span> <span class="special">:</span> 774 <span class="identifier">degrees</span><span class="special">(</span><span class="identifier">d</span><span class="special">),</span> <span class="identifier">minutes</span><span class="special">(</span><span class="identifier">m</span><span class="special">),</span> <span class="identifier">seconds</span><span class="special">(</span><span class="identifier">s</span><span class="special">)</span> 775 <span class="special">{}</span> 776<span class="special">};</span> 777</pre> 778<p> 779 Complete information about making types serializable is beyond the scope 780 of this tutorial. For more information, please see the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a> 781 library tutorial from which the above example was extracted. One important 782 side benefit of making types serializable for Boost.MPI is that they become 783 serializable for any other usage, such as storing the objects to disk and 784 manipulated them in XML. 785 </p> 786<p> 787 Some serializable types, like <a class="link" href="tutorial.html#gps_position"><code class="computeroutput"><span class="identifier">gps_position</span></code></a> above, have a fixed 788 amount of data stored at fixed offsets and are fully defined by the values 789 of their data member (most POD with no pointers are a good example). When 790 this is the case, Boost.MPI can optimize their serialization and transmission 791 by avoiding extraneous copy operations. To enable this optimization, users 792 must specialize the type trait <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>, e.g.: 793 </p> 794<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span> 795 <span class="keyword">template</span> <span class="special"><></span> 796 <span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">gps_position</span><span class="special">></span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span> 797<span class="special">}</span> <span class="special">}</span> 798</pre> 799<p> 800 For non-template types we have defined a macro to simplify declaring a type 801 as an MPI datatype 802 </p> 803<pre class="programlisting"><span class="identifier">BOOST_IS_MPI_DATATYPE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span> 804</pre> 805<p> 806 For composite traits, the specialization of <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code> may depend 807 on <code class="computeroutput"><span class="identifier">is_mpi_datatype</span></code> itself. 808 For instance, a <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">array</span></code> object is fixed only when the type 809 of the parameter it stores is fixed: 810 </p> 811<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span> 812 <span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">T</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span> <span class="identifier">N</span><span class="special">></span> 813 <span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">array</span><span class="special"><</span><span class="identifier">T</span><span class="special">,</span> <span class="identifier">N</span><span class="special">></span> <span class="special">></span> 814 <span class="special">:</span> <span class="keyword">public</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">T</span><span class="special">></span> <span class="special">{</span> <span class="special">};</span> 815<span class="special">}</span> <span class="special">}</span> 816</pre> 817<p> 818 The redundant copy elimination optimization can only be applied when the 819 shape of the data type is completely fixed. Variable-length types (e.g., 820 strings, linked lists) and types that store pointers cannot use the optimization, 821 but Boost.MPI will be unable to detect this error at compile time. Attempting 822 to perform this optimization when it is not correct will likely result in 823 segmentation faults and other strange program behavior. 824 </p> 825<p> 826 Boost.MPI can transmit any user-defined data type from one process to another. 827 Built-in types can be transmitted without any extra effort; library-defined 828 types require the inclusion of a serialization header; and user-defined types 829 will require the addition of serialization code. Fixed data types can be 830 optimized for transmission using the <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code> type trait. 831 </p> 832</div> 833<div class="section"> 834<div class="titlepage"><div><div><h3 class="title"> 835<a name="mpi.tutorial.communicators"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators" title="Communicators">Communicators</a> 836</h3></div></div></div> 837<div class="toc"><dl class="toc"> 838<dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators.managing">Managing communicators</a></span></dt> 839<dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators.cartesian_communicator">Cartesian 840 communicator</a></span></dt> 841</dl></div> 842<div class="section"> 843<div class="titlepage"><div><div><h4 class="title"> 844<a name="mpi.tutorial.communicators.managing"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators.managing" title="Managing communicators">Managing communicators</a> 845</h4></div></div></div> 846<p> 847 Communication with Boost.MPI always occurs over a communicator. A communicator 848 contains a set of processes that can send messages among themselves and 849 perform collective operations. There can be many communicators within a 850 single program, each of which contains its own isolated communication space 851 that acts independently of the other communicators. 852 </p> 853<p> 854 When the MPI environment is initialized, only the "world" communicator 855 (called <code class="computeroutput"><span class="identifier">MPI_COMM_WORLD</span></code> 856 in the MPI C and Fortran bindings) is available. The "world" 857 communicator, accessed by default-constructing a <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">mpi::communicator</a></code> 858 object, contains all of the MPI processes present when the program begins 859 execution. Other communicators can then be constructed by duplicating or 860 building subsets of the "world" communicator. For instance, in 861 the following program we split the processes into two groups: one for processes 862 generating data and the other for processes that will collect the data. 863 (<code class="computeroutput"><span class="identifier">generate_collect</span><span class="special">.</span><span class="identifier">cpp</span></code>) 864 </p> 865<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 866<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 867<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">cstdlib</span><span class="special">></span> 868<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">vector</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 869<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 870 871<span class="keyword">enum</span> <span class="identifier">message_tags</span> <span class="special">{</span><span class="identifier">msg_data_packet</span><span class="special">,</span> <span class="identifier">msg_broadcast_data</span><span class="special">,</span> <span class="identifier">msg_finished</span><span class="special">};</span> 872 873<span class="keyword">void</span> <span class="identifier">generate_data</span><span class="special">(</span><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">);</span> 874<span class="keyword">void</span> <span class="identifier">collect_data</span><span class="special">(</span><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">);</span> 875 876<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 877<span class="special">{</span> 878 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 879 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 880 881 <span class="keyword">bool</span> <span class="identifier">is_generator</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><</span> <span class="number">2</span> <span class="special">*</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> <span class="special">/</span> <span class="number">3</span><span class="special">;</span> 882 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">split</span><span class="special">(</span><span class="identifier">is_generator</span><span class="special">?</span> <span class="number">0</span> <span class="special">:</span> <span class="number">1</span><span class="special">);</span> 883 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">is_generator</span><span class="special">)</span> <span class="identifier">generate_data</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">world</span><span class="special">);</span> 884 <span class="keyword">else</span> <span class="identifier">collect_data</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">world</span><span class="special">);</span> 885 886 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 887<span class="special">}</span> 888</pre> 889<p> 890 When communicators are split in this way, their processes retain membership 891 in both the original communicator (which is not altered by the split) and 892 the new communicator. However, the ranks of the processes may be different 893 from one communicator to the next, because the rank values within a communicator 894 are always contiguous values starting at zero. In the example above, the 895 first two thirds of the processes become "generators" and the 896 remaining processes become "collectors". The ranks of the "collectors" 897 in the <code class="computeroutput"><span class="identifier">world</span></code> communicator 898 will be 2/3 <code class="computeroutput"><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span></code> 899 and greater, whereas the ranks of the same collector processes in the 900 <code class="computeroutput"><span class="identifier">local</span></code> communicator will 901 start at zero. The following excerpt from <code class="computeroutput"><span class="identifier">collect_data</span><span class="special">()</span></code> (in <code class="computeroutput"><span class="identifier">generate_collect</span><span class="special">.</span><span class="identifier">cpp</span></code>) 902 illustrates how to manage multiple communicators: 903 </p> 904<pre class="programlisting"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">status</span> <span class="identifier">msg</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">probe</span><span class="special">();</span> 905<span class="keyword">if</span> <span class="special">(</span><span class="identifier">msg</span><span class="special">.</span><span class="identifier">tag</span><span class="special">()</span> <span class="special">==</span> <span class="identifier">msg_data_packet</span><span class="special">)</span> <span class="special">{</span> 906 <span class="comment">// Receive the packet of data</span> 907 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">data</span><span class="special">;</span> 908 <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="identifier">msg</span><span class="special">.</span><span class="identifier">source</span><span class="special">(),</span> <span class="identifier">msg</span><span class="special">.</span><span class="identifier">tag</span><span class="special">(),</span> <span class="identifier">data</span><span class="special">);</span> 909 910 <span class="comment">// Tell each of the collectors that we'll be broadcasting some data</span> 911 <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">dest</span> <span class="special">=</span> <span class="number">1</span><span class="special">;</span> <span class="identifier">dest</span> <span class="special"><</span> <span class="identifier">local</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">dest</span><span class="special">)</span> 912 <span class="identifier">local</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="identifier">dest</span><span class="special">,</span> <span class="identifier">msg_broadcast_data</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">.</span><span class="identifier">source</span><span class="special">());</span> 913 914 <span class="comment">// Broadcast the actual data.</span> 915 <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">data</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 916<span class="special">}</span> 917</pre> 918<p> 919 The code in this except is executed by the "master" collector, 920 e.g., the node with rank 2/3 <code class="computeroutput"><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span></code> in the <code class="computeroutput"><span class="identifier">world</span></code> 921 communicator and rank 0 in the <code class="computeroutput"><span class="identifier">local</span></code> 922 (collector) communicator. It receives a message from a generator via the 923 <code class="computeroutput"><span class="identifier">world</span></code> communicator, then 924 broadcasts the message to each of the collectors via the <code class="computeroutput"><span class="identifier">local</span></code> communicator. 925 </p> 926<p> 927 For more control in the creation of communicators for subgroups of processes, 928 the Boost.MPI <code class="computeroutput"><a class="link" href="../boost/mpi/group.html" title="Class group">group</a></code> 929 provides facilities to compute the union (<code class="computeroutput"><span class="special">|</span></code>), 930 intersection (<code class="computeroutput"><span class="special">&</span></code>), and 931 difference (<code class="computeroutput"><span class="special">-</span></code>) of two groups, 932 generate arbitrary subgroups, etc. 933 </p> 934</div> 935<div class="section"> 936<div class="titlepage"><div><div><h4 class="title"> 937<a name="mpi.tutorial.communicators.cartesian_communicator"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators.cartesian_communicator" title="Cartesian communicator">Cartesian 938 communicator</a> 939</h4></div></div></div> 940<p> 941 A communicator can be organised as a cartesian grid, here a basic example: 942 </p> 943<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">vector</span><span class="special">></span> 944<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 945 946<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 947<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">collectives</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 948<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 949<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">cartesian_communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 950 951<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">minimal</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 952 953<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 954<span class="keyword">int</span> <span class="identifier">test_main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span> 955<span class="special">{</span> 956 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span> 957 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 958 959 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> <span class="special">!=</span> <span class="number">24</span><span class="special">)</span> <span class="keyword">return</span> <span class="special">-</span><span class="number">1</span><span class="special">;</span> 960 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_dimension</span> <span class="identifier">dims</span><span class="special">[]</span> <span class="special">=</span> <span class="special">{{</span><span class="number">2</span><span class="special">,</span> <span class="keyword">true</span><span class="special">},</span> <span class="special">{</span><span class="number">3</span><span class="special">,</span><span class="keyword">true</span><span class="special">},</span> <span class="special">{</span><span class="number">4</span><span class="special">,</span><span class="keyword">true</span><span class="special">}};</span> 961 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_communicator</span> <span class="identifier">cart</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_topology</span><span class="special">(</span><span class="identifier">dims</span><span class="special">));</span> 962 <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">r</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">r</span> <span class="special"><</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">r</span><span class="special">)</span> <span class="special">{</span> 963 <span class="identifier">cart</span><span class="special">.</span><span class="identifier">barrier</span><span class="special">();</span> 964 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">r</span> <span class="special">==</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">rank</span><span class="special">())</span> <span class="special">{</span> 965 <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">coordinates</span><span class="special">(</span><span class="identifier">r</span><span class="special">);</span> 966 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"rk :"</span> <span class="special"><<</span> <span class="identifier">r</span> <span class="special"><<</span> <span class="string">" coords: "</span> 967 <span class="special"><<</span> <span class="identifier">c</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special"><<</span> <span class="char">' '</span> <span class="special"><<</span> <span class="identifier">c</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special"><<</span> <span class="char">' '</span> <span class="special"><<</span> <span class="identifier">c</span><span class="special">[</span><span class="number">2</span><span class="special">]</span> <span class="special"><<</span> <span class="char">'\n'</span><span class="special">;</span> 968 <span class="special">}</span> 969 <span class="special">}</span> 970 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 971<span class="special">}</span> 972</pre> 973</div> 974</div> 975<div class="section"> 976<div class="titlepage"><div><div><h3 class="title"> 977<a name="mpi.tutorial.threading"></a><a class="link" href="tutorial.html#mpi.tutorial.threading" title="Threads">Threads</a> 978</h3></div></div></div> 979<p> 980 There are an increasing number of hybrid parallel applications that mix distributed 981 and shared memory parallelism. To know how to support that model, one need 982 to know what level of threading support is guaranteed by the MPI implementation. 983 There are 4 ordered level of possible threading support described by <code class="computeroutput"><a class="link" href="../boost/mpi/threading/level.html" title="Type level">mpi::threading::level</a></code>. At the 984 lowest level, you should not use threads at all, at the highest level, any 985 thread can perform MPI call. 986 </p> 987<p> 988 If you want to use multi-threading in your MPI application, you should indicate 989 in the environment constructor your preferred threading support. Then probe 990 the one the library did provide, and decide what you can do with it (it could 991 be nothing, then aborting is a valid option): 992 </p> 993<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 994<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> 995<span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> 996<span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span> 997<span class="keyword">namespace</span> <span class="identifier">mt</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">threading</span><span class="special">;</span> 998 999<span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span> 1000<span class="special">{</span> 1001 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">);</span> 1002 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">env</span><span class="special">.</span><span class="identifier">thread_level</span><span class="special">()</span> <span class="special"><</span> <span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">)</span> <span class="special">{</span> 1003 <span class="identifier">env</span><span class="special">.</span><span class="identifier">abort</span><span class="special">(-</span><span class="number">1</span><span class="special">);</span> 1004 <span class="special">}</span> 1005 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span> 1006 <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"I am process "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special"><<</span> <span class="string">" of "</span> <span class="special"><<</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> 1007 <span class="special"><<</span> <span class="string">"."</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span> 1008 <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span> 1009<span class="special">}</span> 1010</pre> 1011</div> 1012<div class="section"> 1013<div class="titlepage"><div><div><h3 class="title"> 1014<a name="mpi.tutorial.skeleton_and_content"></a><a class="link" href="tutorial.html#mpi.tutorial.skeleton_and_content" title="Separating structure from content">Separating structure 1015 from content</a> 1016</h3></div></div></div> 1017<p> 1018 When communicating data types over MPI that are not fundamental to MPI (such 1019 as strings, lists, and user-defined data types), Boost.MPI must first serialize 1020 these data types into a buffer and then communicate them; the receiver then 1021 copies the results into a buffer before deserializing into an object on the 1022 other end. For some data types, this overhead can be eliminated by using 1023 <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>. 1024 However, variable-length data types such as strings and lists cannot be MPI 1025 data types. 1026 </p> 1027<p> 1028 Boost.MPI supports a second technique for improving performance by separating 1029 the structure of these variable-length data structures from the content stored 1030 in the data structures. This feature is only beneficial when the shape of 1031 the data structure remains the same but the content of the data structure 1032 will need to be communicated several times. For instance, in a finite element 1033 analysis the structure of the mesh may be fixed at the beginning of computation 1034 but the various variables on the cells of the mesh (temperature, stress, 1035 etc.) will be communicated many times within the iterative analysis process. 1036 In this case, Boost.MPI allows one to first send the "skeleton" 1037 of the mesh once, then transmit the "content" multiple times. Since 1038 the content need not contain any information about the structure of the data 1039 type, it can be transmitted without creating separate communication buffers. 1040 </p> 1041<p> 1042 To illustrate the use of skeletons and content, we will take a somewhat more 1043 limited example wherein a master process generates random number sequences 1044 into a list and transmits them to several slave processes. The length of 1045 the list will be fixed at program startup, so the content of the list (i.e., 1046 the current sequence of numbers) can be transmitted efficiently. The complete 1047 example is available in <code class="computeroutput"><span class="identifier">example</span><span class="special">/</span><span class="identifier">random_content</span><span class="special">.</span><span class="identifier">cpp</span></code>. We 1048 being with the master process (rank 0), which builds a list, communicates 1049 its structure via a <code class="computeroutput"><a class="link" href="reference.html#boost.mpi.skeleton">skeleton</a></code>, then repeatedly 1050 generates random number sequences to be broadcast to the slave processes 1051 via <code class="computeroutput">content</code>: 1052 </p> 1053<pre class="programlisting"><span class="comment">// Generate the list and broadcast its structure</span> 1054<span class="identifier">std</span><span class="special">::</span><span class="identifier">list</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">l</span><span class="special">(</span><span class="identifier">list_len</span><span class="special">);</span> 1055<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">l</span><span class="special">),</span> <span class="number">0</span><span class="special">);</span> 1056 1057<span class="comment">// Generate content several times and broadcast out that content</span> 1058<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">content</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">l</span><span class="special">);</span> 1059<span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">i</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">i</span> <span class="special"><</span> <span class="identifier">iterations</span><span class="special">;</span> <span class="special">++</span><span class="identifier">i</span><span class="special">)</span> <span class="special">{</span> 1060 <span class="comment">// Generate new random values</span> 1061 <span class="identifier">std</span><span class="special">::</span><span class="identifier">generate</span><span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="special">&</span><span class="identifier">random</span><span class="special">);</span> 1062 1063 <span class="comment">// Broadcast the new content of l</span> 1064 <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 1065<span class="special">}</span> 1066 1067<span class="comment">// Notify the slaves that we're done by sending all zeroes</span> 1068<span class="identifier">std</span><span class="special">::</span><span class="identifier">fill</span><span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="number">0</span><span class="special">);</span> 1069<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 1070</pre> 1071<p> 1072 The slave processes have a very similar structure to the master. They receive 1073 (via the <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast()</a></code> call) the skeleton of the 1074 data structure, then use it to build their own lists of integers. In each 1075 iteration, they receive via another <code class="computeroutput"><span class="identifier">broadcast</span><span class="special">()</span></code> the new content in the data structure and 1076 compute some property of the data: 1077 </p> 1078<pre class="programlisting"><span class="comment">// Receive the content and build up our own list</span> 1079<span class="identifier">std</span><span class="special">::</span><span class="identifier">list</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="identifier">l</span><span class="special">;</span> 1080<span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">l</span><span class="special">),</span> <span class="number">0</span><span class="special">);</span> 1081 1082<span class="identifier">mpi</span><span class="special">::</span><span class="identifier">content</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">l</span><span class="special">);</span> 1083<span class="keyword">int</span> <span class="identifier">i</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> 1084<span class="keyword">do</span> <span class="special">{</span> 1085 <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span> 1086 1087 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">find_if</span> 1088 <span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> 1089 <span class="identifier">std</span><span class="special">::</span><span class="identifier">bind1st</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">not_equal_to</span><span class="special"><</span><span class="keyword">int</span><span class="special">>(),</span> <span class="number">0</span><span class="special">))</span> <span class="special">==</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">())</span> 1090 <span class="keyword">break</span><span class="special">;</span> 1091 1092 <span class="comment">// Compute some property of the data.</span> 1093 1094 <span class="special">++</span><span class="identifier">i</span><span class="special">;</span> 1095<span class="special">}</span> <span class="keyword">while</span> <span class="special">(</span><span class="keyword">true</span><span class="special">);</span> 1096</pre> 1097<p> 1098 The skeletons and content of any Serializable data type can be transmitted 1099 either via the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">send</a></code> and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">recv</a></code> members of the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">communicator</a></code> 1100 class (for point-to-point communicators) or broadcast via the <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast()</a></code> collective. When separating 1101 a data structure into a skeleton and content, be careful not to modify the 1102 data structure (either on the sender side or the receiver side) without transmitting 1103 the skeleton again. Boost.MPI can not detect these accidental modifications 1104 to the data structure, which will likely result in incorrect data being transmitted 1105 or unstable programs. 1106 </p> 1107</div> 1108<div class="section"> 1109<div class="titlepage"><div><div><h3 class="title"> 1110<a name="mpi.tutorial.performance_optimizations"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations" title="Performance optimizations">Performance optimizations</a> 1111</h3></div></div></div> 1112<div class="toc"><dl class="toc"> 1113<dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations.serialization_optimizations">Serialization 1114 optimizations</a></span></dt> 1115<dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations.homogeneous_machines">Homogeneous 1116 Machines</a></span></dt> 1117</dl></div> 1118<div class="section"> 1119<div class="titlepage"><div><div><h4 class="title"> 1120<a name="mpi.tutorial.performance_optimizations.serialization_optimizations"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations.serialization_optimizations" title="Serialization optimizations">Serialization 1121 optimizations</a> 1122</h4></div></div></div> 1123<p> 1124 To obtain optimal performance for small fixed-length data types not containing 1125 any pointers it is very important to mark them using the type traits of 1126 Boost.MPI and Boost.Serialization. 1127 </p> 1128<p> 1129 It was already discussed that fixed length types containing no pointers 1130 can be using as <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>, e.g.: 1131 </p> 1132<pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span> 1133 <span class="keyword">template</span> <span class="special"><></span> 1134 <span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special"><</span><span class="identifier">gps_position</span><span class="special">></span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span> 1135<span class="special">}</span> <span class="special">}</span> 1136</pre> 1137<p> 1138 or the equivalent macro 1139 </p> 1140<pre class="programlisting"><span class="identifier">BOOST_IS_MPI_DATATYPE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span> 1141</pre> 1142<p> 1143 In addition it can give a substantial performance gain to turn off tracking 1144 and versioning for these types, if no pointers to these types are used, 1145 by using the traits classes or helper macros of Boost.Serialization: 1146 </p> 1147<pre class="programlisting"><span class="identifier">BOOST_CLASS_TRACKING</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">,</span><span class="identifier">track_never</span><span class="special">)</span> 1148<span class="identifier">BOOST_CLASS_IMPLEMENTATION</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">,</span><span class="identifier">object_serializable</span><span class="special">)</span> 1149</pre> 1150</div> 1151<div class="section"> 1152<div class="titlepage"><div><div><h4 class="title"> 1153<a name="mpi.tutorial.performance_optimizations.homogeneous_machines"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations.homogeneous_machines" title="Homogeneous Machines">Homogeneous 1154 Machines</a> 1155</h4></div></div></div> 1156<p> 1157 More optimizations are possible on homogeneous machines, by avoiding MPI_Pack/MPI_Unpack 1158 calls but using direct bitwise copy. This feature is enabled by default 1159 by defining the macro <code class="computeroutput"><a class="link" href="../BOOST_MPI_HOMOGENEOUS.html" title="Macro BOOST_MPI_HOMOGENEOUS">BOOST_MPI_HOMOGENEOUS</a></code> 1160 in the include file <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">config</span><span class="special">.</span><span class="identifier">hpp</span></code>. 1161 That definition must be consistent when building Boost.MPI and when building 1162 the application. 1163 </p> 1164<p> 1165 In addition all classes need to be marked both as is_mpi_datatype and as 1166 is_bitwise_serializable, by using the helper macro of Boost.Serialization: 1167 </p> 1168<pre class="programlisting"><span class="identifier">BOOST_IS_BITWISE_SERIALIZABLE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span> 1169</pre> 1170<p> 1171 Usually it is safe to serialize a class for which is_mpi_datatype is true 1172 by using binary copy of the bits. The exception are classes for which some 1173 members should be skipped for serialization. 1174 </p> 1175</div> 1176</div> 1177<div class="footnotes"> 1178<br><hr style="width:100; text-align:left;margin-left: 0"> 1179<div id="ftn.mpi.tutorial.f0" class="footnote"><p><a href="#mpi.tutorial.f0" class="para"><sup class="para">[11] </sup></a> 1180 According to the MPI standard, initialization must take place at user's 1181 initiative after once the main function has been called. 1182 </p></div> 1183</div> 1184</div> 1185<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> 1186<td align="left"></td> 1187<td align="right"><div class="copyright-footer">Copyright © 2005-2007 Douglas Gregor, 1188 Matthias Troyer, Trustees of Indiana University<p> 1189 Distributed under the Boost Software License, Version 1.0. (See accompanying 1190 file LICENSE_1_0.txt or copy at <ulink url="http://www.boost.org/LICENSE_1_0.txt"> 1191 http://www.boost.org/LICENSE_1_0.txt </ulink>) 1192 </p> 1193</div></td> 1194</tr></table> 1195<hr> 1196<div class="spirit-nav"> 1197<a accesskey="p" href="getting_started.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="c_mapping.html"><img src="../../../doc/src/images/next.png" alt="Next"></a> 1198</div> 1199</body> 1200</html> 1201
