1<?xml version="1.0"?> <!-- -*- sgml -*- --> 2<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 3 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> 4 5 6<chapter id="mc-manual" xreflabel="Memcheck: a memory error detector"> 7<title>Memcheck: a memory error detector</title> 8 9<para>To use this tool, you may specify <option>--tool=memcheck</option> 10on the Valgrind command line. You don't have to, though, since Memcheck 11is the default tool.</para> 12 13 14<sect1 id="mc-manual.overview" xreflabel="Overview"> 15<title>Overview</title> 16 17<para>Memcheck is a memory error detector. It can detect the following 18problems that are common in C and C++ programs.</para> 19 20<itemizedlist> 21 <listitem> 22 <para>Accessing memory you shouldn't, e.g. overrunning and underrunning 23 heap blocks, overrunning the top of the stack, and accessing memory after 24 it has been freed.</para> 25 </listitem> 26 27 <listitem> 28 <para>Using undefined values, i.e. values that have not been initialised, 29 or that have been derived from other undefined values.</para> 30 </listitem> 31 32 <listitem> 33 <para>Incorrect freeing of heap memory, such as double-freeing heap 34 blocks, or mismatched use of 35 <function>malloc</function>/<computeroutput>new</computeroutput>/<computeroutput>new[]</computeroutput> 36 versus 37 <function>free</function>/<computeroutput>delete</computeroutput>/<computeroutput>delete[]</computeroutput></para> 38 </listitem> 39 40 <listitem> 41 <para>Overlapping <computeroutput>src</computeroutput> and 42 <computeroutput>dst</computeroutput> pointers in 43 <computeroutput>memcpy</computeroutput> and related 44 functions.</para> 45 </listitem> 46 47 <listitem> 48 <para>Memory leaks.</para> 49 </listitem> 50</itemizedlist> 51 52<para>Problems like these can be difficult to find by other means, 53often remaining undetected for long periods, then causing occasional, 54difficult-to-diagnose crashes.</para> 55 56</sect1> 57 58 59 60<sect1 id="mc-manual.errormsgs" 61 xreflabel="Explanation of error messages from Memcheck"> 62<title>Explanation of error messages from Memcheck</title> 63 64<para>Memcheck issues a range of error messages. This section presents a 65quick summary of what error messages mean. The precise behaviour of the 66error-checking machinery is described in <xref 67linkend="mc-manual.machine"/>.</para> 68 69 70<sect2 id="mc-manual.badrw" 71 xreflabel="Illegal read / Illegal write errors"> 72<title>Illegal read / Illegal write errors</title> 73 74<para>For example:</para> 75<programlisting><![CDATA[ 76Invalid read of size 4 77 at 0x40F6BBCC: (within /usr/lib/libpng.so.2.1.0.9) 78 by 0x40F6B804: (within /usr/lib/libpng.so.2.1.0.9) 79 by 0x40B07FF4: read_png_image(QImageIO *) (kernel/qpngio.cpp:326) 80 by 0x40AC751B: QImageIO::read() (kernel/qimage.cpp:3621) 81 Address 0xBFFFF0E0 is not stack'd, malloc'd or free'd 82]]></programlisting> 83 84<para>This happens when your program reads or writes memory at a place 85which Memcheck reckons it shouldn't. In this example, the program did a 864-byte read at address 0xBFFFF0E0, somewhere within the system-supplied 87library libpng.so.2.1.0.9, which was called from somewhere else in the 88same library, called from line 326 of <filename>qpngio.cpp</filename>, 89and so on.</para> 90 91<para>Memcheck tries to establish what the illegal address might relate 92to, since that's often useful. So, if it points into a block of memory 93which has already been freed, you'll be informed of this, and also where 94the block was freed. Likewise, if it should turn out to be just off 95the end of a heap block, a common result of off-by-one-errors in 96array subscripting, you'll be informed of this fact, and also where the 97block was allocated. If you use the <option><xref 98linkend="opt.read-var-info"/></option> option Memcheck will run more slowly 99but may give a more detailed description of any illegal address.</para> 100 101<para>In this example, Memcheck can't identify the address. Actually 102the address is on the stack, but, for some reason, this is not a valid 103stack address -- it is below the stack pointer and that isn't allowed. 104In this particular case it's probably caused by GCC generating invalid 105code, a known bug in some ancient versions of GCC.</para> 106 107<para>Note that Memcheck only tells you that your program is about to 108access memory at an illegal address. It can't stop the access from 109happening. So, if your program makes an access which normally would 110result in a segmentation fault, you program will still suffer the same 111fate -- but you will get a message from Memcheck immediately prior to 112this. In this particular example, reading junk on the stack is 113non-fatal, and the program stays alive.</para> 114 115</sect2> 116 117 118 119<sect2 id="mc-manual.uninitvals" 120 xreflabel="Use of uninitialised values"> 121<title>Use of uninitialised values</title> 122 123<para>For example:</para> 124<programlisting><![CDATA[ 125Conditional jump or move depends on uninitialised value(s) 126 at 0x402DFA94: _IO_vfprintf (_itoa.h:49) 127 by 0x402E8476: _IO_printf (printf.c:36) 128 by 0x8048472: main (tests/manuel1.c:8) 129]]></programlisting> 130 131<para>An uninitialised-value use error is reported when your program 132uses a value which hasn't been initialised -- in other words, is 133undefined. Here, the undefined value is used somewhere inside the 134<function>printf</function> machinery of the C library. This error was 135reported when running the following small program:</para> 136<programlisting><![CDATA[ 137int main() 138{ 139 int x; 140 printf ("x = %d\n", x); 141}]]></programlisting> 142 143<para>It is important to understand that your program can copy around 144junk (uninitialised) data as much as it likes. Memcheck observes this 145and keeps track of the data, but does not complain. A complaint is 146issued only when your program attempts to make use of uninitialised 147data in a way that might affect your program's externally-visible behaviour. 148In this example, <varname>x</varname> is uninitialised. Memcheck observes 149the value being passed to <function>_IO_printf</function> and thence to 150<function>_IO_vfprintf</function>, but makes no comment. However, 151<function>_IO_vfprintf</function> has to examine the value of 152<varname>x</varname> so it can turn it into the corresponding ASCII string, 153and it is at this point that Memcheck complains.</para> 154 155<para>Sources of uninitialised data tend to be:</para> 156<itemizedlist> 157 <listitem> 158 <para>Local variables in procedures which have not been initialised, 159 as in the example above.</para> 160 </listitem> 161 <listitem> 162 <para>The contents of heap blocks (allocated with 163 <function>malloc</function>, <function>new</function>, or a similar 164 function) before you (or a constructor) write something there. 165 </para> 166 </listitem> 167</itemizedlist> 168 169<para>To see information on the sources of uninitialised data in your 170program, use the <option>--track-origins=yes</option> option. This 171makes Memcheck run more slowly, but can make it much easier to track down 172the root causes of uninitialised value errors.</para> 173 174</sect2> 175 176 177 178<sect2 id="mc-manual.bad-syscall-args" 179 xreflabel="Use of uninitialised or unaddressable values in system 180 calls"> 181<title>Use of uninitialised or unaddressable values in system 182 calls</title> 183 184<para>Memcheck checks all parameters to system calls: 185<itemizedlist> 186 <listitem> 187 <para>It checks all the direct parameters themselves, whether they are 188 initialised.</para> 189 </listitem> 190 <listitem> 191 <para>Also, if a system call needs to read from a buffer provided by 192 your program, Memcheck checks that the entire buffer is addressable 193 and its contents are initialised.</para> 194 </listitem> 195 <listitem> 196 <para>Also, if the system call needs to write to a user-supplied 197 buffer, Memcheck checks that the buffer is addressable.</para> 198 </listitem> 199</itemizedlist> 200</para> 201 202<para>After the system call, Memcheck updates its tracked information to 203precisely reflect any changes in memory state caused by the system 204call.</para> 205 206<para>Here's an example of two system calls with invalid parameters:</para> 207<programlisting><![CDATA[ 208 #include <stdlib.h> 209 #include <unistd.h> 210 int main( void ) 211 { 212 char* arr = malloc(10); 213 int* arr2 = malloc(sizeof(int)); 214 write( 1 /* stdout */, arr, 10 ); 215 exit(arr2[0]); 216 } 217]]></programlisting> 218 219<para>You get these complaints ...</para> 220<programlisting><![CDATA[ 221 Syscall param write(buf) points to uninitialised byte(s) 222 at 0x25A48723: __write_nocancel (in /lib/tls/libc-2.3.3.so) 223 by 0x259AFAD3: __libc_start_main (in /lib/tls/libc-2.3.3.so) 224 by 0x8048348: (within /auto/homes/njn25/grind/head4/a.out) 225 Address 0x25AB8028 is 0 bytes inside a block of size 10 alloc'd 226 at 0x259852B0: malloc (vg_replace_malloc.c:130) 227 by 0x80483F1: main (a.c:5) 228 229 Syscall param exit(error_code) contains uninitialised byte(s) 230 at 0x25A21B44: __GI__exit (in /lib/tls/libc-2.3.3.so) 231 by 0x8048426: main (a.c:8) 232]]></programlisting> 233 234<para>... because the program has (a) written uninitialised junk 235from the heap block to the standard output, and (b) passed an 236uninitialised value to <function>exit</function>. Note that the first 237error refers to the memory pointed to by 238<computeroutput>buf</computeroutput> (not 239<computeroutput>buf</computeroutput> itself), but the second error 240refers directly to <computeroutput>exit</computeroutput>'s argument 241<computeroutput>arr2[0]</computeroutput>.</para> 242 243</sect2> 244 245 246<sect2 id="mc-manual.badfrees" xreflabel="Illegal frees"> 247<title>Illegal frees</title> 248 249<para>For example:</para> 250<programlisting><![CDATA[ 251Invalid free() 252 at 0x4004FFDF: free (vg_clientmalloc.c:577) 253 by 0x80484C7: main (tests/doublefree.c:10) 254 Address 0x3807F7B4 is 0 bytes inside a block of size 177 free'd 255 at 0x4004FFDF: free (vg_clientmalloc.c:577) 256 by 0x80484C7: main (tests/doublefree.c:10) 257]]></programlisting> 258 259<para>Memcheck keeps track of the blocks allocated by your program 260with <function>malloc</function>/<computeroutput>new</computeroutput>, 261so it can know exactly whether or not the argument to 262<function>free</function>/<computeroutput>delete</computeroutput> is 263legitimate or not. Here, this test program has freed the same block 264twice. As with the illegal read/write errors, Memcheck attempts to 265make sense of the address freed. If, as here, the address is one 266which has previously been freed, you wil be told that -- making 267duplicate frees of the same block easy to spot. You will also get this 268message if you try to free a pointer that doesn't point to the start of a 269heap block.</para> 270 271</sect2> 272 273 274<sect2 id="mc-manual.rudefn" 275 xreflabel="When a heap block is freed with an inappropriate deallocation 276function"> 277<title>When a heap block is freed with an inappropriate deallocation 278function</title> 279 280<para>In the following example, a block allocated with 281<function>new[]</function> has wrongly been deallocated with 282<function>free</function>:</para> 283<programlisting><![CDATA[ 284Mismatched free() / delete / delete [] 285 at 0x40043249: free (vg_clientfuncs.c:171) 286 by 0x4102BB4E: QGArray::~QGArray(void) (tools/qgarray.cpp:149) 287 by 0x4C261C41: PptDoc::~PptDoc(void) (include/qmemarray.h:60) 288 by 0x4C261F0E: PptXml::~PptXml(void) (pptxml.cc:44) 289 Address 0x4BB292A8 is 0 bytes inside a block of size 64 alloc'd 290 at 0x4004318C: operator new[](unsigned int) (vg_clientfuncs.c:152) 291 by 0x4C21BC15: KLaola::readSBStream(int) const (klaola.cc:314) 292 by 0x4C21C155: KLaola::stream(KLaola::OLENode const *) (klaola.cc:416) 293 by 0x4C21788F: OLEFilter::convert(QCString const &) (olefilter.cc:272) 294]]></programlisting> 295 296<para>In <literal>C++</literal> it's important to deallocate memory in a 297way compatible with how it was allocated. The deal is:</para> 298<itemizedlist> 299 <listitem> 300 <para>If allocated with 301 <function>malloc</function>, 302 <function>calloc</function>, 303 <function>realloc</function>, 304 <function>valloc</function> or 305 <function>memalign</function>, you must 306 deallocate with <function>free</function>.</para> 307 </listitem> 308 <listitem> 309 <para>If allocated with <function>new</function>, you must deallocate 310 with <function>delete</function>.</para> 311 </listitem> 312 <listitem> 313 <para>If allocated with <function>new[]</function>, you must 314 deallocate with <function>delete[]</function>.</para> 315 </listitem> 316</itemizedlist> 317 318<para>The worst thing is that on Linux apparently it doesn't matter if 319you do mix these up, but the same program may then crash on a 320different platform, Solaris for example. So it's best to fix it 321properly. According to the KDE folks "it's amazing how many C++ 322programmers don't know this".</para> 323 324<para>The reason behind the requirement is as follows. In some C++ 325implementations, <function>delete[]</function> must be used for 326objects allocated by <function>new[]</function> because the compiler 327stores the size of the array and the pointer-to-member to the 328destructor of the array's content just before the pointer actually 329returned. <function>delete</function> doesn't account for this and will get 330confused, possibly corrupting the heap.</para> 331 332</sect2> 333 334 335 336<sect2 id="mc-manual.overlap" 337 xreflabel="Overlapping source and destination blocks"> 338<title>Overlapping source and destination blocks</title> 339 340<para>The following C library functions copy some data from one 341memory block to another (or something similar): 342<function>memcpy</function>, 343<function>strcpy</function>, 344<function>strncpy</function>, 345<function>strcat</function>, 346<function>strncat</function>. 347The blocks pointed to by their <computeroutput>src</computeroutput> and 348<computeroutput>dst</computeroutput> pointers aren't allowed to overlap. 349The POSIX standards have wording along the lines "If copying takes place 350between objects that overlap, the behavior is undefined." Therefore, 351Memcheck checks for this. 352</para> 353 354<para>For example:</para> 355<programlisting><![CDATA[ 356==27492== Source and destination overlap in memcpy(0xbffff294, 0xbffff280, 21) 357==27492== at 0x40026CDC: memcpy (mc_replace_strmem.c:71) 358==27492== by 0x804865A: main (overlap.c:40) 359]]></programlisting> 360 361<para>You don't want the two blocks to overlap because one of them could 362get partially overwritten by the copying.</para> 363 364<para>You might think that Memcheck is being overly pedantic reporting 365this in the case where <computeroutput>dst</computeroutput> is less than 366<computeroutput>src</computeroutput>. For example, the obvious way to 367implement <function>memcpy</function> is by copying from the first 368byte to the last. However, the optimisation guides of some 369architectures recommend copying from the last byte down to the first. 370Also, some implementations of <function>memcpy</function> zero 371<computeroutput>dst</computeroutput> before copying, because zeroing the 372destination's cache line(s) can improve performance.</para> 373 374<para>The moral of the story is: if you want to write truly portable 375code, don't make any assumptions about the language 376implementation.</para> 377 378</sect2> 379 380 381<sect2 id="mc-manual.leaks" xreflabel="Memory leak detection"> 382<title>Memory leak detection</title> 383 384<para>Memcheck keeps track of all heap blocks issued in response to 385calls to 386<function>malloc</function>/<function>new</function> et al. 387So when the program exits, it knows which blocks have not been freed. 388</para> 389 390<para>If <option>--leak-check</option> is set appropriately, for each 391remaining block, Memcheck determines if the block is reachable from pointers 392within the root-set. The root-set consists of (a) general purpose registers 393of all threads, and (b) initialised, aligned, pointer-sized data words in 394accessible client memory, including stacks.</para> 395 396<para>There are two ways a block can be reached. The first is with a 397"start-pointer", i.e. a pointer to the start of the block. The second is with 398an "interior-pointer", i.e. a pointer to the middle of the block. There are 399several ways we know of that an interior-pointer can occur:</para> 400 401<itemizedlist> 402 <listitem> 403 <para>The pointer might have originally been a start-pointer and have been 404 moved along deliberately (or not deliberately) by the program. In 405 particular, this can happen if your program uses tagged pointers, i.e. 406 if it uses the bottom one, two or three bits of a pointer, which are 407 normally always zero due to alignment, in order to store extra 408 information.</para> 409 </listitem> 410 411 <listitem> 412 <para>It might be a random junk value in memory, entirely unrelated, just 413 a coincidence.</para> 414 </listitem> 415 416 <listitem> 417 <para>It might be a pointer to an array of C++ objects (which possess 418 destructors) allocated with <computeroutput>new[]</computeroutput>. In 419 this case, some compilers store a "magic cookie" containing the array 420 length at the start of the allocated block, and return a pointer to just 421 past that magic cookie, i.e. an interior-pointer. 422 See <ulink url="http://theory.uwinnipeg.ca/gnu/gcc/gxxint_14.html">this 423 page</ulink> for more information.</para> 424 </listitem> 425 426 <listitem> 427 <para>It might be a pointer to the inner char array of a C++ 428 <computeroutput>std::string</computeroutput>. For example, some 429 compilers add 3 words at the beginning of the std::string to 430 store the length, the capacity and a reference count before the 431 memory containing the array of characters. They return a pointer 432 just after these 3 words, pointing at the char array.</para> 433 </listitem> 434 435 <listitem> 436 <para>It might be a pointer to an inner part of a C++ object using 437 multiple inheritance. </para> 438 </listitem> 439</itemizedlist> 440 441<para>You can optionally activate heuristics to use during the leak 442search to detect the interior pointers corresponding to 443the <computeroutput>newarray</computeroutput>, 444<computeroutput>stdstring</computeroutput> and 445<computeroutput>multipleinheritance</computeroutput> cases. If the 446heuristic detects that an interior pointer corresponds to such a case, 447the block will be considered as reachable by the interior 448pointer. In other words, the interior pointer will be treated 449as if it were a start pointer.</para> 450 451 452<para>With that in mind, consider the nine possible cases described by the 453following figure.</para> 454 455<programlisting><![CDATA[ 456 Pointer chain AAA Leak Case BBB Leak Case 457 ------------- ------------- ------------- 458(1) RRR ------------> BBB DR 459(2) RRR ---> AAA ---> BBB DR IR 460(3) RRR BBB DL 461(4) RRR AAA ---> BBB DL IL 462(5) RRR ------?-----> BBB (y)DR, (n)DL 463(6) RRR ---> AAA -?-> BBB DR (y)IR, (n)DL 464(7) RRR -?-> AAA ---> BBB (y)DR, (n)DL (y)IR, (n)IL 465(8) RRR -?-> AAA -?-> BBB (y)DR, (n)DL (y,y)IR, (n,y)IL, (_,n)DL 466(9) RRR AAA -?-> BBB DL (y)IL, (n)DL 467 468Pointer chain legend: 469- RRR: a root set node or DR block 470- AAA, BBB: heap blocks 471- --->: a start-pointer 472- -?->: an interior-pointer 473 474Leak Case legend: 475- DR: Directly reachable 476- IR: Indirectly reachable 477- DL: Directly lost 478- IL: Indirectly lost 479- (y)XY: it's XY if the interior-pointer is a real pointer 480- (n)XY: it's XY if the interior-pointer is not a real pointer 481- (_)XY: it's XY in either case 482]]></programlisting> 483 484<para>Every possible case can be reduced to one of the above nine. Memcheck 485merges some of these cases in its output, resulting in the following four 486leak kinds.</para> 487 488 489<itemizedlist> 490 491 <listitem> 492 <para>"Still reachable". This covers cases 1 and 2 (for the BBB blocks) 493 above. A start-pointer or chain of start-pointers to the block is 494 found. Since the block is still pointed at, the programmer could, at 495 least in principle, have freed it before program exit. "Still reachable" 496 blocks are very common and arguably not a problem. So, by default, 497 Memcheck won't report such blocks individually.</para> 498 </listitem> 499 500 <listitem> 501 <para>"Definitely lost". This covers case 3 (for the BBB blocks) above. 502 This means that no pointer to the block can be found. The block is 503 classified as "lost", because the programmer could not possibly have 504 freed it at program exit, since no pointer to it exists. This is likely 505 a symptom of having lost the pointer at some earlier point in the 506 program. Such cases should be fixed by the programmer.</para> 507 </listitem> 508 509 <listitem> 510 <para>"Indirectly lost". This covers cases 4 and 9 (for the BBB blocks) 511 above. This means that the block is lost, not because there are no 512 pointers to it, but rather because all the blocks that point to it are 513 themselves lost. For example, if you have a binary tree and the root 514 node is lost, all its children nodes will be indirectly lost. Because 515 the problem will disappear if the definitely lost block that caused the 516 indirect leak is fixed, Memcheck won't report such blocks individually 517 by default.</para> 518 </listitem> 519 520 <listitem> 521 <para>"Possibly lost". This covers cases 5--8 (for the BBB blocks) 522 above. This means that a chain of one or more pointers to the block has 523 been found, but at least one of the pointers is an interior-pointer. 524 This could just be a random value in memory that happens to point into a 525 block, and so you shouldn't consider this ok unless you know you have 526 interior-pointers.</para> 527 </listitem> 528 529</itemizedlist> 530 531<para>(Note: This mapping of the nine possible cases onto four leak kinds is 532not necessarily the best way that leaks could be reported; in particular, 533interior-pointers are treated inconsistently. It is possible the 534categorisation may be improved in the future.)</para> 535 536<para>Furthermore, if suppressions exists for a block, it will be reported 537as "suppressed" no matter what which of the above four kinds it belongs 538to.</para> 539 540 541<para>The following is an example leak summary.</para> 542 543<programlisting><![CDATA[ 544LEAK SUMMARY: 545 definitely lost: 48 bytes in 3 blocks. 546 indirectly lost: 32 bytes in 2 blocks. 547 possibly lost: 96 bytes in 6 blocks. 548 still reachable: 64 bytes in 4 blocks. 549 suppressed: 0 bytes in 0 blocks. 550]]></programlisting> 551 552<para>If heuristics have been used to consider some blocks as 553reachable, the leak summary details the heuristically reachable subset 554of 'still reachable:' per heuristic. In the below example, of the 79 555bytes still reachable, 71 bytes (56+7+8) have been considered 556heuristically reachable. 557</para> 558 559<programlisting><![CDATA[ 560LEAK SUMMARY: 561 definitely lost: 4 bytes in 1 blocks 562 indirectly lost: 0 bytes in 0 blocks 563 possibly lost: 0 bytes in 0 blocks 564 still reachable: 79 bytes in 5 blocks 565 of which reachable via heuristic: 566 stdstring : 56 bytes in 2 blocks 567 newarray : 7 bytes in 1 blocks 568 multipleinheritance: 8 bytes in 1 blocks 569 suppressed: 0 bytes in 0 blocks 570]]></programlisting> 571 572<para>If <option>--leak-check=full</option> is specified, 573Memcheck will give details for each definitely lost or possibly lost block, 574including where it was allocated. (Actually, it merges results for all 575blocks that have the same leak kind and sufficiently similar stack traces 576into a single "loss record". The 577<option>--leak-resolution</option> lets you control the 578meaning of "sufficiently similar".) It cannot tell you when or how or why 579the pointer to a leaked block was lost; you have to work that out for 580yourself. In general, you should attempt to ensure your programs do not 581have any definitely lost or possibly lost blocks at exit.</para> 582 583<para>For example:</para> 584<programlisting><![CDATA[ 5858 bytes in 1 blocks are definitely lost in loss record 1 of 14 586 at 0x........: malloc (vg_replace_malloc.c:...) 587 by 0x........: mk (leak-tree.c:11) 588 by 0x........: main (leak-tree.c:39) 589 59088 (8 direct, 80 indirect) bytes in 1 blocks are definitely lost in loss record 13 of 14 591 at 0x........: malloc (vg_replace_malloc.c:...) 592 by 0x........: mk (leak-tree.c:11) 593 by 0x........: main (leak-tree.c:25) 594]]></programlisting> 595 596<para>The first message describes a simple case of a single 8 byte block 597that has been definitely lost. The second case mentions another 8 byte 598block that has been definitely lost; the difference is that a further 80 599bytes in other blocks are indirectly lost because of this lost block. 600The loss records are not presented in any notable order, so the loss record 601numbers aren't particularly meaningful. The loss record numbers can be used 602in the Valgrind gdbserver to list the addresses of the leaked blocks and/or give 603more details about how a block is still reachable.</para> 604 605<para>The option <option>--show-leak-kinds=<set></option> 606controls the set of leak kinds to show 607when <option>--leak-check=full</option> is specified. </para> 608 609<para>The <option><set></option> of leak kinds is specified 610in one of the following ways: 611 612<itemizedlist> 613 <listitem><para>a comma separated list of one or more of 614 <option>definite indirect possible reachable</option>.</para> 615 </listitem> 616 617 <listitem><para><option>all</option> to specify the complete set (all leak kinds).</para> 618 </listitem> 619 620 <listitem><para><option>none</option> for the empty set.</para> 621 </listitem> 622</itemizedlist> 623 624</para> 625 626<para> The default value for the leak kinds to show is 627 <option>--show-leak-kinds=definite,possible</option>. 628</para> 629 630<para>To also show the reachable and indirectly lost blocks in 631addition to the definitely and possibly lost blocks, you can 632use <option>--show-leak-kinds=all</option>. To only show the 633reachable and indirectly lost blocks, use 634<option>--show-leak-kinds=indirect,reachable</option>. The reachable 635and indirectly lost blocks will then be presented as shown in 636the following two examples.</para> 637 638<programlisting><![CDATA[ 63964 bytes in 4 blocks are still reachable in loss record 2 of 4 640 at 0x........: malloc (vg_replace_malloc.c:177) 641 by 0x........: mk (leak-cases.c:52) 642 by 0x........: main (leak-cases.c:74) 643 64432 bytes in 2 blocks are indirectly lost in loss record 1 of 4 645 at 0x........: malloc (vg_replace_malloc.c:177) 646 by 0x........: mk (leak-cases.c:52) 647 by 0x........: main (leak-cases.c:80) 648]]></programlisting> 649 650<para>Because there are different kinds of leaks with different 651severities, an interesting question is: which leaks should be 652counted as true "errors" and which should not? 653</para> 654 655<para> The answer to this question affects the numbers printed in 656the <computeroutput>ERROR SUMMARY</computeroutput> line, and also the 657effect of the <option>--error-exitcode</option> option. First, a leak 658is only counted as a true "error" 659if <option>--leak-check=full</option> is specified. Then, the 660option <option>--errors-for-leak-kinds=<set></option> controls 661the set of leak kinds to consider as errors. The default value 662is <option>--errors-for-leak-kinds=definite,possible</option> 663</para> 664 665</sect2> 666 667</sect1> 668 669 670 671<sect1 id="mc-manual.options" 672 xreflabel="Memcheck Command-Line Options"> 673<title>Memcheck Command-Line Options</title> 674 675<!-- start of xi:include in the manpage --> 676<variablelist id="mc.opts.list"> 677 678 <varlistentry id="opt.leak-check" xreflabel="--leak-check"> 679 <term> 680 <option><![CDATA[--leak-check=<no|summary|yes|full> [default: summary] ]]></option> 681 </term> 682 <listitem> 683 <para>When enabled, search for memory leaks when the client 684 program finishes. If set to <varname>summary</varname>, it says how 685 many leaks occurred. If set to <varname>full</varname> or 686 <varname>yes</varname>, it also gives details of each individual 687 leak.</para> 688 </listitem> 689 </varlistentry> 690 691 <varlistentry id="opt.leak-resolution" xreflabel="--leak-resolution"> 692 <term> 693 <option><![CDATA[--leak-resolution=<low|med|high> [default: high] ]]></option> 694 </term> 695 <listitem> 696 <para>When doing leak checking, determines how willing 697 Memcheck is to consider different backtraces to 698 be the same for the purposes of merging multiple leaks into a single 699 leak report. When set to <varname>low</varname>, only the first 700 two entries need match. When <varname>med</varname>, four entries 701 have to match. When <varname>high</varname>, all entries need to 702 match.</para> 703 704 <para>For hardcore leak debugging, you probably want to use 705 <option>--leak-resolution=high</option> together with 706 <option>--num-callers=40</option> or some such large number. 707 </para> 708 709 <para>Note that the <option>--leak-resolution</option> setting 710 does not affect Memcheck's ability to find 711 leaks. It only changes how the results are presented.</para> 712 </listitem> 713 </varlistentry> 714 715 <varlistentry id="opt.show-leak-kinds" xreflabel="--show-leak-kinds"> 716 <term> 717 <option><![CDATA[--show-leak-kinds=<set> [default: definite,possible] ]]></option> 718 </term> 719 <listitem> 720 <para>Specifies the leak kinds to show in a full leak search, in 721 one of the following ways: </para> 722 723 <itemizedlist> 724 <listitem><para>a comma separated list of one or more of 725 <option>definite indirect possible reachable</option>.</para> 726 </listitem> 727 728 <listitem><para><option>all</option> to specify the complete set (all leak kinds). 729 It is equivalent to 730 <option>--show-leak-kinds=definite,indirect,possible,reachable</option>.</para> 731 </listitem> 732 733 <listitem><para><option>none</option> for the empty set.</para> 734 </listitem> 735 </itemizedlist> 736 </listitem> 737 </varlistentry> 738 739 740 <varlistentry id="opt.errors-for-leak-kinds" xreflabel="--errors-for-leak-kinds"> 741 <term> 742 <option><![CDATA[--errors-for-leak-kinds=<set> [default: definite,possible] ]]></option> 743 </term> 744 <listitem> 745 <para>Specifies the leak kinds to count as errors in a full leak search. The 746 <option><![CDATA[<set>]]></option> is specified similarly to 747 <option>--show-leak-kinds</option> 748 </para> 749 </listitem> 750 </varlistentry> 751 752 753 <varlistentry id="opt.leak-check-heuristics" xreflabel="--leak-check-heuristics"> 754 <term> 755 <option><![CDATA[--leak-check-heuristics=<set> [default: none] ]]></option> 756 </term> 757 <listitem> 758 <para>Specifies the set of leak check heuristics to be used 759 during leak searches. The heuristics control which interior pointers 760 to a block cause it to be considered as reachable. 761 The heuristic set is specified in one of the following ways:</para> 762 763 <itemizedlist> 764 <listitem><para>a comma separated list of one or more of 765 <option>stdstring newarray multipleinheritance</option>.</para> 766 </listitem> 767 768 <listitem><para><option>all</option> to activate the complete set of 769 heuristics. 770 It is equivalent to 771 <option>--leak-check-heuristics=stdstring,newarray,multipleinheritance</option>.</para> 772 </listitem> 773 774 <listitem><para><option>none</option> for the empty set.</para> 775 </listitem> 776 </itemizedlist> 777 </listitem> 778 779 <para>Note that these heuristics are dependent on the layout of the objects 780 produced by the C++ compiler. They have been tested with some gcc versions 781 (e.g. 4.4 and 4.7). They might not work properly with other C++ compilers. 782 </para> 783 </varlistentry> 784 785 786 <varlistentry id="opt.show-reachable" xreflabel="--show-reachable"> 787 <term> 788 <option><![CDATA[--show-reachable=<yes|no> ]]></option> 789 </term> 790 <term> 791 <option><![CDATA[--show-possibly-lost=<yes|no> ]]></option> 792 </term> 793 <listitem> 794 <para>These options provide an alternative way to specify the leak kinds to show: 795 </para> 796 <itemizedlist> 797 <listitem> 798 <para> 799 <option>--show-reachable=no --show-possibly-lost=yes</option> is equivalent to 800 <option>--show-leak-kinds=definite,possible</option>. 801 </para> 802 </listitem> 803 <listitem> 804 <para> 805 <option>--show-reachable=no --show-possibly-lost=no</option> is equivalent to 806 <option>--show-leak-kinds=definite</option>. 807 </para> 808 </listitem> 809 <listitem> 810 <para> 811 <option>--show-reachable=yes</option> is equivalent to 812 <option>--show-leak-kinds=all</option>. 813 </para> 814 </listitem> 815 </itemizedlist> 816 </listitem> 817 <para> Note that <option>--show-possibly-lost=no</option> has no effect 818 if <option>--show-reachable=yes</option> is specified.</para> 819 </varlistentry> 820 821 <varlistentry id="opt.undef-value-errors" xreflabel="--undef-value-errors"> 822 <term> 823 <option><![CDATA[--undef-value-errors=<yes|no> [default: yes] ]]></option> 824 </term> 825 <listitem> 826 <para>Controls whether Memcheck reports 827 uses of undefined value errors. Set this to 828 <varname>no</varname> if you don't want to see undefined value 829 errors. It also has the side effect of speeding up 830 Memcheck somewhat. 831 </para> 832 </listitem> 833 </varlistentry> 834 835 <varlistentry id="opt.track-origins" xreflabel="--track-origins"> 836 <term> 837 <option><![CDATA[--track-origins=<yes|no> [default: no] ]]></option> 838 </term> 839 <listitem> 840 <para>Controls whether Memcheck tracks 841 the origin of uninitialised values. By default, it does not, 842 which means that although it can tell you that an 843 uninitialised value is being used in a dangerous way, it 844 cannot tell you where the uninitialised value came from. This 845 often makes it difficult to track down the root problem. 846 </para> 847 <para>When set 848 to <varname>yes</varname>, Memcheck keeps 849 track of the origins of all uninitialised values. Then, when 850 an uninitialised value error is 851 reported, Memcheck will try to show the 852 origin of the value. An origin can be one of the following 853 four places: a heap block, a stack allocation, a client 854 request, or miscellaneous other sources (eg, a call 855 to <varname>brk</varname>). 856 </para> 857 <para>For uninitialised values originating from a heap 858 block, Memcheck shows where the block was 859 allocated. For uninitialised values originating from a stack 860 allocation, Memcheck can tell you which 861 function allocated the value, but no more than that -- typically 862 it shows you the source location of the opening brace of the 863 function. So you should carefully check that all of the 864 function's local variables are initialised properly. 865 </para> 866 <para>Performance overhead: origin tracking is expensive. It 867 halves Memcheck's speed and increases 868 memory use by a minimum of 100MB, and possibly more. 869 Nevertheless it can drastically reduce the effort required to 870 identify the root cause of uninitialised value errors, and so 871 is often a programmer productivity win, despite running 872 more slowly. 873 </para> 874 <para>Accuracy: Memcheck tracks origins 875 quite accurately. To avoid very large space and time 876 overheads, some approximations are made. It is possible, 877 although unlikely, that Memcheck will report an incorrect origin, or 878 not be able to identify any origin. 879 </para> 880 <para>Note that the combination 881 <option>--track-origins=yes</option> 882 and <option>--undef-value-errors=no</option> is 883 nonsensical. Memcheck checks for and 884 rejects this combination at startup. 885 </para> 886 </listitem> 887 </varlistentry> 888 889 <varlistentry id="opt.partial-loads-ok" xreflabel="--partial-loads-ok"> 890 <term> 891 <option><![CDATA[--partial-loads-ok=<yes|no> [default: no] ]]></option> 892 </term> 893 <listitem> 894 <para>Controls how Memcheck handles 32-, 64-, 128- and 256-bit 895 naturally aligned loads from addresses for which some bytes are 896 addressable and others are not. When <varname>yes</varname>, such 897 loads do not produce an address error. Instead, loaded bytes 898 originating from illegal addresses are marked as uninitialised, and 899 those corresponding to legal addresses are handled in the normal 900 way.</para> 901 902 <para>When <varname>no</varname>, loads from partially invalid 903 addresses are treated the same as loads from completely invalid 904 addresses: an illegal-address error is issued, and the resulting 905 bytes are marked as initialised.</para> 906 907 <para>Note that code that behaves in this way is in violation of 908 the ISO C/C++ standards, and should be considered broken. If 909 at all possible, such code should be fixed. This option should be 910 used only as a last resort.</para> 911 </listitem> 912 </varlistentry> 913 914 <varlistentry id="opt.keep-stacktraces" xreflabel="--keep-stacktraces"> 915 <term> 916 <option><![CDATA[--keep-stacktraces=alloc|free|alloc-and-free|alloc-then-free|none [default: alloc-then-free] ]]></option> 917 </term> 918 <listitem> 919 <para>Controls which stack trace(s) to keep for malloc'd and/or 920 free'd blocks. 921 </para> 922 923 <para>With <varname>alloc-then-free</varname>, a stack trace is 924 recorded at allocation time, and is associated with the block. 925 When the block is freed, a second stack trace is recorded, and 926 this replaces the allocation stack trace. As a result, any "use 927 after free" errors relating to this block can only show a stack 928 trace for where the block was freed. 929 </para> 930 931 <para>With <varname>alloc-and-free</varname>, both allocation 932 and the deallocation stack traces for the block are stored. 933 Hence a "use after free" error will 934 show both, which may make the error easier to diagnose. 935 Compared to <varname>alloc-then-free</varname>, this setting 936 slightly increases Valgrind's memory use as the block contains two 937 references instead of one. 938 </para> 939 940 <para>With <varname>alloc</varname>, only the allocation stack 941 trace is recorded (and reported). With <varname>free</varname>, 942 only the deallocation stack trace is recorded (and reported). 943 These values somewhat decrease Valgrind's memory and cpu usage. 944 They can be useful depending on the error types you are 945 searching for and the level of detail you need to analyse 946 them. For example, if you are only interested in memory leak 947 errors, it is sufficient to record the allocation stack traces. 948 </para> 949 950 <para>With <varname>none</varname>, no stack traces are recorded 951 for malloc and free operations. If your program allocates a lot 952 of blocks and/or allocates/frees from many different stack 953 traces, this can significantly decrease cpu and/or memory 954 required. Of course, few details will be reported for errors 955 related to heap blocks. 956 </para> 957 958 <para>Note that once a stack trace is recorded, Valgrind keeps 959 the stack trace in memory even if it is not referenced by any 960 block. Some programs (for example, recursive algorithms) can 961 generate a huge number of stack traces. If Valgrind uses too 962 much memory in such circumstances, you can reduce the memory 963 required with the options <varname>--keep-stacktraces</varname> 964 and/or by using a smaller value for the 965 option <varname>--num-callers</varname>. 966 </para> 967 </listitem> 968 </varlistentry> 969 970 <varlistentry id="opt.freelist-vol" xreflabel="--freelist-vol"> 971 <term> 972 <option><![CDATA[--freelist-vol=<number> [default: 20000000] ]]></option> 973 </term> 974 <listitem> 975 <para>When the client program releases memory using 976 <function>free</function> (in <literal>C</literal>) or 977 <computeroutput>delete</computeroutput> 978 (<literal>C++</literal>), that memory is not immediately made 979 available for re-allocation. Instead, it is marked inaccessible 980 and placed in a queue of freed blocks. The purpose is to defer as 981 long as possible the point at which freed-up memory comes back 982 into circulation. This increases the chance that 983 Memcheck will be able to detect invalid 984 accesses to blocks for some significant period of time after they 985 have been freed.</para> 986 987 <para>This option specifies the maximum total size, in bytes, of the 988 blocks in the queue. The default value is twenty million bytes. 989 Increasing this increases the total amount of memory used by 990 Memcheck but may detect invalid uses of freed 991 blocks which would otherwise go undetected.</para> 992 </listitem> 993 </varlistentry> 994 995 <varlistentry id="opt.freelist-big-blocks" xreflabel="--freelist-big-blocks"> 996 <term> 997 <option><![CDATA[--freelist-big-blocks=<number> [default: 1000000] ]]></option> 998 </term> 999 <listitem> 1000 <para>When making blocks from the queue of freed blocks available 1001 for re-allocation, Memcheck will in priority re-circulate the blocks 1002 with a size greater or equal to <option>--freelist-big-blocks</option>. 1003 This ensures that freeing big blocks (in particular freeing blocks bigger than 1004 <option>--freelist-vol</option>) does not immediately lead to a re-circulation 1005 of all (or a lot of) the small blocks in the free list. In other words, 1006 this option increases the likelihood to discover dangling pointers 1007 for the "small" blocks, even when big blocks are freed.</para> 1008 <para>Setting a value of 0 means that all the blocks are re-circulated 1009 in a FIFO order. </para> 1010 </listitem> 1011 </varlistentry> 1012 1013 <varlistentry id="opt.workaround-gcc296-bugs" xreflabel="--workaround-gcc296-bugs"> 1014 <term> 1015 <option><![CDATA[--workaround-gcc296-bugs=<yes|no> [default: no] ]]></option> 1016 </term> 1017 <listitem> 1018 <para>When enabled, assume that reads and writes some small 1019 distance below the stack pointer are due to bugs in GCC 2.96, and 1020 does not report them. The "small distance" is 256 bytes by 1021 default. Note that GCC 2.96 is the default compiler on some ancient 1022 Linux distributions (RedHat 7.X) and so you may need to use this 1023 option. Do not use it if you do not have to, as it can cause real 1024 errors to be overlooked. A better alternative is to use a more 1025 recent GCC in which this bug is fixed.</para> 1026 1027 <para>You may also need to use this option when working with 1028 GCC 3.X or 4.X on 32-bit PowerPC Linux. This is because 1029 GCC generates code which occasionally accesses below the 1030 stack pointer, particularly for floating-point to/from integer 1031 conversions. This is in violation of the 32-bit PowerPC ELF 1032 specification, which makes no provision for locations below the 1033 stack pointer to be accessible.</para> 1034 </listitem> 1035 </varlistentry> 1036 1037 <varlistentry id="opt.ignore-ranges" xreflabel="--ignore-ranges"> 1038 <term> 1039 <option><![CDATA[--ignore-ranges=0xPP-0xQQ[,0xRR-0xSS] ]]></option> 1040 </term> 1041 <listitem> 1042 <para>Any ranges listed in this option (and multiple ranges can be 1043 specified, separated by commas) will be ignored by Memcheck's 1044 addressability checking.</para> 1045 </listitem> 1046 </varlistentry> 1047 1048 <varlistentry id="opt.malloc-fill" xreflabel="--malloc-fill"> 1049 <term> 1050 <option><![CDATA[--malloc-fill=<hexnumber> ]]></option> 1051 </term> 1052 <listitem> 1053 <para>Fills blocks allocated 1054 by <computeroutput>malloc</computeroutput>, 1055 <computeroutput>new</computeroutput>, etc, but not 1056 by <computeroutput>calloc</computeroutput>, with the specified 1057 byte. This can be useful when trying to shake out obscure 1058 memory corruption problems. The allocated area is still 1059 regarded by Memcheck as undefined -- this option only affects its 1060 contents. Note that <option>--malloc-fill</option> does not 1061 affect a block of memory when it is used as argument 1062 to client requests VALGRIND_MEMPOOL_ALLOC or 1063 VALGRIND_MALLOCLIKE_BLOCK. 1064 </para> 1065 </listitem> 1066 </varlistentry> 1067 1068 <varlistentry id="opt.free-fill" xreflabel="--free-fill"> 1069 <term> 1070 <option><![CDATA[--free-fill=<hexnumber> ]]></option> 1071 </term> 1072 <listitem> 1073 <para>Fills blocks freed 1074 by <computeroutput>free</computeroutput>, 1075 <computeroutput>delete</computeroutput>, etc, with the 1076 specified byte value. This can be useful when trying to shake out 1077 obscure memory corruption problems. The freed area is still 1078 regarded by Memcheck as not valid for access -- this option only 1079 affects its contents. Note that <option>--free-fill</option> does not 1080 affect a block of memory when it is used as argument to 1081 client requests VALGRIND_MEMPOOL_FREE or VALGRIND_FREELIKE_BLOCK. 1082 </para> 1083 </listitem> 1084 </varlistentry> 1085 1086</variablelist> 1087<!-- end of xi:include in the manpage --> 1088 1089</sect1> 1090 1091 1092<sect1 id="mc-manual.suppfiles" xreflabel="Writing suppression files"> 1093<title>Writing suppression files</title> 1094 1095<para>The basic suppression format is described in 1096<xref linkend="manual-core.suppress"/>.</para> 1097 1098<para>The suppression-type (second) line should have the form:</para> 1099<programlisting><![CDATA[ 1100Memcheck:suppression_type]]></programlisting> 1101 1102<para>The Memcheck suppression types are as follows:</para> 1103 1104<itemizedlist> 1105 <listitem> 1106 <para><varname>Value1</varname>, 1107 <varname>Value2</varname>, 1108 <varname>Value4</varname>, 1109 <varname>Value8</varname>, 1110 <varname>Value16</varname>, 1111 meaning an uninitialised-value error when 1112 using a value of 1, 2, 4, 8 or 16 bytes.</para> 1113 </listitem> 1114 1115 <listitem> 1116 <para><varname>Cond</varname> (or its old 1117 name, <varname>Value0</varname>), meaning use 1118 of an uninitialised CPU condition code.</para> 1119 </listitem> 1120 1121 <listitem> 1122 <para><varname>Addr1</varname>, 1123 <varname>Addr2</varname>, 1124 <varname>Addr4</varname>, 1125 <varname>Addr8</varname>, 1126 <varname>Addr16</varname>, 1127 meaning an invalid address during a 1128 memory access of 1, 2, 4, 8 or 16 bytes respectively.</para> 1129 </listitem> 1130 1131 <listitem> 1132 <para><varname>Jump</varname>, meaning an 1133 jump to an unaddressable location error.</para> 1134 </listitem> 1135 1136 <listitem> 1137 <para><varname>Param</varname>, meaning an 1138 invalid system call parameter error.</para> 1139 </listitem> 1140 1141 <listitem> 1142 <para><varname>Free</varname>, meaning an 1143 invalid or mismatching free.</para> 1144 </listitem> 1145 1146 <listitem> 1147 <para><varname>Overlap</varname>, meaning a 1148 <computeroutput>src</computeroutput> / 1149 <computeroutput>dst</computeroutput> overlap in 1150 <function>memcpy</function> or a similar function.</para> 1151 </listitem> 1152 1153 <listitem> 1154 <para><varname>Leak</varname>, meaning 1155 a memory leak.</para> 1156 </listitem> 1157 1158</itemizedlist> 1159 1160<para><computeroutput>Param</computeroutput> errors have a mandatory extra 1161information line at this point, which is the name of the offending 1162system call parameter. </para> 1163 1164<para><computeroutput>Leak</computeroutput> errors have an optional 1165extra information line, with the following format:</para> 1166<programlisting><![CDATA[ 1167match-leak-kinds:<set>]]></programlisting> 1168<para>where <computeroutput><set></computeroutput> specifies which 1169leak kinds are matched by this suppression entry. 1170<computeroutput><set></computeroutput> is specified in the 1171same way as with the option <option>--show-leak-kinds</option>, that is, 1172one of the following:</para> 1173<itemizedlist> 1174 <listitem>a comma separated list of one or more of 1175 <option>definite indirect possible reachable</option>. 1176 </listitem> 1177 1178 <listitem><option>all</option> to specify the complete set (all leak kinds). 1179 </listitem> 1180 1181 <listitem><option>none</option> for the empty set. 1182 </listitem> 1183</itemizedlist> 1184<para>If this optional extra line is not present, the suppression 1185entry will match all leak kinds.</para> 1186 1187<para>Be aware that leak suppressions that are created using 1188<option>--gen-suppressions</option> will contain this optional extra 1189line, and therefore may match fewer leaks than you expect. You may 1190want to remove the line before using the generated 1191suppressions.</para> 1192 1193<para>The other Memcheck error kinds do not have extra lines.</para> 1194 1195<para> 1196If you give the <option>-v</option> option, Valgrind will print 1197the list of used suppressions at the end of execution. 1198For a leak suppression, this output gives the number of different 1199loss records that match the suppression, and the number of bytes 1200and blocks suppressed by the suppression. 1201If the run contains multiple leak checks, the number of bytes and blocks 1202are reset to zero before each new leak check. Note that the number of different 1203loss records is not reset to zero.</para> 1204<para>In the example below, in the last leak search, 7 blocks and 96 bytes have 1205been suppressed by a suppression with the name 1206<option>some_leak_suppression</option>:</para> 1207<programlisting><![CDATA[ 1208--21041-- used_suppression: 10 some_other_leak_suppression s.supp:14 suppressed: 12,400 bytes in 1 blocks 1209--21041-- used_suppression: 39 some_leak_suppression s.supp:2 suppressed: 96 bytes in 7 blocks 1210]]></programlisting> 1211 1212<para>For <varname>ValueN</varname> and <varname>AddrN</varname> 1213errors, the first line of the calling context is either the name of 1214the function in which the error occurred, or, failing that, the full 1215path of the <filename>.so</filename> file or executable containing the 1216error location. For <varname>Free</varname> errors, the first line is 1217the name of the function doing the freeing (eg, 1218<function>free</function>, <function>__builtin_vec_delete</function>, 1219etc). For <varname>Overlap</varname> errors, the first line is the name of the 1220function with the overlapping arguments (eg. 1221<function>memcpy</function>, <function>strcpy</function>, etc).</para> 1222 1223<para>The last part of any suppression specifies the rest of the 1224calling context that needs to be matched.</para> 1225 1226</sect1> 1227 1228 1229 1230<sect1 id="mc-manual.machine" 1231 xreflabel="Details of Memcheck's checking machinery"> 1232<title>Details of Memcheck's checking machinery</title> 1233 1234<para>Read this section if you want to know, in detail, exactly 1235what and how Memcheck is checking.</para> 1236 1237 1238<sect2 id="mc-manual.value" xreflabel="Valid-value (V) bit"> 1239<title>Valid-value (V) bits</title> 1240 1241<para>It is simplest to think of Memcheck implementing a synthetic CPU 1242which is identical to a real CPU, except for one crucial detail. Every 1243bit (literally) of data processed, stored and handled by the real CPU 1244has, in the synthetic CPU, an associated "valid-value" bit, which says 1245whether or not the accompanying bit has a legitimate value. In the 1246discussions which follow, this bit is referred to as the V (valid-value) 1247bit.</para> 1248 1249<para>Each byte in the system therefore has a 8 V bits which follow it 1250wherever it goes. For example, when the CPU loads a word-size item (4 1251bytes) from memory, it also loads the corresponding 32 V bits from a 1252bitmap which stores the V bits for the process' entire address space. 1253If the CPU should later write the whole or some part of that value to 1254memory at a different address, the relevant V bits will be stored back 1255in the V-bit bitmap.</para> 1256 1257<para>In short, each bit in the system has (conceptually) an associated V 1258bit, which follows it around everywhere, even inside the CPU. Yes, all the 1259CPU's registers (integer, floating point, vector and condition registers) 1260have their own V bit vectors. For this to work, Memcheck uses a great deal 1261of compression to represent the V bits compactly.</para> 1262 1263<para>Copying values around does not cause Memcheck to check for, or 1264report on, errors. However, when a value is used in a way which might 1265conceivably affect your program's externally-visible behaviour, 1266the associated V bits are immediately checked. If any of these indicate 1267that the value is undefined (even partially), an error is reported.</para> 1268 1269<para>Here's an (admittedly nonsensical) example:</para> 1270<programlisting><![CDATA[ 1271int i, j; 1272int a[10], b[10]; 1273for ( i = 0; i < 10; i++ ) { 1274 j = a[i]; 1275 b[i] = j; 1276}]]></programlisting> 1277 1278<para>Memcheck emits no complaints about this, since it merely copies 1279uninitialised values from <varname>a[]</varname> into 1280<varname>b[]</varname>, and doesn't use them in a way which could 1281affect the behaviour of the program. However, if 1282the loop is changed to:</para> 1283<programlisting><![CDATA[ 1284for ( i = 0; i < 10; i++ ) { 1285 j += a[i]; 1286} 1287if ( j == 77 ) 1288 printf("hello there\n"); 1289]]></programlisting> 1290 1291<para>then Memcheck will complain, at the 1292<computeroutput>if</computeroutput>, that the condition depends on 1293uninitialised values. Note that it <command>doesn't</command> complain 1294at the <varname>j += a[i];</varname>, since at that point the 1295undefinedness is not "observable". It's only when a decision has to be 1296made as to whether or not to do the <function>printf</function> -- an 1297observable action of your program -- that Memcheck complains.</para> 1298 1299<para>Most low level operations, such as adds, cause Memcheck to use the 1300V bits for the operands to calculate the V bits for the result. Even if 1301the result is partially or wholly undefined, it does not 1302complain.</para> 1303 1304<para>Checks on definedness only occur in three places: when a value is 1305used to generate a memory address, when control flow decision needs to 1306be made, and when a system call is detected, Memcheck checks definedness 1307of parameters as required.</para> 1308 1309<para>If a check should detect undefinedness, an error message is 1310issued. The resulting value is subsequently regarded as well-defined. 1311To do otherwise would give long chains of error messages. In other 1312words, once Memcheck reports an undefined value error, it tries to 1313avoid reporting further errors derived from that same undefined 1314value.</para> 1315 1316<para>This sounds overcomplicated. Why not just check all reads from 1317memory, and complain if an undefined value is loaded into a CPU 1318register? Well, that doesn't work well, because perfectly legitimate C 1319programs routinely copy uninitialised values around in memory, and we 1320don't want endless complaints about that. Here's the canonical example. 1321Consider a struct like this:</para> 1322<programlisting><![CDATA[ 1323struct S { int x; char c; }; 1324struct S s1, s2; 1325s1.x = 42; 1326s1.c = 'z'; 1327s2 = s1; 1328]]></programlisting> 1329 1330<para>The question to ask is: how large is <varname>struct S</varname>, 1331in bytes? An <varname>int</varname> is 4 bytes and a 1332<varname>char</varname> one byte, so perhaps a <varname>struct 1333S</varname> occupies 5 bytes? Wrong. All non-toy compilers we know 1334of will round the size of <varname>struct S</varname> up to a whole 1335number of words, in this case 8 bytes. Not doing this forces compilers 1336to generate truly appalling code for accessing arrays of 1337<varname>struct S</varname>'s on some architectures.</para> 1338 1339<para>So <varname>s1</varname> occupies 8 bytes, yet only 5 of them will 1340be initialised. For the assignment <varname>s2 = s1</varname>, GCC 1341generates code to copy all 8 bytes wholesale into <varname>s2</varname> 1342without regard for their meaning. If Memcheck simply checked values as 1343they came out of memory, it would yelp every time a structure assignment 1344like this happened. So the more complicated behaviour described above 1345is necessary. This allows GCC to copy 1346<varname>s1</varname> into <varname>s2</varname> any way it likes, and a 1347warning will only be emitted if the uninitialised values are later 1348used.</para> 1349 1350</sect2> 1351 1352 1353<sect2 id="mc-manual.vaddress" xreflabel=" Valid-address (A) bits"> 1354<title>Valid-address (A) bits</title> 1355 1356<para>Notice that the previous subsection describes how the validity of 1357values is established and maintained without having to say whether the 1358program does or does not have the right to access any particular memory 1359location. We now consider the latter question.</para> 1360 1361<para>As described above, every bit in memory or in the CPU has an 1362associated valid-value (V) bit. In addition, all bytes in memory, but 1363not in the CPU, have an associated valid-address (A) bit. This 1364indicates whether or not the program can legitimately read or write that 1365location. It does not give any indication of the validity of the data 1366at that location -- that's the job of the V bits -- only whether or not 1367the location may be accessed.</para> 1368 1369<para>Every time your program reads or writes memory, Memcheck checks 1370the A bits associated with the address. If any of them indicate an 1371invalid address, an error is emitted. Note that the reads and writes 1372themselves do not change the A bits, only consult them.</para> 1373 1374<para>So how do the A bits get set/cleared? Like this:</para> 1375 1376<itemizedlist> 1377 <listitem> 1378 <para>When the program starts, all the global data areas are 1379 marked as accessible.</para> 1380 </listitem> 1381 1382 <listitem> 1383 <para>When the program does 1384 <function>malloc</function>/<computeroutput>new</computeroutput>, 1385 the A bits for exactly the area allocated, and not a byte more, 1386 are marked as accessible. Upon freeing the area the A bits are 1387 changed to indicate inaccessibility.</para> 1388 </listitem> 1389 1390 <listitem> 1391 <para>When the stack pointer register (<literal>SP</literal>) moves 1392 up or down, A bits are set. The rule is that the area from 1393 <literal>SP</literal> up to the base of the stack is marked as 1394 accessible, and below <literal>SP</literal> is inaccessible. (If 1395 that sounds illogical, bear in mind that the stack grows down, not 1396 up, on almost all Unix systems, including GNU/Linux.) Tracking 1397 <literal>SP</literal> like this has the useful side-effect that the 1398 section of stack used by a function for local variables etc is 1399 automatically marked accessible on function entry and inaccessible 1400 on exit.</para> 1401 </listitem> 1402 1403 <listitem> 1404 <para>When doing system calls, A bits are changed appropriately. 1405 For example, <literal>mmap</literal> 1406 magically makes files appear in the process' 1407 address space, so the A bits must be updated if <literal>mmap</literal> 1408 succeeds.</para> 1409 </listitem> 1410 1411 <listitem> 1412 <para>Optionally, your program can tell Memcheck about such changes 1413 explicitly, using the client request mechanism described 1414 above.</para> 1415 </listitem> 1416 1417</itemizedlist> 1418 1419</sect2> 1420 1421 1422<sect2 id="mc-manual.together" xreflabel="Putting it all together"> 1423<title>Putting it all together</title> 1424 1425<para>Memcheck's checking machinery can be summarised as 1426follows:</para> 1427 1428<itemizedlist> 1429 <listitem> 1430 <para>Each byte in memory has 8 associated V (valid-value) bits, 1431 saying whether or not the byte has a defined value, and a single A 1432 (valid-address) bit, saying whether or not the program currently has 1433 the right to read/write that address. As mentioned above, heavy 1434 use of compression means the overhead is typically around 25%.</para> 1435 </listitem> 1436 1437 <listitem> 1438 <para>When memory is read or written, the relevant A bits are 1439 consulted. If they indicate an invalid address, Memcheck emits an 1440 Invalid read or Invalid write error.</para> 1441 </listitem> 1442 1443 <listitem> 1444 <para>When memory is read into the CPU's registers, the relevant V 1445 bits are fetched from memory and stored in the simulated CPU. They 1446 are not consulted.</para> 1447 </listitem> 1448 1449 <listitem> 1450 <para>When a register is written out to memory, the V bits for that 1451 register are written back to memory too.</para> 1452 </listitem> 1453 1454 <listitem> 1455 <para>When values in CPU registers are used to generate a memory 1456 address, or to determine the outcome of a conditional branch, the V 1457 bits for those values are checked, and an error emitted if any of 1458 them are undefined.</para> 1459 </listitem> 1460 1461 <listitem> 1462 <para>When values in CPU registers are used for any other purpose, 1463 Memcheck computes the V bits for the result, but does not check 1464 them.</para> 1465 </listitem> 1466 1467 <listitem> 1468 <para>Once the V bits for a value in the CPU have been checked, they 1469 are then set to indicate validity. This avoids long chains of 1470 errors.</para> 1471 </listitem> 1472 1473 <listitem> 1474 <para>When values are loaded from memory, Memcheck checks the A bits 1475 for that location and issues an illegal-address warning if needed. 1476 In that case, the V bits loaded are forced to indicate Valid, 1477 despite the location being invalid.</para> 1478 1479 <para>This apparently strange choice reduces the amount of confusing 1480 information presented to the user. It avoids the unpleasant 1481 phenomenon in which memory is read from a place which is both 1482 unaddressable and contains invalid values, and, as a result, you get 1483 not only an invalid-address (read/write) error, but also a 1484 potentially large set of uninitialised-value errors, one for every 1485 time the value is used.</para> 1486 1487 <para>There is a hazy boundary case to do with multi-byte loads from 1488 addresses which are partially valid and partially invalid. See 1489 details of the option <option>--partial-loads-ok</option> for details. 1490 </para> 1491 </listitem> 1492 1493</itemizedlist> 1494 1495 1496<para>Memcheck intercepts calls to <function>malloc</function>, 1497<function>calloc</function>, <function>realloc</function>, 1498<function>valloc</function>, <function>memalign</function>, 1499<function>free</function>, <computeroutput>new</computeroutput>, 1500<computeroutput>new[]</computeroutput>, 1501<computeroutput>delete</computeroutput> and 1502<computeroutput>delete[]</computeroutput>. The behaviour you get 1503is:</para> 1504 1505<itemizedlist> 1506 1507 <listitem> 1508 <para><function>malloc</function>/<function>new</function>/<computeroutput>new[]</computeroutput>: 1509 the returned memory is marked as addressable but not having valid 1510 values. This means you have to write to it before you can read 1511 it.</para> 1512 </listitem> 1513 1514 <listitem> 1515 <para><function>calloc</function>: returned memory is marked both 1516 addressable and valid, since <function>calloc</function> clears 1517 the area to zero.</para> 1518 </listitem> 1519 1520 <listitem> 1521 <para><function>realloc</function>: if the new size is larger than 1522 the old, the new section is addressable but invalid, as with 1523 <function>malloc</function>. If the new size is smaller, the 1524 dropped-off section is marked as unaddressable. You may only pass to 1525 <function>realloc</function> a pointer previously issued to you by 1526 <function>malloc</function>/<function>calloc</function>/<function>realloc</function>.</para> 1527 </listitem> 1528 1529 <listitem> 1530 <para><function>free</function>/<computeroutput>delete</computeroutput>/<computeroutput>delete[]</computeroutput>: 1531 you may only pass to these functions a pointer previously issued 1532 to you by the corresponding allocation function. Otherwise, 1533 Memcheck complains. If the pointer is indeed valid, Memcheck 1534 marks the entire area it points at as unaddressable, and places 1535 the block in the freed-blocks-queue. The aim is to defer as long 1536 as possible reallocation of this block. Until that happens, all 1537 attempts to access it will elicit an invalid-address error, as you 1538 would hope.</para> 1539 </listitem> 1540 1541</itemizedlist> 1542 1543</sect2> 1544</sect1> 1545 1546<sect1 id="mc-manual.monitor-commands" xreflabel="Memcheck Monitor Commands"> 1547<title>Memcheck Monitor Commands</title> 1548<para>The Memcheck tool provides monitor commands handled by Valgrind's 1549built-in gdbserver (see <xref linkend="manual-core-adv.gdbserver-commandhandling"/>). 1550</para> 1551 1552<itemizedlist> 1553 <listitem> 1554 <para><varname>get_vbits <addr> [<len>]</varname> 1555 shows the definedness (V) bits for <len> (default 1) bytes 1556 starting at <addr>. The definedness of each byte in the 1557 range is given using two hexadecimal digits. These hexadecimal 1558 digits encode the validity of each bit of the corresponding byte, 1559 using 0 if the bit is defined and 1 if the bit is undefined. 1560 If a byte is not addressable, its validity bits are replaced 1561 by <varname>__</varname> (a double underscore). 1562 </para> 1563 <para> 1564 In the following example, <varname>string10</varname> is an array 1565 of 10 characters, in which the even numbered bytes are 1566 undefined. In the below example, the byte corresponding 1567 to <varname>string10[5]</varname> is not addressable. 1568 </para> 1569<programlisting><![CDATA[ 1570(gdb) p &string10 1571$4 = (char (*)[10]) 0x8049e28 1572(gdb) monitor get_vbits 0x8049e28 10 1573ff00ff00 ff__ff00 ff00 1574(gdb) 1575]]></programlisting> 1576 1577 <para> The command get_vbits cannot be used with registers. To get 1578 the validity bits of a register, you must start Valgrind with the 1579 option <option>--vgdb-shadow-registers=yes</option>. The validity 1580 bits of a register can be obtained by printing the 'shadow 1' 1581 corresponding register. In the below x86 example, the register 1582 eax has all its bits undefined, while the register ebx is fully 1583 defined. 1584 </para> 1585<programlisting><![CDATA[ 1586(gdb) p /x $eaxs1 1587$9 = 0xffffffff 1588(gdb) p /x $ebxs1 1589$10 = 0x0 1590(gdb) 1591]]></programlisting> 1592 1593 </listitem> 1594 1595 <listitem> 1596 <para><varname>make_memory 1597 [noaccess|undefined|defined|Definedifaddressable] <addr> 1598 [<len>]</varname> marks the range of <len> (default 1) 1599 bytes at <addr> as having the given status. Parameter 1600 <varname>noaccess</varname> marks the range as non-accessible, so 1601 Memcheck will report an error on any access to it. 1602 <varname>undefined</varname> or <varname>defined</varname> mark 1603 the area as accessible, but Memcheck regards the bytes in it 1604 respectively as having undefined or defined values. 1605 <varname>Definedifaddressable</varname> marks as defined, bytes in 1606 the range which are already addressible, but makes no change to 1607 the status of bytes in the range which are not addressible. Note 1608 that the first letter of <varname>Definedifaddressable</varname> 1609 is an uppercase D to avoid confusion with <varname>defined</varname>. 1610 </para> 1611 1612 <para> 1613 In the following example, the first byte of the 1614 <varname>string10</varname> is marked as defined: 1615 </para> 1616<programlisting><![CDATA[ 1617(gdb) monitor make_memory defined 0x8049e28 1 1618(gdb) monitor get_vbits 0x8049e28 10 16190000ff00 ff00ff00 ff00 1620(gdb) 1621]]></programlisting> 1622 </listitem> 1623 1624 <listitem> 1625 <para><varname>check_memory [addressable|defined] <addr> 1626 [<len>]</varname> checks that the range of <len> 1627 (default 1) bytes at <addr> has the specified accessibility. 1628 It then outputs a description of <addr>. In the following 1629 example, a detailed description is available because the 1630 option <option>--read-var-info=yes</option> was given at Valgrind 1631 startup: 1632 </para> 1633<programlisting><![CDATA[ 1634(gdb) monitor check_memory defined 0x8049e28 1 1635Address 0x8049E28 len 1 defined 1636==14698== Location 0x8049e28 is 0 bytes inside string10[0], 1637==14698== declared at prog.c:10, in frame #0 of thread 1 1638(gdb) 1639]]></programlisting> 1640 </listitem> 1641 1642 <listitem> 1643 <para><varname>leak_check [full*|summary] 1644 [kinds <set>|reachable|possibleleak*|definiteleak] 1645 [heuristics heur1,heur2,...] 1646 [increased*|changed|any] 1647 [unlimited*|limited <max_loss_records_output>] 1648 </varname> 1649 performs a leak check. The <varname>*</varname> in the arguments 1650 indicates the default values. </para> 1651 1652 <para> If the <varname>[full*|summary]</varname> argument is 1653 <varname>summary</varname>, only a summary of the leak search is given; 1654 otherwise a full leak report is produced. A full leak report gives 1655 detailed information for each leak: the stack trace where the leaked blocks 1656 were allocated, the number of blocks leaked and their total size. When a 1657 full report is requested, the next two arguments further specify what 1658 kind of leaks to report. A leak's details are shown if they match 1659 both the second and third argument. A full leak report might 1660 output detailed information for many leaks. The nr of leaks for 1661 which information is output can be controlled using 1662 the <varname>limited</varname> argument followed by the maximum nr 1663 of leak records to output. If this maximum is reached, the leak 1664 search outputs the records with the biggest number of bytes. 1665 </para> 1666 1667 <para>The <varname>kinds</varname> argument controls what kind of blocks 1668 are shown for a <varname>full</varname> leak search. The set of leak kinds 1669 to show can be specified using a <varname><set></varname> similarly 1670 to the command line option <option>--show-leak-kinds</option>. 1671 Alternatively, the value <varname>definiteleak</varname> 1672 is equivalent to <varname>kinds definite</varname>, the 1673 value <varname>possibleleak</varname> is equivalent to 1674 <varname>kinds definite,possible</varname> : it will also show 1675 possibly leaked blocks, .i.e those for which only an interior 1676 pointer was found. The value <varname>reachable</varname> will 1677 show all block categories (i.e. is equivalent to <varname>kinds 1678 all</varname>). 1679 </para> 1680 1681 <para>The <varname>heuristics</varname> argument controls the heuristics 1682 used during the leak search. The set of heuristics to use can be specified 1683 using a <varname><set></varname> similarly 1684 to the command line option <option>--leak-check-heuristics</option>. 1685 The default value for the <varname>heuristics</varname> argument is 1686 <varname>heuristics none</varname>. 1687 </para> 1688 1689 <para>The <varname>[increased*|changed|any]</varname> argument controls what 1690 kinds of changes are shown for a <varname>full</varname> leak search. The 1691 value <varname>increased</varname> specifies that only block 1692 allocation stacks with an increased number of leaked bytes or 1693 blocks since the previous leak check should be shown. The 1694 value <varname>changed</varname> specifies that allocation stacks 1695 with any change since the previous leak check should be shown. 1696 The value <varname>any</varname> specifies that all leak entries 1697 should be shown, regardless of any increase or decrease. When 1698 If <varname>increased</varname> or <varname>changed</varname> are 1699 specified, the leak report entries will show the delta relative to 1700 the previous leak report. 1701 </para> 1702 1703 <para>The following example shows usage of the 1704 <varname>leak_check</varname> monitor command on 1705 the <varname>memcheck/tests/leak-cases.c</varname> regression 1706 test. The first command outputs one entry having an increase in 1707 the leaked bytes. The second command is the same as the first 1708 command, but uses the abbreviated forms accepted by GDB and the 1709 Valgrind gdbserver. It only outputs the summary information, as 1710 there was no increase since the previous leak search.</para> 1711<programlisting><![CDATA[ 1712(gdb) monitor leak_check full possibleleak increased 1713==19520== 16 (+16) bytes in 1 (+1) blocks are possibly lost in loss record 9 of 12 1714==19520== at 0x40070B4: malloc (vg_replace_malloc.c:263) 1715==19520== by 0x80484D5: mk (leak-cases.c:52) 1716==19520== by 0x804855F: f (leak-cases.c:81) 1717==19520== by 0x80488E0: main (leak-cases.c:107) 1718==19520== 1719==19520== LEAK SUMMARY: 1720==19520== definitely lost: 32 (+0) bytes in 2 (+0) blocks 1721==19520== indirectly lost: 16 (+0) bytes in 1 (+0) blocks 1722==19520== possibly lost: 32 (+16) bytes in 2 (+1) blocks 1723==19520== still reachable: 96 (+16) bytes in 6 (+1) blocks 1724==19520== suppressed: 0 (+0) bytes in 0 (+0) blocks 1725==19520== Reachable blocks (those to which a pointer was found) are not shown. 1726==19520== To see them, add 'reachable any' args to leak_check 1727==19520== 1728(gdb) mo l 1729==19520== LEAK SUMMARY: 1730==19520== definitely lost: 32 (+0) bytes in 2 (+0) blocks 1731==19520== indirectly lost: 16 (+0) bytes in 1 (+0) blocks 1732==19520== possibly lost: 32 (+0) bytes in 2 (+0) blocks 1733==19520== still reachable: 96 (+0) bytes in 6 (+0) blocks 1734==19520== suppressed: 0 (+0) bytes in 0 (+0) blocks 1735==19520== Reachable blocks (those to which a pointer was found) are not shown. 1736==19520== To see them, add 'reachable any' args to leak_check 1737==19520== 1738(gdb) 1739]]></programlisting> 1740 <para>Note that when using Valgrind's gdbserver, it is not 1741 necessary to rerun 1742 with <option>--leak-check=full</option> 1743 <option>--show-reachable=yes</option> to see the reachable 1744 blocks. You can obtain the same information without rerunning by 1745 using the GDB command <computeroutput>monitor leak_check full 1746 reachable any</computeroutput> (or, using 1747 abbreviation: <computeroutput>mo l f r a</computeroutput>). 1748 </para> 1749 </listitem> 1750 1751 <listitem> 1752 <para><varname>block_list <loss_record_nr> </varname> 1753 shows the list of blocks belonging to <loss_record_nr>. 1754 </para> 1755 1756 <para> A leak search merges the allocated blocks in loss records : 1757 a loss record re-groups all blocks having the same state (for 1758 example, Definitely Lost) and the same allocation backtrace. 1759 Each loss record is identified in the leak search result 1760 by a loss record number. 1761 The <varname>block_list</varname> command shows the loss record information 1762 followed by the addresses and sizes of the blocks which have been 1763 merged in the loss record. 1764 </para> 1765 1766 <para> If a directly lost block causes some other blocks to be indirectly 1767 lost, the block_list command will also show these indirectly lost blocks. 1768 The indirectly lost blocks will be indented according to the level of indirection 1769 between the directly lost block and the indirectly lost block(s). 1770 Each indirectly lost block is followed by the reference of its loss record. 1771 </para> 1772 1773 <para> The block_list command can be used on the results of a leak search as long 1774 as no block has been freed after this leak search: as soon as the program frees 1775 a block, a new leak search is needed before block_list can be used again. 1776 </para> 1777 1778 <para> 1779 In the below example, the program leaks a tree structure by losing the pointer to 1780 the block A (top of the tree). 1781 So, the block A is directly lost, causing an indirect 1782 loss of blocks B to G. The first block_list command shows the loss record of A 1783 (a definitely lost block with address 0x4028028, size 16). The addresses and sizes 1784 of the indirectly lost blocks due to block A are shown below the block A. 1785 The second command shows the details of one of the indirect loss records output 1786 by the first command. 1787 </para> 1788<programlisting><![CDATA[ 1789 A 1790 / \ 1791 B C 1792 / \ / \ 1793 D E F G 1794]]></programlisting> 1795 1796<programlisting><![CDATA[ 1797(gdb) bt 1798#0 main () at leak-tree.c:69 1799(gdb) monitor leak_check full any 1800==19552== 112 (16 direct, 96 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 7 1801==19552== at 0x40070B4: malloc (vg_replace_malloc.c:263) 1802==19552== by 0x80484D5: mk (leak-tree.c:28) 1803==19552== by 0x80484FC: f (leak-tree.c:41) 1804==19552== by 0x8048856: main (leak-tree.c:63) 1805==19552== 1806==19552== LEAK SUMMARY: 1807==19552== definitely lost: 16 bytes in 1 blocks 1808==19552== indirectly lost: 96 bytes in 6 blocks 1809==19552== possibly lost: 0 bytes in 0 blocks 1810==19552== still reachable: 0 bytes in 0 blocks 1811==19552== suppressed: 0 bytes in 0 blocks 1812==19552== 1813(gdb) monitor block_list 7 1814==19552== 112 (16 direct, 96 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 7 1815==19552== at 0x40070B4: malloc (vg_replace_malloc.c:263) 1816==19552== by 0x80484D5: mk (leak-tree.c:28) 1817==19552== by 0x80484FC: f (leak-tree.c:41) 1818==19552== by 0x8048856: main (leak-tree.c:63) 1819==19552== 0x4028028[16] 1820==19552== 0x4028068[16] indirect loss record 1 1821==19552== 0x40280E8[16] indirect loss record 3 1822==19552== 0x4028128[16] indirect loss record 4 1823==19552== 0x40280A8[16] indirect loss record 2 1824==19552== 0x4028168[16] indirect loss record 5 1825==19552== 0x40281A8[16] indirect loss record 6 1826(gdb) mo b 2 1827==19552== 16 bytes in 1 blocks are indirectly lost in loss record 2 of 7 1828==19552== at 0x40070B4: malloc (vg_replace_malloc.c:263) 1829==19552== by 0x80484D5: mk (leak-tree.c:28) 1830==19552== by 0x8048519: f (leak-tree.c:43) 1831==19552== by 0x8048856: main (leak-tree.c:63) 1832==19552== 0x40280A8[16] 1833==19552== 0x4028168[16] indirect loss record 5 1834==19552== 0x40281A8[16] indirect loss record 6 1835(gdb) 1836 1837]]></programlisting> 1838 1839 </listitem> 1840 1841 <listitem> 1842 <para><varname>who_points_at <addr> [<len>]</varname> 1843 shows all the locations where a pointer to addr is found. 1844 If len is equal to 1, the command only shows the locations pointing 1845 exactly at addr (i.e. the "start pointers" to addr). 1846 If len is > 1, "interior pointers" pointing at the len first bytes 1847 will also be shown. 1848 </para> 1849 1850 <para>The locations searched for are the same as the locations 1851 used in the leak search. So, <varname>who_points_at</varname> can a.o. 1852 be used to show why the leak search still can reach a block, or can 1853 search for dangling pointers to a freed block. 1854 Each location pointing at addr (or pointing inside addr if interior pointers 1855 are being searched for) will be described. 1856 </para> 1857 1858 <para>In the below example, the pointers to the 'tree block A' (see example 1859 in command <varname>block_list</varname>) is shown before the tree was leaked. 1860 The descriptions are detailed as the option <option>--read-var-info=yes</option> 1861 was given at Valgrind startup. The second call shows the pointers (start and interior 1862 pointers) to block G. The block G (0x40281A8) is reachable via block C (0x40280a8) 1863 and register ECX of tid 1 (tid is the Valgrind thread id). 1864 It is "interior reachable" via the register EBX. 1865 </para> 1866 1867<programlisting><![CDATA[ 1868(gdb) monitor who_points_at 0x4028028 1869==20852== Searching for pointers to 0x4028028 1870==20852== *0x8049e20 points at 0x4028028 1871==20852== Location 0x8049e20 is 0 bytes inside global var "t" 1872==20852== declared at leak-tree.c:35 1873(gdb) monitor who_points_at 0x40281A8 16 1874==20852== Searching for pointers pointing in 16 bytes from 0x40281a8 1875==20852== *0x40280ac points at 0x40281a8 1876==20852== Address 0x40280ac is 4 bytes inside a block of size 16 alloc'd 1877==20852== at 0x40070B4: malloc (vg_replace_malloc.c:263) 1878==20852== by 0x80484D5: mk (leak-tree.c:28) 1879==20852== by 0x8048519: f (leak-tree.c:43) 1880==20852== by 0x8048856: main (leak-tree.c:63) 1881==20852== tid 1 register ECX points at 0x40281a8 1882==20852== tid 1 register EBX interior points at 2 bytes inside 0x40281a8 1883(gdb) 1884]]></programlisting> 1885 1886 <para> When <varname>who_points_at</varname> finds an interior pointer, 1887 it will report the heuristic(s) with which this interior pointer 1888 will be considered as reachable. Note that this is done independently 1889 of the value of the option <option>--leak-check-heuristics</option>. 1890 In the below example, the loss record 6 indicates a possibly lost 1891 block. <varname>who_points_at</varname> reports that there is an interior 1892 pointer pointing in this block, and that the block can be considered 1893 reachable using the heuristic 1894 <computeroutput>multipleinheritance</computeroutput>. 1895 </para> 1896 1897<programlisting><![CDATA[ 1898(gdb) monitor block_list 6 1899==3748== 8 bytes in 1 blocks are possibly lost in loss record 6 of 7 1900==3748== at 0x4007D77: operator new(unsigned int) (vg_replace_malloc.c:313) 1901==3748== by 0x8048954: main (leak_cpp_interior.cpp:43) 1902==3748== 0x402A0E0[8] 1903(gdb) monitor who_points_at 0x402A0E0 8 1904==3748== Searching for pointers pointing in 8 bytes from 0x402a0e0 1905==3748== *0xbe8ee078 interior points at 4 bytes inside 0x402a0e0 1906==3748== Address 0xbe8ee078 is on thread 1's stack 1907==3748== block at 0x402a0e0 considered reachable by ptr 0x402a0e4 using multipleinheritance heuristic 1908(gdb) 1909]]></programlisting> 1910 1911 </listitem> 1912 1913</itemizedlist> 1914 1915</sect1> 1916 1917<sect1 id="mc-manual.clientreqs" xreflabel="Client requests"> 1918<title>Client Requests</title> 1919 1920<para>The following client requests are defined in 1921<filename>memcheck.h</filename>. 1922See <filename>memcheck.h</filename> for exact details of their 1923arguments.</para> 1924 1925<itemizedlist> 1926 1927 <listitem> 1928 <para><varname>VALGRIND_MAKE_MEM_NOACCESS</varname>, 1929 <varname>VALGRIND_MAKE_MEM_UNDEFINED</varname> and 1930 <varname>VALGRIND_MAKE_MEM_DEFINED</varname>. 1931 These mark address ranges as completely inaccessible, 1932 accessible but containing undefined data, and accessible and 1933 containing defined data, respectively.</para> 1934 </listitem> 1935 1936 <listitem> 1937 <para><varname>VALGRIND_MAKE_MEM_DEFINED_IF_ADDRESSABLE</varname>. 1938 This is just like <varname>VALGRIND_MAKE_MEM_DEFINED</varname> but only 1939 affects those bytes that are already addressable.</para> 1940 </listitem> 1941 1942 <listitem> 1943 <para><varname>VALGRIND_CHECK_MEM_IS_ADDRESSABLE</varname> and 1944 <varname>VALGRIND_CHECK_MEM_IS_DEFINED</varname>: check immediately 1945 whether or not the given address range has the relevant property, 1946 and if not, print an error message. Also, for the convenience of 1947 the client, returns zero if the relevant property holds; otherwise, 1948 the returned value is the address of the first byte for which the 1949 property is not true. Always returns 0 when not run on 1950 Valgrind.</para> 1951 </listitem> 1952 1953 <listitem> 1954 <para><varname>VALGRIND_CHECK_VALUE_IS_DEFINED</varname>: a quick and easy 1955 way to find out whether Valgrind thinks a particular value 1956 (lvalue, to be precise) is addressable and defined. Prints an error 1957 message if not. It has no return value.</para> 1958 </listitem> 1959 1960 <listitem> 1961 <para><varname>VALGRIND_DO_LEAK_CHECK</varname>: does a full memory leak 1962 check (like <option>--leak-check=full</option>) right now. 1963 This is useful for incrementally checking for leaks between arbitrary 1964 places in the program's execution. It has no return value.</para> 1965 </listitem> 1966 1967 <listitem> 1968 <para><varname>VALGRIND_DO_ADDED_LEAK_CHECK</varname>: same as 1969 <varname> VALGRIND_DO_LEAK_CHECK</varname> but only shows the 1970 entries for which there was an increase in leaked bytes or leaked 1971 number of blocks since the previous leak search. It has no return 1972 value.</para> 1973 </listitem> 1974 1975 <listitem> 1976 <para><varname>VALGRIND_DO_CHANGED_LEAK_CHECK</varname>: same as 1977 <varname>VALGRIND_DO_LEAK_CHECK</varname> but only shows the 1978 entries for which there was an increase or decrease in leaked 1979 bytes or leaked number of blocks since the previous leak search. It 1980 has no return value.</para> 1981 </listitem> 1982 1983 <listitem> 1984 <para><varname>VALGRIND_DO_QUICK_LEAK_CHECK</varname>: like 1985 <varname>VALGRIND_DO_LEAK_CHECK</varname>, except it produces only a leak 1986 summary (like <option>--leak-check=summary</option>). 1987 It has no return value.</para> 1988 </listitem> 1989 1990 <listitem> 1991 <para><varname>VALGRIND_COUNT_LEAKS</varname>: fills in the four 1992 arguments with the number of bytes of memory found by the previous 1993 leak check to be leaked (i.e. the sum of direct leaks and indirect leaks), 1994 dubious, reachable and suppressed. This is useful in test harness code, 1995 after calling <varname>VALGRIND_DO_LEAK_CHECK</varname> or 1996 <varname>VALGRIND_DO_QUICK_LEAK_CHECK</varname>.</para> 1997 </listitem> 1998 1999 <listitem> 2000 <para><varname>VALGRIND_COUNT_LEAK_BLOCKS</varname>: identical to 2001 <varname>VALGRIND_COUNT_LEAKS</varname> except that it returns the 2002 number of blocks rather than the number of bytes in each 2003 category.</para> 2004 </listitem> 2005 2006 <listitem> 2007 <para><varname>VALGRIND_GET_VBITS</varname> and 2008 <varname>VALGRIND_SET_VBITS</varname>: allow you to get and set the 2009 V (validity) bits for an address range. You should probably only 2010 set V bits that you have got with 2011 <varname>VALGRIND_GET_VBITS</varname>. Only for those who really 2012 know what they are doing.</para> 2013 </listitem> 2014 2015 <listitem> 2016 <para><varname>VALGRIND_CREATE_BLOCK</varname> and 2017 <varname>VALGRIND_DISCARD</varname>. <varname>VALGRIND_CREATE_BLOCK</varname> 2018 takes an address, a number of bytes and a character string. The 2019 specified address range is then associated with that string. When 2020 Memcheck reports an invalid access to an address in the range, it 2021 will describe it in terms of this block rather than in terms of 2022 any other block it knows about. Note that the use of this macro 2023 does not actually change the state of memory in any way -- it 2024 merely gives a name for the range. 2025 </para> 2026 2027 <para>At some point you may want Memcheck to stop reporting errors 2028 in terms of the block named 2029 by <varname>VALGRIND_CREATE_BLOCK</varname>. To make this 2030 possible, <varname>VALGRIND_CREATE_BLOCK</varname> returns a 2031 "block handle", which is a C <varname>int</varname> value. You 2032 can pass this block handle to <varname>VALGRIND_DISCARD</varname>. 2033 After doing so, Valgrind will no longer relate addressing errors 2034 in the specified range to the block. Passing invalid handles to 2035 <varname>VALGRIND_DISCARD</varname> is harmless. 2036 </para> 2037 </listitem> 2038 2039</itemizedlist> 2040 2041</sect1> 2042 2043 2044 2045 2046<sect1 id="mc-manual.mempools" xreflabel="Memory Pools"> 2047<title>Memory Pools: describing and working with custom allocators</title> 2048 2049<para>Some programs use custom memory allocators, often for performance 2050reasons. Left to itself, Memcheck is unable to understand the 2051behaviour of custom allocation schemes as well as it understands the 2052standard allocators, and so may miss errors and leaks in your program. What 2053this section describes is a way to give Memcheck enough of a description of 2054your custom allocator that it can make at least some sense of what is 2055happening.</para> 2056 2057<para>There are many different sorts of custom allocator, so Memcheck 2058attempts to reason about them using a loose, abstract model. We 2059use the following terminology when describing custom allocation 2060systems:</para> 2061 2062<itemizedlist> 2063 <listitem> 2064 <para>Custom allocation involves a set of independent "memory pools". 2065 </para> 2066 </listitem> 2067 <listitem> 2068 <para>Memcheck's notion of a a memory pool consists of a single "anchor 2069 address" and a set of non-overlapping "chunks" associated with the 2070 anchor address.</para> 2071 </listitem> 2072 <listitem> 2073 <para>Typically a pool's anchor address is the address of a 2074 book-keeping "header" structure.</para> 2075 </listitem> 2076 <listitem> 2077 <para>Typically the pool's chunks are drawn from a contiguous 2078 "superblock" acquired through the system 2079 <function>malloc</function> or 2080 <function>mmap</function>.</para> 2081 </listitem> 2082 2083</itemizedlist> 2084 2085<para>Keep in mind that the last two points above say "typically": the 2086Valgrind mempool client request API is intentionally vague about the 2087exact structure of a mempool. There is no specific mention made of 2088headers or superblocks. Nevertheless, the following picture may help 2089elucidate the intention of the terms in the API:</para> 2090 2091<programlisting><![CDATA[ 2092 "pool" 2093 (anchor address) 2094 | 2095 v 2096 +--------+---+ 2097 | header | o | 2098 +--------+-|-+ 2099 | 2100 v superblock 2101 +------+---+--------------+---+------------------+ 2102 | |rzB| allocation |rzB| | 2103 +------+---+--------------+---+------------------+ 2104 ^ ^ 2105 | | 2106 "addr" "addr"+"size" 2107]]></programlisting> 2108 2109<para> 2110Note that the header and the superblock may be contiguous or 2111discontiguous, and there may be multiple superblocks associated with a 2112single header; such variations are opaque to Memcheck. The API 2113only requires that your allocation scheme can present sensible values 2114of "pool", "addr" and "size".</para> 2115 2116<para> 2117Typically, before making client requests related to mempools, a client 2118program will have allocated such a header and superblock for their 2119mempool, and marked the superblock NOACCESS using the 2120<varname>VALGRIND_MAKE_MEM_NOACCESS</varname> client request.</para> 2121 2122<para> 2123When dealing with mempools, the goal is to maintain a particular 2124invariant condition: that Memcheck believes the unallocated portions 2125of the pool's superblock (including redzones) are NOACCESS. To 2126maintain this invariant, the client program must ensure that the 2127superblock starts out in that state; Memcheck cannot make it so, since 2128Memcheck never explicitly learns about the superblock of a pool, only 2129the allocated chunks within the pool.</para> 2130 2131<para> 2132Once the header and superblock for a pool are established and properly 2133marked, there are a number of client requests programs can use to 2134inform Memcheck about changes to the state of a mempool:</para> 2135 2136<itemizedlist> 2137 2138 <listitem> 2139 <para> 2140 <varname>VALGRIND_CREATE_MEMPOOL(pool, rzB, is_zeroed)</varname>: 2141 This request registers the address <varname>pool</varname> as the anchor 2142 address for a memory pool. It also provides a size 2143 <varname>rzB</varname>, specifying how large the redzones placed around 2144 chunks allocated from the pool should be. Finally, it provides an 2145 <varname>is_zeroed</varname> argument that specifies whether the pool's 2146 chunks are zeroed (more precisely: defined) when allocated. 2147 </para> 2148 <para> 2149 Upon completion of this request, no chunks are associated with the 2150 pool. The request simply tells Memcheck that the pool exists, so that 2151 subsequent calls can refer to it as a pool. 2152 </para> 2153 </listitem> 2154 2155 <listitem> 2156 <para><varname>VALGRIND_DESTROY_MEMPOOL(pool)</varname>: 2157 This request tells Memcheck that a pool is being torn down. Memcheck 2158 then removes all records of chunks associated with the pool, as well 2159 as its record of the pool's existence. While destroying its records of 2160 a mempool, Memcheck resets the redzones of any live chunks in the pool 2161 to NOACCESS. 2162 </para> 2163 </listitem> 2164 2165 <listitem> 2166 <para><varname>VALGRIND_MEMPOOL_ALLOC(pool, addr, size)</varname>: 2167 This request informs Memcheck that a <varname>size</varname>-byte chunk 2168 has been allocated at <varname>addr</varname>, and associates the chunk with the 2169 specified 2170 <varname>pool</varname>. If the pool was created with nonzero 2171 <varname>rzB</varname> redzones, Memcheck will mark the 2172 <varname>rzB</varname> bytes before and after the chunk as NOACCESS. If 2173 the pool was created with the <varname>is_zeroed</varname> argument set, 2174 Memcheck will mark the chunk as DEFINED, otherwise Memcheck will mark 2175 the chunk as UNDEFINED. 2176 </para> 2177 </listitem> 2178 2179 <listitem> 2180 <para><varname>VALGRIND_MEMPOOL_FREE(pool, addr)</varname>: 2181 This request informs Memcheck that the chunk at <varname>addr</varname> 2182 should no longer be considered allocated. Memcheck will mark the chunk 2183 associated with <varname>addr</varname> as NOACCESS, and delete its 2184 record of the chunk's existence. 2185 </para> 2186 </listitem> 2187 2188 <listitem> 2189 <para><varname>VALGRIND_MEMPOOL_TRIM(pool, addr, size)</varname>: 2190 This request trims the chunks associated with <varname>pool</varname>. 2191 The request only operates on chunks associated with 2192 <varname>pool</varname>. Trimming is formally defined as:</para> 2193 <itemizedlist> 2194 <listitem> 2195 <para> All chunks entirely inside the range 2196 <varname>addr..(addr+size-1)</varname> are preserved.</para> 2197 </listitem> 2198 <listitem> 2199 <para>All chunks entirely outside the range 2200 <varname>addr..(addr+size-1)</varname> are discarded, as though 2201 <varname>VALGRIND_MEMPOOL_FREE</varname> was called on them. </para> 2202 </listitem> 2203 <listitem> 2204 <para>All other chunks must intersect with the range 2205 <varname>addr..(addr+size-1)</varname>; areas outside the 2206 intersection are marked as NOACCESS, as though they had been 2207 independently freed with 2208 <varname>VALGRIND_MEMPOOL_FREE</varname>.</para> 2209 </listitem> 2210 </itemizedlist> 2211 <para>This is a somewhat rare request, but can be useful in 2212 implementing the type of mass-free operations common in custom 2213 LIFO allocators.</para> 2214 </listitem> 2215 2216 <listitem> 2217 <para><varname>VALGRIND_MOVE_MEMPOOL(poolA, poolB)</varname>: This 2218 request informs Memcheck that the pool previously anchored at 2219 address <varname>poolA</varname> has moved to anchor address 2220 <varname>poolB</varname>. This is a rare request, typically only needed 2221 if you <function>realloc</function> the header of a mempool.</para> 2222 <para>No memory-status bits are altered by this request.</para> 2223 </listitem> 2224 2225 <listitem> 2226 <para> 2227 <varname>VALGRIND_MEMPOOL_CHANGE(pool, addrA, addrB, 2228 size)</varname>: This request informs Memcheck that the chunk 2229 previously allocated at address <varname>addrA</varname> within 2230 <varname>pool</varname> has been moved and/or resized, and should be 2231 changed to cover the region <varname>addrB..(addrB+size-1)</varname>. This 2232 is a rare request, typically only needed if you 2233 <function>realloc</function> a superblock or wish to extend a chunk 2234 without changing its memory-status bits. 2235 </para> 2236 <para>No memory-status bits are altered by this request. 2237 </para> 2238 </listitem> 2239 2240 <listitem> 2241 <para><varname>VALGRIND_MEMPOOL_EXISTS(pool)</varname>: 2242 This request informs the caller whether or not Memcheck is currently 2243 tracking a mempool at anchor address <varname>pool</varname>. It 2244 evaluates to 1 when there is a mempool associated with that address, 0 2245 otherwise. This is a rare request, only useful in circumstances when 2246 client code might have lost track of the set of active mempools. 2247 </para> 2248 </listitem> 2249 2250</itemizedlist> 2251 2252</sect1> 2253 2254 2255 2256 2257 2258 2259 2260<sect1 id="mc-manual.mpiwrap" xreflabel="MPI Wrappers"> 2261<title>Debugging MPI Parallel Programs with Valgrind</title> 2262 2263<para>Memcheck supports debugging of distributed-memory applications 2264which use the MPI message passing standard. This support consists of a 2265library of wrapper functions for the 2266<computeroutput>PMPI_*</computeroutput> interface. When incorporated 2267into the application's address space, either by direct linking or by 2268<computeroutput>LD_PRELOAD</computeroutput>, the wrappers intercept 2269calls to <computeroutput>PMPI_Send</computeroutput>, 2270<computeroutput>PMPI_Recv</computeroutput>, etc. They then 2271use client requests to inform Memcheck of memory state changes caused 2272by the function being wrapped. This reduces the number of false 2273positives that Memcheck otherwise typically reports for MPI 2274applications.</para> 2275 2276<para>The wrappers also take the opportunity to carefully check 2277size and definedness of buffers passed as arguments to MPI functions, hence 2278detecting errors such as passing undefined data to 2279<computeroutput>PMPI_Send</computeroutput>, or receiving data into a 2280buffer which is too small.</para> 2281 2282<para>Unlike most of the rest of Valgrind, the wrapper library is subject to a 2283BSD-style license, so you can link it into any code base you like. 2284See the top of <computeroutput>mpi/libmpiwrap.c</computeroutput> 2285for license details.</para> 2286 2287 2288<sect2 id="mc-manual.mpiwrap.build" xreflabel="Building MPI Wrappers"> 2289<title>Building and installing the wrappers</title> 2290 2291<para> The wrapper library will be built automatically if possible. 2292Valgrind's configure script will look for a suitable 2293<computeroutput>mpicc</computeroutput> to build it with. This must be 2294the same <computeroutput>mpicc</computeroutput> you use to build the 2295MPI application you want to debug. By default, Valgrind tries 2296<computeroutput>mpicc</computeroutput>, but you can specify a 2297different one by using the configure-time option 2298<option>--with-mpicc</option>. Currently the 2299wrappers are only buildable with 2300<computeroutput>mpicc</computeroutput>s which are based on GNU 2301GCC or Intel's C++ Compiler.</para> 2302 2303<para>Check that the configure script prints a line like this:</para> 2304 2305<programlisting><![CDATA[ 2306checking for usable MPI2-compliant mpicc and mpi.h... yes, mpicc 2307]]></programlisting> 2308 2309<para>If it says <computeroutput>... no</computeroutput>, your 2310<computeroutput>mpicc</computeroutput> has failed to compile and link 2311a test MPI2 program.</para> 2312 2313<para>If the configure test succeeds, continue in the usual way with 2314<computeroutput>make</computeroutput> and <computeroutput>make 2315install</computeroutput>. The final install tree should then contain 2316<computeroutput>libmpiwrap-<platform>.so</computeroutput>. 2317</para> 2318 2319<para>Compile up a test MPI program (eg, MPI hello-world) and try 2320this:</para> 2321 2322<programlisting><![CDATA[ 2323LD_PRELOAD=$prefix/lib/valgrind/libmpiwrap-<platform>.so \ 2324 mpirun [args] $prefix/bin/valgrind ./hello 2325]]></programlisting> 2326 2327<para>You should see something similar to the following</para> 2328 2329<programlisting><![CDATA[ 2330valgrind MPI wrappers 31901: Active for pid 31901 2331valgrind MPI wrappers 31901: Try MPIWRAP_DEBUG=help for possible options 2332]]></programlisting> 2333 2334<para>repeated for every process in the group. If you do not see 2335these, there is an build/installation problem of some kind.</para> 2336 2337<para> The MPI functions to be wrapped are assumed to be in an ELF 2338shared object with soname matching 2339<computeroutput>libmpi.so*</computeroutput>. This is known to be 2340correct at least for Open MPI and Quadrics MPI, and can easily be 2341changed if required.</para> 2342</sect2> 2343 2344 2345<sect2 id="mc-manual.mpiwrap.gettingstarted" 2346 xreflabel="Getting started with MPI Wrappers"> 2347<title>Getting started</title> 2348 2349<para>Compile your MPI application as usual, taking care to link it 2350using the same <computeroutput>mpicc</computeroutput> that your 2351Valgrind build was configured with.</para> 2352 2353<para> 2354Use the following basic scheme to run your application on Valgrind with 2355the wrappers engaged:</para> 2356 2357<programlisting><![CDATA[ 2358MPIWRAP_DEBUG=[wrapper-args] \ 2359 LD_PRELOAD=$prefix/lib/valgrind/libmpiwrap-<platform>.so \ 2360 mpirun [mpirun-args] \ 2361 $prefix/bin/valgrind [valgrind-args] \ 2362 [application] [app-args] 2363]]></programlisting> 2364 2365<para>As an alternative to 2366<computeroutput>LD_PRELOAD</computeroutput>ing 2367<computeroutput>libmpiwrap-<platform>.so</computeroutput>, you can 2368simply link it to your application if desired. This should not disturb 2369native behaviour of your application in any way.</para> 2370</sect2> 2371 2372 2373<sect2 id="mc-manual.mpiwrap.controlling" 2374 xreflabel="Controlling the MPI Wrappers"> 2375<title>Controlling the wrapper library</title> 2376 2377<para>Environment variable 2378<computeroutput>MPIWRAP_DEBUG</computeroutput> is consulted at 2379startup. The default behaviour is to print a starting banner</para> 2380 2381<programlisting><![CDATA[ 2382valgrind MPI wrappers 16386: Active for pid 16386 2383valgrind MPI wrappers 16386: Try MPIWRAP_DEBUG=help for possible options 2384]]></programlisting> 2385 2386<para> and then be relatively quiet.</para> 2387 2388<para>You can give a list of comma-separated options in 2389<computeroutput>MPIWRAP_DEBUG</computeroutput>. These are</para> 2390 2391<itemizedlist> 2392 <listitem> 2393 <para><computeroutput>verbose</computeroutput>: 2394 show entries/exits of all wrappers. Also show extra 2395 debugging info, such as the status of outstanding 2396 <computeroutput>MPI_Request</computeroutput>s resulting 2397 from uncompleted <computeroutput>MPI_Irecv</computeroutput>s.</para> 2398 </listitem> 2399 <listitem> 2400 <para><computeroutput>quiet</computeroutput>: 2401 opposite of <computeroutput>verbose</computeroutput>, only print 2402 anything when the wrappers want 2403 to report a detected programming error, or in case of catastrophic 2404 failure of the wrappers.</para> 2405 </listitem> 2406 <listitem> 2407 <para><computeroutput>warn</computeroutput>: 2408 by default, functions which lack proper wrappers 2409 are not commented on, just silently 2410 ignored. This causes a warning to be printed for each unwrapped 2411 function used, up to a maximum of three warnings per function.</para> 2412 </listitem> 2413 <listitem> 2414 <para><computeroutput>strict</computeroutput>: 2415 print an error message and abort the program if 2416 a function lacking a wrapper is used.</para> 2417 </listitem> 2418</itemizedlist> 2419 2420<para> If you want to use Valgrind's XML output facility 2421(<option>--xml=yes</option>), you should pass 2422<computeroutput>quiet</computeroutput> in 2423<computeroutput>MPIWRAP_DEBUG</computeroutput> so as to get rid of any 2424extraneous printing from the wrappers.</para> 2425 2426</sect2> 2427 2428 2429<sect2 id="mc-manual.mpiwrap.limitations.functions" 2430 xreflabel="Functions: Abilities and Limitations"> 2431<title>Functions</title> 2432 2433<para>All MPI2 functions except 2434<computeroutput>MPI_Wtick</computeroutput>, 2435<computeroutput>MPI_Wtime</computeroutput> and 2436<computeroutput>MPI_Pcontrol</computeroutput> have wrappers. The 2437first two are not wrapped because they return a 2438<computeroutput>double</computeroutput>, which Valgrind's 2439function-wrap mechanism cannot handle (but it could easily be 2440extended to do so). <computeroutput>MPI_Pcontrol</computeroutput> cannot be 2441wrapped as it has variable arity: 2442<computeroutput>int MPI_Pcontrol(const int level, ...)</computeroutput></para> 2443 2444<para>Most functions are wrapped with a default wrapper which does 2445nothing except complain or abort if it is called, depending on 2446settings in <computeroutput>MPIWRAP_DEBUG</computeroutput> listed 2447above. The following functions have "real", do-something-useful 2448wrappers:</para> 2449 2450<programlisting><![CDATA[ 2451PMPI_Send PMPI_Bsend PMPI_Ssend PMPI_Rsend 2452 2453PMPI_Recv PMPI_Get_count 2454 2455PMPI_Isend PMPI_Ibsend PMPI_Issend PMPI_Irsend 2456 2457PMPI_Irecv 2458PMPI_Wait PMPI_Waitall 2459PMPI_Test PMPI_Testall 2460 2461PMPI_Iprobe PMPI_Probe 2462 2463PMPI_Cancel 2464 2465PMPI_Sendrecv 2466 2467PMPI_Type_commit PMPI_Type_free 2468 2469PMPI_Pack PMPI_Unpack 2470 2471PMPI_Bcast PMPI_Gather PMPI_Scatter PMPI_Alltoall 2472PMPI_Reduce PMPI_Allreduce PMPI_Op_create 2473 2474PMPI_Comm_create PMPI_Comm_dup PMPI_Comm_free PMPI_Comm_rank PMPI_Comm_size 2475 2476PMPI_Error_string 2477PMPI_Init PMPI_Initialized PMPI_Finalize 2478]]></programlisting> 2479 2480<para> A few functions such as 2481<computeroutput>PMPI_Address</computeroutput> are listed as 2482<computeroutput>HAS_NO_WRAPPER</computeroutput>. They have no wrapper 2483at all as there is nothing worth checking, and giving a no-op wrapper 2484would reduce performance for no reason.</para> 2485 2486<para> Note that the wrapper library itself can itself generate large 2487numbers of calls to the MPI implementation, especially when walking 2488complex types. The most common functions called are 2489<computeroutput>PMPI_Extent</computeroutput>, 2490<computeroutput>PMPI_Type_get_envelope</computeroutput>, 2491<computeroutput>PMPI_Type_get_contents</computeroutput>, and 2492<computeroutput>PMPI_Type_free</computeroutput>. </para> 2493</sect2> 2494 2495<sect2 id="mc-manual.mpiwrap.limitations.types" 2496 xreflabel="Types: Abilities and Limitations"> 2497<title>Types</title> 2498 2499<para> MPI-1.1 structured types are supported, and walked exactly. 2500The currently supported combiners are 2501<computeroutput>MPI_COMBINER_NAMED</computeroutput>, 2502<computeroutput>MPI_COMBINER_CONTIGUOUS</computeroutput>, 2503<computeroutput>MPI_COMBINER_VECTOR</computeroutput>, 2504<computeroutput>MPI_COMBINER_HVECTOR</computeroutput> 2505<computeroutput>MPI_COMBINER_INDEXED</computeroutput>, 2506<computeroutput>MPI_COMBINER_HINDEXED</computeroutput> and 2507<computeroutput>MPI_COMBINER_STRUCT</computeroutput>. This should 2508cover all MPI-1.1 types. The mechanism (function 2509<computeroutput>walk_type</computeroutput>) should extend easily to 2510cover MPI2 combiners.</para> 2511 2512<para>MPI defines some named structured types 2513(<computeroutput>MPI_FLOAT_INT</computeroutput>, 2514<computeroutput>MPI_DOUBLE_INT</computeroutput>, 2515<computeroutput>MPI_LONG_INT</computeroutput>, 2516<computeroutput>MPI_2INT</computeroutput>, 2517<computeroutput>MPI_SHORT_INT</computeroutput>, 2518<computeroutput>MPI_LONG_DOUBLE_INT</computeroutput>) which are pairs 2519of some basic type and a C <computeroutput>int</computeroutput>. 2520Unfortunately the MPI specification makes it impossible to look inside 2521these types and see where the fields are. Therefore these wrappers 2522assume the types are laid out as <computeroutput>struct { float val; 2523int loc; }</computeroutput> (for 2524<computeroutput>MPI_FLOAT_INT</computeroutput>), etc, and act 2525accordingly. This appears to be correct at least for Open MPI 1.0.2 2526and for Quadrics MPI.</para> 2527 2528<para>If <computeroutput>strict</computeroutput> is an option specified 2529in <computeroutput>MPIWRAP_DEBUG</computeroutput>, the application 2530will abort if an unhandled type is encountered. Otherwise, the 2531application will print a warning message and continue.</para> 2532 2533<para>Some effort is made to mark/check memory ranges corresponding to 2534arrays of values in a single pass. This is important for performance 2535since asking Valgrind to mark/check any range, no matter how small, 2536carries quite a large constant cost. This optimisation is applied to 2537arrays of primitive types (<computeroutput>double</computeroutput>, 2538<computeroutput>float</computeroutput>, 2539<computeroutput>int</computeroutput>, 2540<computeroutput>long</computeroutput>, <computeroutput>long 2541long</computeroutput>, <computeroutput>short</computeroutput>, 2542<computeroutput>char</computeroutput>, and <computeroutput>long 2543double</computeroutput> on platforms where <computeroutput>sizeof(long 2544double) == 8</computeroutput>). For arrays of all other types, the 2545wrappers handle each element individually and so there can be a very 2546large performance cost.</para> 2547 2548</sect2> 2549 2550 2551<sect2 id="mc-manual.mpiwrap.writingwrappers" 2552 xreflabel="Writing new MPI Wrappers"> 2553<title>Writing new wrappers</title> 2554 2555<para> 2556For the most part the wrappers are straightforward. The only 2557significant complexity arises with nonblocking receives.</para> 2558 2559<para>The issue is that <computeroutput>MPI_Irecv</computeroutput> 2560states the recv buffer and returns immediately, giving a handle 2561(<computeroutput>MPI_Request</computeroutput>) for the transaction. 2562Later the user will have to poll for completion with 2563<computeroutput>MPI_Wait</computeroutput> etc, and when the 2564transaction completes successfully, the wrappers have to paint the 2565recv buffer. But the recv buffer details are not presented to 2566<computeroutput>MPI_Wait</computeroutput> -- only the handle is. The 2567library therefore maintains a shadow table which associates 2568uncompleted <computeroutput>MPI_Request</computeroutput>s with the 2569corresponding buffer address/count/type. When an operation completes, 2570the table is searched for the associated address/count/type info, and 2571memory is marked accordingly.</para> 2572 2573<para>Access to the table is guarded by a (POSIX pthreads) lock, so as 2574to make the library thread-safe.</para> 2575 2576<para>The table is allocated with 2577<computeroutput>malloc</computeroutput> and never 2578<computeroutput>free</computeroutput>d, so it will show up in leak 2579checks.</para> 2580 2581<para>Writing new wrappers should be fairly easy. The source file is 2582<computeroutput>mpi/libmpiwrap.c</computeroutput>. If possible, 2583find an existing wrapper for a function of similar behaviour to the 2584one you want to wrap, and use it as a starting point. The wrappers 2585are organised in sections in the same order as the MPI 1.1 spec, to 2586aid navigation. When adding a wrapper, remember to comment out the 2587definition of the default wrapper in the long list of defaults at the 2588bottom of the file (do not remove it, just comment it out).</para> 2589</sect2> 2590 2591<sect2 id="mc-manual.mpiwrap.whattoexpect" 2592 xreflabel="What to expect with MPI Wrappers"> 2593<title>What to expect when using the wrappers</title> 2594 2595<para>The wrappers should reduce Memcheck's false-error rate on MPI 2596applications. Because the wrapping is done at the MPI interface, 2597there will still potentially be a large number of errors reported in 2598the MPI implementation below the interface. The best you can do is 2599try to suppress them.</para> 2600 2601<para>You may also find that the input-side (buffer 2602length/definedness) checks find errors in your MPI use, for example 2603passing too short a buffer to 2604<computeroutput>MPI_Recv</computeroutput>.</para> 2605 2606<para>Functions which are not wrapped may increase the false 2607error rate. A possible approach is to run with 2608<computeroutput>MPI_DEBUG</computeroutput> containing 2609<computeroutput>warn</computeroutput>. This will show you functions 2610which lack proper wrappers but which are nevertheless used. You can 2611then write wrappers for them. 2612</para> 2613 2614<para>A known source of potential false errors are the 2615<computeroutput>PMPI_Reduce</computeroutput> family of functions, when 2616using a custom (user-defined) reduction function. In a reduction 2617operation, each node notionally sends data to a "central point" which 2618uses the specified reduction function to merge the data items into a 2619single item. Hence, in general, data is passed between nodes and fed 2620to the reduction function, but the wrapper library cannot mark the 2621transferred data as initialised before it is handed to the reduction 2622function, because all that happens "inside" the 2623<computeroutput>PMPI_Reduce</computeroutput> call. As a result you 2624may see false positives reported in your reduction function.</para> 2625 2626</sect2> 2627 2628</sect1> 2629 2630 2631 2632 2633 2634</chapter> 2635