1 2==================================================================== 3 414 October 2011 5 6Protocols 1 through 3 supported Memcheck only. Protocol 4 provides 7XML output for Memcheck, Helgrind, DRD and SGcheck. Technically there 8are four variants of Protocol 4, one for each tool, since they 9produce different errors. The four variants differ only in the 10definition of the ERROR nonterminal and are otherwise identical. 11 12NOTE that Protocol 4 (for the current svn trunk, which will eventually 13become 3.7.x) is still under development. The text herein should not 14be regarded as the final definition. 15 16 17Identification of Protocols 18~~~~~~~~~~~~~~~~~~~~~~~~~~~ 19 20In Protocols 1 through 3, a <protocolversion>INT<protocolversion> 21close to the start of the stream makes it possible for parsers to 22ascertain the version, so they can tell whether or not they can handle 23it. The presence of support for multiple tools brings a complication, 24though: it is not enough merely to state the protocol version -- the 25tool name must also be stated. Hence in Protocol 4, the 26<protocolversion>INT<protocolversion> is followed immediately by 27<protocoltool>TEXT</protocoltool>, to identify the tool. 28 29This duplicates the tool name present later in the preamble, but it 30was felt important to place the tool name right at the front along 31with the protocol number, for easy determination of parseability. 32 33 34How this specification is structured 35~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 36 37The TOPLEVEL nonterminal specifies top level XML output structure. It 38is common to all error producing tools. 39 40TOPLEVEL references TOOLSPECIFICs for each tool, and these are defined 41differently for each tool. Each TOOLSPECIFIC is an error, which is 42tool-specific. For Helgrind and DRD, a TOOLSPECIFIC may also contain a 43so-called thread-announcement record (described below). 44 45Overall there is a very high degree of format commonality between the 46three tools. Once a GUI is able to display the output correctly for 47one tool, it should be easy to extend it for the other two. 48 49 50Protocol 4 changes for Memcheck 51~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 52 53Protocol 4 for Memcheck is similar to Protocol 3, but has a number 54of changes to make it fit in the common framework: 55 56- the SUPPCOUNTS nonterminal now appears after the "Zero or more 57 ERRORs" block, and not before it. 58 59- the abovementioned "Zero or more ERRORs" block now becomes 60 "Zero or more of (either ERROR or ERRORCOUNTS)". 61 62- ERRORs for Memcheck may contain a SUPPRESSION field, which gives 63 the corresponding suppression for it. 64 65- ERRORs for Memcheck now use the XWHAT and XAUXWHAT nonterminals, as 66 well as WHAT and XWHAT. 67 68- The ad-hoc blocks <leakedbytes> and <leakedblocks> used by Memcheck 69 have been moved inside the XWHAT for the relevant error kinds. This 70 facilitates a common definition of ERROR across all three tools. 71 72The first two changes are required in order to correct a longstanding 73design flaw in the way Memcheck interacts with Valgrind's error 74management mechanism. See bug #186790 75(https://bugs.kde.org/show_bug.cgi?id=186790). The third change was 76requested in #191189 (https://bugs.kde.org/show_bug.cgi?id=191189). 77 78For GUI authors upgrading from Protocol 3 or earlier, the most 79significant new concept to grasp is the relationship between WHAT and 80XWHAT, and between AUXWHAT and XAUXWHAT. 81 82The definition of Protocol 4 now follows. It is structured similarly 83to that of the previous protocols, except that there is a separate 84definition of a nonterminal called TOOLSPECIFIC for each of Memcheck, 85Helgrind, DRD and SGcheck. The XWHAT and XAUXWHAT nonterminals also 86have tool-specific components. Apart from that, the structure is 87common to all supported tools. 88 89 90==================================================================== 91 92TOPLEVEL 93-------- 94 95The first line output is always this: 96 97 <?xml version="1.0"?> 98 99All remaining output is contained within the tag-pair 100<valgrindoutput>. 101 102Inside that, the first entity is an indication of the protocol 103version. This is provided so that existing parsers can identify XML 104created by future versions of Valgrind merely by observing that the 105protocol version is one they don't understand. Hence TOPLEVEL is: 106 107 <?xml version="1.0"?> 108 <valgrindoutput> 109 <protocolversion>INT<protocolversion> 110 <protocoltool>TEXT</protocoltool> 111 PROTOCOL 112 </valgrindoutput> 113 114Valgrind versions 3.0.0 and 3.0.1 emit protocol version 1. Versions 1153.1.X and 3.2.X [and 3.3.X ??] emit protocol version 2. 3.4.X emits 116protocol version 3. 3.5.X emits version 4. 117 118The TEXT in <protocoltool> is either "memcheck", "helgrind", "drd" or 119"exp-ptrcheck" and determines the allowed format of the ERROR 120nonterminal. Note that <protocoltool> is only present when the 121protocol version is 4 or above. 122 123 124PROTOCOL for version 4 125---------------------- 126 127This is the main top-level construction. Roughly speaking, it 128contains a preamble, a program-started marker, the errors from the run 129of the program, a program-ended marker, and any further errors 130resulting from post-run analysis (eg, memory leak detection). Hence 131the following in sequence: 132 133* Various preamble lines which give version info for the various 134 components. The text in them can be anything; it is not intended 135 for interpretation by the GUI: 136 137 <preamble> 138 <line>Misc version/copyright text</line> (zero or more of) 139 </preamble> 140 141* The PID of this process and of its parent: 142 143 <pid>INT</pid> 144 <ppid>INT</ppid> 145 146* The name of the tool being used: 147 148 <tool>TEXT</tool> 149 150 This can be anything, and it doesn't have to match the 151 <protocoltool> entry, although that might be wise. 152 153* Zero or more bindings of environment variable names to actual 154 values. These describe precisely the instantiations of %q format 155 specifiers used in the --xml-file= argument for the run, if any. 156 There is one <logfilequalifier> entry for each %q expanded: 157 158 <logfilequalifier> <var>VAR</var> <value>$VAR</value> 159 </logfilequalifier> 160 161* OPTIONALLY, if --xml-user-comment=STRING was given: 162 163 <usercomment>STRING</usercomment> 164 165 STRING is not escaped in any way, so that it itself may be a piece 166 of XML with arbitrary tags etc. 167 168* The program and args: first those pertaining to Valgrind itself, and 169 then those pertaining to the program to be run under Valgrind (the 170 client): 171 172 <args> 173 <vargv> 174 <exe>TEXT</exe> 175 <arg>TEXT</arg> (zero or more of) 176 </vargv> 177 <argv> 178 <exe>TEXT</exe> 179 <arg>TEXT</arg> (zero or more of) 180 </argv> 181 </args> 182 183* The following, indicating that the program has now started: 184 185 <status> <state>RUNNING</state> 186 <time>human-readable-time-string</time> 187 </status> 188 189 The format of this string is not defined, but it is expected to be 190 human-understandable. In current Valgrind versions it is the 191 elapsed wallclock time since process start. 192 193* Zero or more of (either ERRORCOUNTS, TOOLSPECIFIC, or CLIENTMSG). 194 195* The following, indicating that the program has now finished, and 196 that the any final wrapup (eg, for Memcheck, leak checking) is happening. 197 198 <status> <state>FINISHED</state> 199 <time>human-readable-time-string</time> 200 </status> 201 202* Zero or more of (either ERRORCOUNTS or TOOLSPECIFIC). In Memcheck's 203 case these will be complaints from the leak checker. For SGcheck 204 and Helgrind we don't expect any output here (but the spec does not 205 guarantee that either). 206 207* SUPPCOUNTS, indicating how many times each suppression was used. 208 209 210That's it. The tool-specific definitions for TOOLSPECIFIC are below; 211however let's first continue with some smaller nonterminals used in 212the construction of errors for all the tool types. 213 214 215==================================================================== 216 217Nonterminals used in construction of ERRORs 218------------------------------------------- 219 220STACK 221----- 222STACK indicates locations in the program being debugged. A STACK 223is one or more FRAMEs. The first is the innermost frame, the 224next its caller, etc. 225 226 <stack> 227 one or more FRAME 228 </stack> 229 230 231FRAME 232----- 233FRAME records a single program location: 234 235 <frame> 236 <ip>HEX64</ip> 237 optionally <obj>TEXT</obj> 238 optionally <fn>TEXT</fn> 239 optionally <dir>TEXT</dir> 240 optionally <file>TEXT</file> 241 optionally <line>INT</line> 242 </frame> 243 244Only the <ip> field is guaranteed to be present. It indicates a 245code ("instruction pointer") address. 246 247The optional fields, if present, appear in the order stated: 248 249* obj: gives the name of the ELF object containing the code address 250 251* fn: gives the name of the function containing the code address 252 253* dir: gives the source directory associated with the name specified 254 by <file>. Note the current implementation often does not 255 put anything useful in this field. 256 257* file: gives the name of the source file containing the code address 258 259* line: gives the line number in the source file 260 261 262ERRORCOUNTS 263----------- 264This specifies, for each error that has been so far presented, 265the number of occurrences of that error. 266 267 <errorcounts> 268 zero or more of 269 <pair> <count>INT</count> <unique>HEX64</unique> </pair> 270 </errorcounts> 271 272Each <pair> gives the current error count <count> for the error with 273unique tag </unique>. The counts do not have to give a count for each 274error so far presented - partial information is allowable. 275 276As at Valgrind rev 3793, error counts are only emitted at program 277termination. However, it is perfectly acceptable to periodically emit 278error counts as the program is running. Doing so would facilitate a 279GUI to dynamically update its error-count display as the program runs. 280 281 282SUPPCOUNTS 283---------- 284A SUPPCOUNTS block appears exactly once, after the program terminates. 285It specifies the number of times each error-suppression was used. 286Suppressions not mentioned were used zero times. 287 288 <suppcounts> 289 zero or more of 290 <pair> <count>INT</count> <name>TEXT</name> </pair> 291 </suppcounts> 292 293The <name> is as specified in the suppression name fields in .supp 294files. 295 296 297SUPPRESSION 298----------- 299These are optionally emitted as part of ERRORs, and specify the 300suppression that would be needed to suppress the containing error. 301For convenience, the suppression is presented twice, once in 302a structured nicely wrapped up in tags, and once as raw text 303suitable for direct copying and pasting into a suppressions file. 304 305 <suppression> 306 <sname>TEXT</sname> name of the suppression 307 <skind>TEXT</skind> kind, eg "Memcheck:Param" 308 <skaux>TEXT</skaux> (optional) aux kind, eg "write(buf)" 309 SFRAME (one or more) frames 310 <rawtext> CDATAS </rawtext> 311 </suppression> 312 313where CDATAS is a sequence of one or more <![CDATA[ .. ]]> blocks 314holding the raw text. Unfortunately, CDATA provides no way to escape 315the ending marker "]]>", which means that if the raw data contains 316such a sequence, it has to be split between two CDATA blocks, one 317ending with data "]]" and the other beginning with data "<". This is 318why the spec calls for one or more CDATA blocks rather than exactly 319one. 320 321Note that, so far, we cannot envisage a circumstance in which a 322generated suppression would contain the string "]]>", since neither 323"]" nor ">" appear to turn up in mangled symbol names. Hence it is 324not envisaged that there will ever be more than one CDATA block, and 325indeed the implementation as of Valgrind 3.5.0 will only ever generate 326one block (it ignores any possible escaping problems). Nevertheless 327the specification allows multiple blocks, as a matter of safety. 328 329 330SFRAME 331------ 332Either 333 334 <sframe> <obj>TEXT</obj> </sframe> 335 336eg denoting "obj:/usr/X11R6/lib*/libX11.so.6.2", or 337 338 <sframe> <fun>TEXT</fun> </sframe> 339 340eg denoting "fun:*libc_write" 341 342 343WHAT and XWHAT 344-------------- 345 346WHAT supplies a single line of text, which is a human-understandable, 347primary description of an error. 348 349XWHAT is an extended version of WHAT. It also contains a piece of 350text intended for human reading, but in addition may contain arbitrary 351other tagged data. This extra data is tool-specific. One of its 352purposes is to supply GUIs with links to other data in the sequence of 353TOOLSPECIFICs, that are associated with the error. Another purpose is 354wrap certain quantities (numbers, file names, etc) embedded in the 355message, so that the GUIs can get hold of them without having to parse 356the text itself. 357 358For example, we could get: 359 360 <what>Possible data race on address 0x12345678</what> 361 362or alternatively 363 364 <xwhat> 365 <text>Possible data race by thread #17 on address 0x12345678</text> 366 <threadid>17</threadid> 367 </xwhat> 368 369And presumably the <threadid>17</threadid> refers to some previously 370emitted entity in the stream of TOOLSPECIFICs for this tool. 371 372In an XWHAT, the <text> tag-pair is mandatory. GUIs which don't want 373to handle the extra fields can just ignore them and display the text 374part. In this way they have the option to present at least something 375useful to the user even in the case where the extra fields can't be 376handled, for whatever reason. 377 378A corollary of this is that the degenerate extended case 379 380 <xwhat> <text>T</text> </xwhat> 381 382is exactly equivalent to 383 384 <what>T</what> 385 386 387AUXWHAT and XAUXWHAT 388-------------------- 389 390AUXWHAT is exactly like WHAT: a single line of text. It provides 391additional, secondary description of an error, that should be shown to 392the user. 393 394XAUXWHAT relates to AUXWHAT in the same way XWHAT relates to WHAT: it 395wraps up extra tagged info along with the line of text that would be 396in the AUXWHAT. 397 398 399==================================================================== 400 401ERROR definition -- common structure 402------------------------------------ 403 404ERROR defines an error, and is the most complex nonterminal. For all 405of the tools, the structure is common, and always conforms to the 406following: 407 408 <error> 409 <unique>HEX64</unique> 410 <tid>INT</tid> 411 <threadname>NAME</threadname> if set 412 <kind>KIND</kind> 413 414 (either WHAT or XWHAT) 415 optionally: (either WHAT or XWHAT) 416 417 STACK 418 419 zero or more: (either AUXWHAT or XAUXWHAT or STACK) 420 421 optionally: SUPPRESSION 422 </error> 423 424 425* Each error contains a unique, arbitrary 64-bit hex number. This is 426 used to refer to the error in ERRORCOUNTS nonterminals (see above). 427 428* The <tid> tag indicates the Valgrind thread number. This value 429 is arbitrary but may be used to determine which threads produced 430 which errors (at least, the first instance of each error). 431 432* The <threadname> tag identifies the name of the thread if it was 433 set by the client application. If no name was set, the tag is 434 omitted. 435 436* The <kind> tag specifies one of a small number of fixed error types, 437 so that GUIs may roughly categorise errors by type if they want. 438 The tags themselves are tool-specific and are defined further 439 below, for each tool. 440 441* The "(either WHAT or XWHAT)" gives a primary description of the 442 error. WHAT and XWHAT are defined earlier in this file. Any XWHATs 443 appearing here may contain tool-specific subcomponents. 444 445* Optionally, a second line of primary description may be present. 446 447* A STACK gives the primary source location for the error. 448 449* There then follow zero or more of "(either AUXWHAT or XAUXWHAT or 450 STACK)". These give further (auxiliary) information about the 451 error, possibly including stack traces. They should be shown to the 452 user in the order they appear. AUXWHAT and XAUXWHAT are defined 453 earlier in this file. Any XAUXWHATs appearing here may contain 454 tool-specific subcomponents. 455 456* Optionally, as the last field, a SUPPRESSION may be provided. This 457 contains a suppression that would hide the error. 458 459 460==================================================================== 461 462TOOLSPECIFIC definition for Memcheck 463------------------------------------ 464 465For Memcheck, a TOOLSPECIFIC is simply an ERROR: 466 467TOOLSPECIFIC = ERROR 468 469 470ERROR details for Memcheck 471-------------------------- 472 473XWHATs (for definition, see above) may contain the following extra 474components (along with the mandatory <text>...</text> component): 475 476* <leakedbytes>INT</leakedbytes> 477 478* <leakedblocks>INT</leakedblocks> 479 480These fields are used in errors that have a <kind> tag specifying a 481KIND of the form "Leak_*", to indicate the number of leaked bytes and 482blocks. 483 484 485XAUXWHATs (for definition, see above) may contain the following extra 486components (along with the mandatory <text>...</text> component): 487 488* <file>TEXT</file>, as defined in FRAME 489 490* <line>INT</line>, as defined in FRAME 491 492* <dir>TEXT</dir>, as defined in FRAME 493 494 495KIND for Memcheck 496----------------- 497 498This is a small enumeration indicating roughly the nature of an error. 499The possible values are: 500 501 InvalidFree 502 503 free/delete/delete[] on an invalid pointer 504 505 MismatchedFree 506 507 free/delete/delete[] does not match allocation function 508 (eg doing new[] then free on the result) 509 510 InvalidRead 511 512 read of an invalid address 513 514 InvalidWrite 515 516 write of an invalid address 517 518 InvalidJump 519 520 jump to an invalid address 521 522 Overlap 523 524 args overlap other otherwise bogus in eg memcpy 525 526 InvalidMemPool 527 528 invalid mem pool specified in client request 529 530 UninitCondition 531 532 conditional jump/move depends on undefined value 533 534 UninitValue 535 536 other use of undefined value (primarily memory addresses) 537 538 SyscallParam 539 540 system call params are undefined or point to 541 undefined/unaddressible memory 542 543 ClientCheck 544 545 "error" resulting from a client check request 546 547 Leak_DefinitelyLost 548 549 memory leak; the referenced blocks are definitely lost 550 551 Leak_IndirectlyLost 552 553 memory leak; the referenced blocks are lost because all pointers 554 to them are also in leaked blocks 555 556 Leak_PossiblyLost 557 558 memory leak; only interior pointers to referenced blocks were 559 found 560 561 Leak_StillReachable 562 563 memory leak; pointers to un-freed blocks are still available 564 565 566==================================================================== 567 568TOOLSPECIFIC definition for SGcheck 569----------------------------------- 570 571For SGcheck, a TOOLSPECIFIC is simply an ERROR: 572 573TOOLSPECIFIC = ERROR 574 575 576ERROR details for SGcheck 577------------------------- 578 579SGcheck does not produce any XWHAT records, despite the fact that 580"ERROR definition -- common structure" says that tools may do so. 581 582 583XAUXWHATs (for definition, see above) may contain the following extra 584components (along with the mandatory <text>...</text> component): 585 586* <file>TEXT</file>, as defined in FRAME 587 588* <line>INT</line>, as defined in FRAME 589 590* <dir>TEXT</dir>, as defined in FRAME 591 592 593KIND for SGcheck 594---------------- 595This is a small enumeration indicating roughly the nature of an error. 596The possible values are: 597 598 SorG 599 600 Stack or global array inconsistency (roughly speaking, an 601 overrun of a stack or global array). The <auxwhat> blocks give 602 further details. 603 604 605==================================================================== 606 607TOOLSPECIFIC definition for Helgrind 608------------------------------------- 609 610For Helgrind, a TOOLSPECIFIC may be one of two things: 611 612TOOLSPECIFIC = either ERROR or ANNOUNCETHREAD 613 614 615ANNOUNCETHREAD 616-------------- 617 618The definition is 619 620 <announcethread> 621 <hthreadid>INT</hthreadid> 622 STACK 623 </announcethread> 624 625This states the creation point of a thread, and gives it a unique 626"hthreadid", which may be referred to in subsequent ERRORs. Note that 627 6281. The appearance of ANNOUNCETHREAD does not mean that the thread was 629 actually created at that point relative to any preceding or 630 following ERRORs in the output stream -- in general the thread will 631 have been created arbitrarily earlier. Helgrind only "announces" a 632 thread when it needs to refer to it for the first time, in a 633 subsequent ERROR. 634 6352. The "hthreadid" is a number which uniquely identifies the thread 636 for the run - no other thread will have the same hthreadid. The 637 hthreadid is a Helgrind-specific piece of information and is 638 unrelated to the <tid> fields in the common part of an ERROR. 639 Be careful not to confuse the two. 640 641 642ERROR details for Helgrind 643-------------------------- 644 645XWHATs (for definition, see above) may contain the following extra 646components (along with the mandatory <text>...</text> component): 647 648* <hthreadid>INT</hthreadid> fields. These refer to ANNOUNCETHREADs 649 appearing previously in the scheme, and state the creation points of 650 the thread(s) concerned in the ERROR. Hence it should be possible 651 for GUIs to show users stacks of the creation points of all threads 652 involved in each ERROR. 653 654 655XAUXWHATs (for definition, see above) may contain the following extra 656components (along with the mandatory <text>...</text> component): 657 658* <hthreadid>INT</hthreadid>, same meaning as when referred to in 659 XWHAT 660 661* <file>TEXT</file>, as defined in FRAME 662 663* <line>INT</line>, as defined in FRAME 664 665* <dir>TEXT</dir>, as defined in FRAME 666 667 668KIND for Helgrind 669----------------- 670This is a small enumeration indicating roughly the nature of an error. 671The possible values are: 672 673 Race 674 675 Data race. Helgrind will try to show the stacks for both 676 conflicting accesses if it can; it will always show the stack 677 for at least one of them. 678 679 UnlockUnlocked 680 681 Unlocking a not-locked lock 682 683 UnlockForeign 684 685 Unlocking a lock held by some other thread 686 687 UnlockBogus 688 689 Unlocking an address which is not known to be a lock 690 691 PthAPIerror 692 693 One of the POSIX pthread_ functions that are intercepted 694 by Helgrind, failed with an error code. Usually indicates 695 something bad happening. 696 697 LockOrder 698 699 An inconsistency in the acquisition order of locks was observed; 700 dangerous, as it can potentially lead to deadlocks 701 702 Misc 703 704 One of various miscellaneous noteworthy conditions was observed 705 (eg, thread exited whilst holding locks, "impossible" behaviour 706 from the underlying threading library, etc) 707 708==================================================================== 709 710CLIENTMSG 711 712CLIENTMSG defines a message that was caused by one of the following 713client requests: 714 715- VALGRIND_PRINTF 716- VALGRIND_PRINTF_BACKTRACE 717 718Definition: 719 720<clientmsg> 721 <tid>INT</tid> 722 <threadname>NAME</threadname> if set 723 <text>...</text> 724</clientmsg> 725 726OR 727 728<clientmsg> 729 <tid>INT</tid> 730 <threadname>NAME</threadname> if set 731 <text>...</text> 732 STACK 733</clientmsg> 734 735* The <tid> tag indicates the Valgrind thread number. 736 737* The <threadname> tag identifies the name of the thread if it was 738 set by the client application. If no name was set, the tag is 739 omitted. 740 741* The <text> tag indicates the message as specified in the client request 742 (properly translated to XML). 743 744* STACK is only present in case of VALGRIND_PRINTF_BACKTRACE. See above 745 for a definition of STACK. 746 747==================================================================== 748 749FATAL_SIGNAL 750 751FATAL_SIGNAL defines a message that was caused by a signal that killed them 752process. 753 754Definition: 755 756 <fatal_signal> 757 <tid>INT</tid> 758 <threadname>NAME</threadname> if set 759 760 <signo>INT</signo> 761 <signame>NAME</signame> 762 763 <sicode>INT</sicode> 764 <event>NAME</event> 765 <siaddr>ADDR</siaddr> 766 767 STACK 768 769 </fatal_signal> 770 771* The <tid> tag indicates the Valgrind thread number. This value 772 is arbitrary but may be used to determine which threads produced 773 which errors (at least, the first instance of each error). 774 775* The <threadname> tag identifies the name of the thread if it was 776 set by the client application. If no name was set, the tag is 777 omitted. 778 779* The <signo> tag indicates signo value from struct siginfo. 780 781* In <signame> tag there is the decoded name of signo. 782 783* The <sicode> tag contains the sicode from struct siginfo. 784 785* The <event> tag indicates the decoded name of the sicode. If sicode 786 has no name, the tag is omitted. 787 788* The <siaddr> tag indicates the address that is the reason 789 why the signal was triggered. This can be an unaligned pointer value or 790 just the address of not mapped memory that is accessed nevertheless. 791 If the signal reason is not related to an address, the tag is omitted. 792 793* STACK is defined above and shows where the thread was when it 794 caught the signal. When sending the signal to itself using raise, 795 then raise is visible in this stack. 796