1<html> 2<head> 3<title>pcre2pattern specification</title> 4</head> 5<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> 6<h1>pcre2pattern man page</h1> 7<p> 8Return to the <a href="index.html">PCRE2 index page</a>. 9</p> 10<p> 11This page is part of the PCRE2 HTML documentation. It was generated 12automatically from the original man page. If there is any nonsense in it, 13please consult the man page, in case the conversion went wrong. 14<br> 15<ul> 16<li><a name="TOC1" href="#SEC1">PCRE2 REGULAR EXPRESSION DETAILS</a> 17<li><a name="TOC2" href="#SEC2">SPECIAL START-OF-PATTERN ITEMS</a> 18<li><a name="TOC3" href="#SEC3">EBCDIC CHARACTER CODES</a> 19<li><a name="TOC4" href="#SEC4">CHARACTERS AND METACHARACTERS</a> 20<li><a name="TOC5" href="#SEC5">BACKSLASH</a> 21<li><a name="TOC6" href="#SEC6">CIRCUMFLEX AND DOLLAR</a> 22<li><a name="TOC7" href="#SEC7">FULL STOP (PERIOD, DOT) AND \N</a> 23<li><a name="TOC8" href="#SEC8">MATCHING A SINGLE CODE UNIT</a> 24<li><a name="TOC9" href="#SEC9">SQUARE BRACKETS AND CHARACTER CLASSES</a> 25<li><a name="TOC10" href="#SEC10">POSIX CHARACTER CLASSES</a> 26<li><a name="TOC11" href="#SEC11">COMPATIBILITY FEATURE FOR WORD BOUNDARIES</a> 27<li><a name="TOC12" href="#SEC12">VERTICAL BAR</a> 28<li><a name="TOC13" href="#SEC13">INTERNAL OPTION SETTING</a> 29<li><a name="TOC14" href="#SEC14">SUBPATTERNS</a> 30<li><a name="TOC15" href="#SEC15">DUPLICATE SUBPATTERN NUMBERS</a> 31<li><a name="TOC16" href="#SEC16">NAMED SUBPATTERNS</a> 32<li><a name="TOC17" href="#SEC17">REPETITION</a> 33<li><a name="TOC18" href="#SEC18">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a> 34<li><a name="TOC19" href="#SEC19">BACK REFERENCES</a> 35<li><a name="TOC20" href="#SEC20">ASSERTIONS</a> 36<li><a name="TOC21" href="#SEC21">CONDITIONAL SUBPATTERNS</a> 37<li><a name="TOC22" href="#SEC22">COMMENTS</a> 38<li><a name="TOC23" href="#SEC23">RECURSIVE PATTERNS</a> 39<li><a name="TOC24" href="#SEC24">SUBPATTERNS AS SUBROUTINES</a> 40<li><a name="TOC25" href="#SEC25">ONIGURUMA SUBROUTINE SYNTAX</a> 41<li><a name="TOC26" href="#SEC26">CALLOUTS</a> 42<li><a name="TOC27" href="#SEC27">BACKTRACKING CONTROL</a> 43<li><a name="TOC28" href="#SEC28">SEE ALSO</a> 44<li><a name="TOC29" href="#SEC29">AUTHOR</a> 45<li><a name="TOC30" href="#SEC30">REVISION</a> 46</ul> 47<br><a name="SEC1" href="#TOC1">PCRE2 REGULAR EXPRESSION DETAILS</a><br> 48<P> 49The syntax and semantics of the regular expressions that are supported by PCRE2 50are described in detail below. There is a quick-reference syntax summary in the 51<a href="pcre2syntax.html"><b>pcre2syntax</b></a> 52page. PCRE2 tries to match Perl syntax and semantics as closely as it can. 53PCRE2 also supports some alternative regular expression syntax (which does not 54conflict with the Perl syntax) in order to provide some compatibility with 55regular expressions in Python, .NET, and Oniguruma. 56</P> 57<P> 58Perl's regular expressions are described in its own documentation, and regular 59expressions in general are covered in a number of books, some of which have 60copious examples. Jeffrey Friedl's "Mastering Regular Expressions", published 61by O'Reilly, covers regular expressions in great detail. This description of 62PCRE2's regular expressions is intended as reference material. 63</P> 64<P> 65This document discusses the patterns that are supported by PCRE2 when its main 66matching function, <b>pcre2_match()</b>, is used. PCRE2 also has an alternative 67matching function, <b>pcre2_dfa_match()</b>, which matches using a different 68algorithm that is not Perl-compatible. Some of the features discussed below are 69not available when DFA matching is used. The advantages and disadvantages of 70the alternative function, and how it differs from the normal function, are 71discussed in the 72<a href="pcre2matching.html"><b>pcre2matching</b></a> 73page. 74</P> 75<br><a name="SEC2" href="#TOC1">SPECIAL START-OF-PATTERN ITEMS</a><br> 76<P> 77A number of options that can be passed to <b>pcre2_compile()</b> can also be set 78by special items at the start of a pattern. These are not Perl-compatible, but 79are provided to make these options accessible to pattern writers who are not 80able to change the program that processes the pattern. Any number of these 81items may appear, but they must all be together right at the start of the 82pattern string, and the letters must be in upper case. 83</P> 84<br><b> 85UTF support 86</b><br> 87<P> 88In the 8-bit and 16-bit PCRE2 libraries, characters may be coded either as 89single code units, or as multiple UTF-8 or UTF-16 code units. UTF-32 can be 90specified for the 32-bit library, in which case it constrains the character 91values to valid Unicode code points. To process UTF strings, PCRE2 must be 92built to include Unicode support (which is the default). When using UTF strings 93you must either call the compiling function with the PCRE2_UTF option, or the 94pattern must start with the special sequence (*UTF), which is equivalent to 95setting the relevant option. How setting a UTF mode affects pattern matching is 96mentioned in several places below. There is also a summary of features in the 97<a href="pcre2unicode.html"><b>pcre2unicode</b></a> 98page. 99</P> 100<P> 101Some applications that allow their users to supply patterns may wish to 102restrict them to non-UTF data for security reasons. If the PCRE2_NEVER_UTF 103option is passed to <b>pcre2_compile()</b>, (*UTF) is not allowed, and its 104appearance in a pattern causes an error. 105</P> 106<br><b> 107Unicode property support 108</b><br> 109<P> 110Another special sequence that may appear at the start of a pattern is (*UCP). 111This has the same effect as setting the PCRE2_UCP option: it causes sequences 112such as \d and \w to use Unicode properties to determine character types, 113instead of recognizing only characters with codes less than 256 via a lookup 114table. 115</P> 116<P> 117Some applications that allow their users to supply patterns may wish to 118restrict them for security reasons. If the PCRE2_NEVER_UCP option is passed to 119<b>pcre2_compile()</b>, (*UCP) is not allowed, and its appearance in a pattern 120causes an error. 121</P> 122<br><b> 123Locking out empty string matching 124</b><br> 125<P> 126Starting a pattern with (*NOTEMPTY) or (*NOTEMPTY_ATSTART) has the same effect 127as passing the PCRE2_NOTEMPTY or PCRE2_NOTEMPTY_ATSTART option to whichever 128matching function is subsequently called to match the pattern. These options 129lock out the matching of empty strings, either entirely, or only at the start 130of the subject. 131</P> 132<br><b> 133Disabling auto-possessification 134</b><br> 135<P> 136If a pattern starts with (*NO_AUTO_POSSESS), it has the same effect as setting 137the PCRE2_NO_AUTO_POSSESS option. This stops PCRE2 from making quantifiers 138possessive when what follows cannot match the repeated item. For example, by 139default a+b is treated as a++b. For more details, see the 140<a href="pcre2api.html"><b>pcre2api</b></a> 141documentation. 142</P> 143<br><b> 144Disabling start-up optimizations 145</b><br> 146<P> 147If a pattern starts with (*NO_START_OPT), it has the same effect as setting the 148PCRE2_NO_START_OPTIMIZE option. This disables several optimizations for quickly 149reaching "no match" results. For more details, see the 150<a href="pcre2api.html"><b>pcre2api</b></a> 151documentation. 152</P> 153<br><b> 154Disabling automatic anchoring 155</b><br> 156<P> 157If a pattern starts with (*NO_DOTSTAR_ANCHOR), it has the same effect as 158setting the PCRE2_NO_DOTSTAR_ANCHOR option. This disables optimizations that 159apply to patterns whose top-level branches all start with .* (match any number 160of arbitrary characters). For more details, see the 161<a href="pcre2api.html"><b>pcre2api</b></a> 162documentation. 163</P> 164<br><b> 165Disabling JIT compilation 166</b><br> 167<P> 168If a pattern that starts with (*NO_JIT) is successfully compiled, an attempt by 169the application to apply the JIT optimization by calling 170<b>pcre2_jit_compile()</b> is ignored. 171</P> 172<br><b> 173Setting match and recursion limits 174</b><br> 175<P> 176The caller of <b>pcre2_match()</b> can set a limit on the number of times the 177internal <b>match()</b> function is called and on the maximum depth of 178recursive calls. These facilities are provided to catch runaway matches that 179are provoked by patterns with huge matching trees (a typical example is a 180pattern with nested unlimited repeats) and to avoid running out of system stack 181by too much recursion. When one of these limits is reached, <b>pcre2_match()</b> 182gives an error return. The limits can also be set by items at the start of the 183pattern of the form 184<pre> 185 (*LIMIT_MATCH=d) 186 (*LIMIT_RECURSION=d) 187</pre> 188where d is any number of decimal digits. However, the value of the setting must 189be less than the value set (or defaulted) by the caller of <b>pcre2_match()</b> 190for it to have any effect. In other words, the pattern writer can lower the 191limits set by the programmer, but not raise them. If there is more than one 192setting of one of these limits, the lower value is used. 193<a name="newlines"></a></P> 194<br><b> 195Newline conventions 196</b><br> 197<P> 198PCRE2 supports five different conventions for indicating line breaks in 199strings: a single CR (carriage return) character, a single LF (linefeed) 200character, the two-character sequence CRLF, any of the three preceding, or any 201Unicode newline sequence. The 202<a href="pcre2api.html"><b>pcre2api</b></a> 203page has 204<a href="pcre2api.html#newlines">further discussion</a> 205about newlines, and shows how to set the newline convention when calling 206<b>pcre2_compile()</b>. 207</P> 208<P> 209It is also possible to specify a newline convention by starting a pattern 210string with one of the following five sequences: 211<pre> 212 (*CR) carriage return 213 (*LF) linefeed 214 (*CRLF) carriage return, followed by linefeed 215 (*ANYCRLF) any of the three above 216 (*ANY) all Unicode newline sequences 217</pre> 218These override the default and the options given to the compiling function. For 219example, on a Unix system where LF is the default newline sequence, the pattern 220<pre> 221 (*CR)a.b 222</pre> 223changes the convention to CR. That pattern matches "a\nb" because LF is no 224longer a newline. If more than one of these settings is present, the last one 225is used. 226</P> 227<P> 228The newline convention affects where the circumflex and dollar assertions are 229true. It also affects the interpretation of the dot metacharacter when 230PCRE2_DOTALL is not set, and the behaviour of \N. However, it does not affect 231what the \R escape sequence matches. By default, this is any Unicode newline 232sequence, for Perl compatibility. However, this can be changed; see the 233description of \R in the section entitled 234<a href="#newlineseq">"Newline sequences"</a> 235below. A change of \R setting can be combined with a change of newline 236convention. 237</P> 238<br><b> 239Specifying what \R matches 240</b><br> 241<P> 242It is possible to restrict \R to match only CR, LF, or CRLF (instead of the 243complete set of Unicode line endings) by setting the option PCRE2_BSR_ANYCRLF 244at compile time. This effect can also be achieved by starting a pattern with 245(*BSR_ANYCRLF). For completeness, (*BSR_UNICODE) is also recognized, 246corresponding to PCRE2_BSR_UNICODE. 247</P> 248<br><a name="SEC3" href="#TOC1">EBCDIC CHARACTER CODES</a><br> 249<P> 250PCRE2 can be compiled to run in an environment that uses EBCDIC as its 251character code rather than ASCII or Unicode (typically a mainframe system). In 252the sections below, character code values are ASCII or Unicode; in an EBCDIC 253environment these characters may have different code values, and there are no 254code points greater than 255. 255</P> 256<br><a name="SEC4" href="#TOC1">CHARACTERS AND METACHARACTERS</a><br> 257<P> 258A regular expression is a pattern that is matched against a subject string from 259left to right. Most characters stand for themselves in a pattern, and match the 260corresponding characters in the subject. As a trivial example, the pattern 261<pre> 262 The quick brown fox 263</pre> 264matches a portion of a subject string that is identical to itself. When 265caseless matching is specified (the PCRE2_CASELESS option), letters are matched 266independently of case. 267</P> 268<P> 269The power of regular expressions comes from the ability to include alternatives 270and repetitions in the pattern. These are encoded in the pattern by the use of 271<i>metacharacters</i>, which do not stand for themselves but instead are 272interpreted in some special way. 273</P> 274<P> 275There are two different sets of metacharacters: those that are recognized 276anywhere in the pattern except within square brackets, and those that are 277recognized within square brackets. Outside square brackets, the metacharacters 278are as follows: 279<pre> 280 \ general escape character with several uses 281 ^ assert start of string (or line, in multiline mode) 282 $ assert end of string (or line, in multiline mode) 283 . match any character except newline (by default) 284 [ start character class definition 285 | start of alternative branch 286 ( start subpattern 287 ) end subpattern 288 ? extends the meaning of ( 289 also 0 or 1 quantifier 290 also quantifier minimizer 291 * 0 or more quantifier 292 + 1 or more quantifier 293 also "possessive quantifier" 294 { start min/max quantifier 295</pre> 296Part of a pattern that is in square brackets is called a "character class". In 297a character class the only metacharacters are: 298<pre> 299 \ general escape character 300 ^ negate the class, but only if the first character 301 - indicates character range 302 [ POSIX character class (only if followed by POSIX syntax) 303 ] terminates the character class 304</pre> 305The following sections describe the use of each of the metacharacters. 306</P> 307<br><a name="SEC5" href="#TOC1">BACKSLASH</a><br> 308<P> 309The backslash character has several uses. Firstly, if it is followed by a 310character that is not a number or a letter, it takes away any special meaning 311that character may have. This use of backslash as an escape character applies 312both inside and outside character classes. 313</P> 314<P> 315For example, if you want to match a * character, you write \* in the pattern. 316This escaping action applies whether or not the following character would 317otherwise be interpreted as a metacharacter, so it is always safe to precede a 318non-alphanumeric with backslash to specify that it stands for itself. In 319particular, if you want to match a backslash, you write \\. 320</P> 321<P> 322In a UTF mode, only ASCII numbers and letters have any special meaning after a 323backslash. All other characters (in particular, those whose codepoints are 324greater than 127) are treated as literals. 325</P> 326<P> 327If a pattern is compiled with the PCRE2_EXTENDED option, most white space in 328the pattern (other than in a character class), and characters between a # 329outside a character class and the next newline, inclusive, are ignored. An 330escaping backslash can be used to include a white space or # character as part 331of the pattern. 332</P> 333<P> 334If you want to remove the special meaning from a sequence of characters, you 335can do so by putting them between \Q and \E. This is different from Perl in 336that $ and @ are handled as literals in \Q...\E sequences in PCRE2, whereas 337in Perl, $ and @ cause variable interpolation. Note the following examples: 338<pre> 339 Pattern PCRE2 matches Perl matches 340 341 \Qabc$xyz\E abc$xyz abc followed by the contents of $xyz 342 \Qabc\$xyz\E abc\$xyz abc\$xyz 343 \Qabc\E\$\Qxyz\E abc$xyz abc$xyz 344</pre> 345The \Q...\E sequence is recognized both inside and outside character classes. 346An isolated \E that is not preceded by \Q is ignored. If \Q is not followed 347by \E later in the pattern, the literal interpretation continues to the end of 348the pattern (that is, \E is assumed at the end). If the isolated \Q is inside 349a character class, this causes an error, because the character class is not 350terminated. 351<a name="digitsafterbackslash"></a></P> 352<br><b> 353Non-printing characters 354</b><br> 355<P> 356A second use of backslash provides a way of encoding non-printing characters 357in patterns in a visible manner. There is no restriction on the appearance of 358non-printing characters in a pattern, but when a pattern is being prepared by 359text editing, it is often easier to use one of the following escape sequences 360than the binary character it represents. In an ASCII or Unicode environment, 361these escapes are as follows: 362<pre> 363 \a alarm, that is, the BEL character (hex 07) 364 \cx "control-x", where x is any printable ASCII character 365 \e escape (hex 1B) 366 \f form feed (hex 0C) 367 \n linefeed (hex 0A) 368 \r carriage return (hex 0D) 369 \t tab (hex 09) 370 \0dd character with octal code 0dd 371 \ddd character with octal code ddd, or back reference 372 \o{ddd..} character with octal code ddd.. 373 \xhh character with hex code hh 374 \x{hhh..} character with hex code hhh.. (default mode) 375 \uhhhh character with hex code hhhh (when PCRE2_ALT_BSUX is set) 376</pre> 377The precise effect of \cx on ASCII characters is as follows: if x is a lower 378case letter, it is converted to upper case. Then bit 6 of the character (hex 37940) is inverted. Thus \cA to \cZ become hex 01 to hex 1A (A is 41, Z is 5A), 380but \c{ becomes hex 3B ({ is 7B), and \c; becomes hex 7B (; is 3B). If the 381code unit following \c has a value less than 32 or greater than 126, a 382compile-time error occurs. This locks out non-printable ASCII characters in all 383modes. 384</P> 385<P> 386When PCRE2 is compiled in EBCDIC mode, \a, \e, \f, \n, \r, and \t 387generate the appropriate EBCDIC code values. The \c escape is processed 388as specified for Perl in the <b>perlebcdic</b> document. The only characters 389that are allowed after \c are A-Z, a-z, or one of @, [, \, ], ^, _, or ?. Any 390other character provokes a compile-time error. The sequence \@ encodes 391character code 0; the letters (in either case) encode characters 1-26 (hex 01 392to hex 1A); [, \, ], ^, and _ encode characters 27-31 (hex 1B to hex 1F), and 393\? becomes either 255 (hex FF) or 95 (hex 5F). 394</P> 395<P> 396Thus, apart from \?, these escapes generate the same character code values as 397they do in an ASCII environment, though the meanings of the values mostly 398differ. For example, \G always generates code value 7, which is BEL in ASCII 399but DEL in EBCDIC. 400</P> 401<P> 402The sequence \? generates DEL (127, hex 7F) in an ASCII environment, but 403because 127 is not a control character in EBCDIC, Perl makes it generate the 404APC character. Unfortunately, there are several variants of EBCDIC. In most of 405them the APC character has the value 255 (hex FF), but in the one Perl calls 406POSIX-BC its value is 95 (hex 5F). If certain other characters have POSIX-BC 407values, PCRE2 makes \? generate 95; otherwise it generates 255. 408</P> 409<P> 410After \0 up to two further octal digits are read. If there are fewer than two 411digits, just those that are present are used. Thus the sequence \0\x\015 412specifies two binary zeros followed by a CR character (code value 13). Make 413sure you supply two digits after the initial zero if the pattern character that 414follows is itself an octal digit. 415</P> 416<P> 417The escape \o must be followed by a sequence of octal digits, enclosed in 418braces. An error occurs if this is not the case. This escape is a recent 419addition to Perl; it provides way of specifying character code points as octal 420numbers greater than 0777, and it also allows octal numbers and back references 421to be unambiguously specified. 422</P> 423<P> 424For greater clarity and unambiguity, it is best to avoid following \ by a 425digit greater than zero. Instead, use \o{} or \x{} to specify character 426numbers, and \g{} to specify back references. The following paragraphs 427describe the old, ambiguous syntax. 428</P> 429<P> 430The handling of a backslash followed by a digit other than 0 is complicated, 431and Perl has changed over time, causing PCRE2 also to change. 432</P> 433<P> 434Outside a character class, PCRE2 reads the digit and any following digits as a 435decimal number. If the number is less than 10, begins with the digit 8 or 9, or 436if there are at least that many previous capturing left parentheses in the 437expression, the entire sequence is taken as a <i>back reference</i>. A 438description of how this works is given 439<a href="#backreferences">later,</a> 440following the discussion of 441<a href="#subpattern">parenthesized subpatterns.</a> 442Otherwise, up to three octal digits are read to form a character code. 443</P> 444<P> 445Inside a character class, PCRE2 handles \8 and \9 as the literal characters 446"8" and "9", and otherwise reads up to three octal digits following the 447backslash, using them to generate a data character. Any subsequent digits stand 448for themselves. For example, outside a character class: 449<pre> 450 \040 is another way of writing an ASCII space 451 \40 is the same, provided there are fewer than 40 previous capturing subpatterns 452 \7 is always a back reference 453 \11 might be a back reference, or another way of writing a tab 454 \011 is always a tab 455 \0113 is a tab followed by the character "3" 456 \113 might be a back reference, otherwise the character with octal code 113 457 \377 might be a back reference, otherwise the value 255 (decimal) 458 \81 is always a back reference .sp 459</pre> 460Note that octal values of 100 or greater that are specified using this syntax 461must not be introduced by a leading zero, because no more than three octal 462digits are ever read. 463</P> 464<P> 465By default, after \x that is not followed by {, from zero to two hexadecimal 466digits are read (letters can be in upper or lower case). Any number of 467hexadecimal digits may appear between \x{ and }. If a character other than 468a hexadecimal digit appears between \x{ and }, or if there is no terminating 469}, an error occurs. 470</P> 471<P> 472If the PCRE2_ALT_BSUX option is set, the interpretation of \x is as just 473described only when it is followed by two hexadecimal digits. Otherwise, it 474matches a literal "x" character. In this mode mode, support for code points 475greater than 256 is provided by \u, which must be followed by four hexadecimal 476digits; otherwise it matches a literal "u" character. 477</P> 478<P> 479Characters whose value is less than 256 can be defined by either of the two 480syntaxes for \x (or by \u in PCRE2_ALT_BSUX mode). There is no difference in 481the way they are handled. For example, \xdc is exactly the same as \x{dc} (or 482\u00dc in PCRE2_ALT_BSUX mode). 483</P> 484<br><b> 485Constraints on character values 486</b><br> 487<P> 488Characters that are specified using octal or hexadecimal numbers are 489limited to certain values, as follows: 490<pre> 491 8-bit non-UTF mode less than 0x100 492 8-bit UTF-8 mode less than 0x10ffff and a valid codepoint 493 16-bit non-UTF mode less than 0x10000 494 16-bit UTF-16 mode less than 0x10ffff and a valid codepoint 495 32-bit non-UTF mode less than 0x100000000 496 32-bit UTF-32 mode less than 0x10ffff and a valid codepoint 497</pre> 498Invalid Unicode codepoints are the range 0xd800 to 0xdfff (the so-called 499"surrogate" codepoints), and 0xffef. 500</P> 501<br><b> 502Escape sequences in character classes 503</b><br> 504<P> 505All the sequences that define a single character value can be used both inside 506and outside character classes. In addition, inside a character class, \b is 507interpreted as the backspace character (hex 08). 508</P> 509<P> 510\N is not allowed in a character class. \B, \R, and \X are not special 511inside a character class. Like other unrecognized alphabetic escape sequences, 512they cause an error. Outside a character class, these sequences have different 513meanings. 514</P> 515<br><b> 516Unsupported escape sequences 517</b><br> 518<P> 519In Perl, the sequences \l, \L, \u, and \U are recognized by its string 520handler and used to modify the case of following characters. By default, PCRE2 521does not support these escape sequences. However, if the PCRE2_ALT_BSUX option 522is set, \U matches a "U" character, and \u can be used to define a character 523by code point, as described in the previous section. 524</P> 525<br><b> 526Absolute and relative back references 527</b><br> 528<P> 529The sequence \g followed by an unsigned or a negative number, optionally 530enclosed in braces, is an absolute or relative back reference. A named back 531reference can be coded as \g{name}. Back references are discussed 532<a href="#backreferences">later,</a> 533following the discussion of 534<a href="#subpattern">parenthesized subpatterns.</a> 535</P> 536<br><b> 537Absolute and relative subroutine calls 538</b><br> 539<P> 540For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or 541a number enclosed either in angle brackets or single quotes, is an alternative 542syntax for referencing a subpattern as a "subroutine". Details are discussed 543<a href="#onigurumasubroutines">later.</a> 544Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> 545synonymous. The former is a back reference; the latter is a 546<a href="#subpatternsassubroutines">subroutine</a> 547call. 548<a name="genericchartypes"></a></P> 549<br><b> 550Generic character types 551</b><br> 552<P> 553Another use of backslash is for specifying generic character types: 554<pre> 555 \d any decimal digit 556 \D any character that is not a decimal digit 557 \h any horizontal white space character 558 \H any character that is not a horizontal white space character 559 \s any white space character 560 \S any character that is not a white space character 561 \v any vertical white space character 562 \V any character that is not a vertical white space character 563 \w any "word" character 564 \W any "non-word" character 565</pre> 566There is also the single sequence \N, which matches a non-newline character. 567This is the same as 568<a href="#fullstopdot">the "." metacharacter</a> 569when PCRE2_DOTALL is not set. Perl also uses \N to match characters by name; 570PCRE2 does not support this. 571</P> 572<P> 573Each pair of lower and upper case escape sequences partitions the complete set 574of characters into two disjoint sets. Any given character matches one, and only 575one, of each pair. The sequences can appear both inside and outside character 576classes. They each match one character of the appropriate type. If the current 577matching point is at the end of the subject string, all of them fail, because 578there is no character to match. 579</P> 580<P> 581The default \s characters are HT (9), LF (10), VT (11), FF (12), CR (13), and 582space (32), which are defined as white space in the "C" locale. This list may 583vary if locale-specific matching is taking place. For example, in some locales 584the "non-breaking space" character (\xA0) is recognized as white space, and in 585others the VT character is not. 586</P> 587<P> 588A "word" character is an underscore or any character that is a letter or digit. 589By default, the definition of letters and digits is controlled by PCRE2's 590low-valued character tables, and may vary if locale-specific matching is taking 591place (see 592<a href="pcre2api.html#localesupport">"Locale support"</a> 593in the 594<a href="pcre2api.html"><b>pcre2api</b></a> 595page). For example, in a French locale such as "fr_FR" in Unix-like systems, 596or "french" in Windows, some character codes greater than 127 are used for 597accented letters, and these are then matched by \w. The use of locales with 598Unicode is discouraged. 599</P> 600<P> 601By default, characters whose code points are greater than 127 never match \d, 602\s, or \w, and always match \D, \S, and \W, although this may be different 603for characters in the range 128-255 when locale-specific matching is happening. 604These escape sequences retain their original meanings from before Unicode 605support was available, mainly for efficiency reasons. If the PCRE2_UCP option 606is set, the behaviour is changed so that Unicode properties are used to 607determine character types, as follows: 608<pre> 609 \d any character that matches \p{Nd} (decimal digit) 610 \s any character that matches \p{Z} or \h or \v 611 \w any character that matches \p{L} or \p{N}, plus underscore 612</pre> 613The upper case escapes match the inverse sets of characters. Note that \d 614matches only decimal digits, whereas \w matches any Unicode digit, as well as 615any Unicode letter, and underscore. Note also that PCRE2_UCP affects \b, and 616\B because they are defined in terms of \w and \W. Matching these sequences 617is noticeably slower when PCRE2_UCP is set. 618</P> 619<P> 620The sequences \h, \H, \v, and \V, in contrast to the other sequences, which 621match only ASCII characters by default, always match a specific list of code 622points, whether or not PCRE2_UCP is set. The horizontal space characters are: 623<pre> 624 U+0009 Horizontal tab (HT) 625 U+0020 Space 626 U+00A0 Non-break space 627 U+1680 Ogham space mark 628 U+180E Mongolian vowel separator 629 U+2000 En quad 630 U+2001 Em quad 631 U+2002 En space 632 U+2003 Em space 633 U+2004 Three-per-em space 634 U+2005 Four-per-em space 635 U+2006 Six-per-em space 636 U+2007 Figure space 637 U+2008 Punctuation space 638 U+2009 Thin space 639 U+200A Hair space 640 U+202F Narrow no-break space 641 U+205F Medium mathematical space 642 U+3000 Ideographic space 643</pre> 644The vertical space characters are: 645<pre> 646 U+000A Linefeed (LF) 647 U+000B Vertical tab (VT) 648 U+000C Form feed (FF) 649 U+000D Carriage return (CR) 650 U+0085 Next line (NEL) 651 U+2028 Line separator 652 U+2029 Paragraph separator 653</pre> 654In 8-bit, non-UTF-8 mode, only the characters with code points less than 256 655are relevant. 656<a name="newlineseq"></a></P> 657<br><b> 658Newline sequences 659</b><br> 660<P> 661Outside a character class, by default, the escape sequence \R matches any 662Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent to the 663following: 664<pre> 665 (?>\r\n|\n|\x0b|\f|\r|\x85) 666</pre> 667This is an example of an "atomic group", details of which are given 668<a href="#atomicgroup">below.</a> 669This particular group matches either the two-character sequence CR followed by 670LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, 671U+000B), FF (form feed, U+000C), CR (carriage return, U+000D), or NEL (next 672line, U+0085). Because this is an atomic group, the two-character sequence is 673treated as a single unit that cannot be split. 674</P> 675<P> 676In other modes, two additional characters whose codepoints are greater than 255 677are added: LS (line separator, U+2028) and PS (paragraph separator, U+2029). 678Unicode support is not needed for these characters to be recognized. 679</P> 680<P> 681It is possible to restrict \R to match only CR, LF, or CRLF (instead of the 682complete set of Unicode line endings) by setting the option PCRE2_BSR_ANYCRLF 683at compile time. (BSR is an abbrevation for "backslash R".) This can be made 684the default when PCRE2 is built; if this is the case, the other behaviour can 685be requested via the PCRE2_BSR_UNICODE option. It is also possible to specify 686these settings by starting a pattern string with one of the following 687sequences: 688<pre> 689 (*BSR_ANYCRLF) CR, LF, or CRLF only 690 (*BSR_UNICODE) any Unicode newline sequence 691</pre> 692These override the default and the options given to the compiling function. 693Note that these special settings, which are not Perl-compatible, are recognized 694only at the very start of a pattern, and that they must be in upper case. If 695more than one of them is present, the last one is used. They can be combined 696with a change of newline convention; for example, a pattern can start with: 697<pre> 698 (*ANY)(*BSR_ANYCRLF) 699</pre> 700They can also be combined with the (*UTF) or (*UCP) special sequences. Inside a 701character class, \R is treated as an unrecognized escape sequence, and causes 702an error. 703<a name="uniextseq"></a></P> 704<br><b> 705Unicode character properties 706</b><br> 707<P> 708When PCRE2 is built with Unicode support (the default), three additional escape 709sequences that match characters with specific properties are available. In 7108-bit non-UTF-8 mode, these sequences are of course limited to testing 711characters whose codepoints are less than 256, but they do work in this mode. 712The extra escape sequences are: 713<pre> 714 \p{<i>xx</i>} a character with the <i>xx</i> property 715 \P{<i>xx</i>} a character without the <i>xx</i> property 716 \X a Unicode extended grapheme cluster 717</pre> 718The property names represented by <i>xx</i> above are limited to the Unicode 719script names, the general category properties, "Any", which matches any 720character (including newline), and some special PCRE2 properties (described 721in the 722<a href="#extraprops">next section).</a> 723Other Perl properties such as "InMusicalSymbols" are not supported by PCRE2. 724Note that \P{Any} does not match any characters, so always causes a match 725failure. 726</P> 727<P> 728Sets of Unicode characters are defined as belonging to certain scripts. A 729character from one of these sets can be matched using a script name. For 730example: 731<pre> 732 \p{Greek} 733 \P{Han} 734</pre> 735Those that are not part of an identified script are lumped together as 736"Common". The current list of scripts is: 737</P> 738<P> 739Ahom, 740Anatolian_Hieroglyphs, 741Arabic, 742Armenian, 743Avestan, 744Balinese, 745Bamum, 746Bassa_Vah, 747Batak, 748Bengali, 749Bopomofo, 750Brahmi, 751Braille, 752Buginese, 753Buhid, 754Canadian_Aboriginal, 755Carian, 756Caucasian_Albanian, 757Chakma, 758Cham, 759Cherokee, 760Common, 761Coptic, 762Cuneiform, 763Cypriot, 764Cyrillic, 765Deseret, 766Devanagari, 767Duployan, 768Egyptian_Hieroglyphs, 769Elbasan, 770Ethiopic, 771Georgian, 772Glagolitic, 773Gothic, 774Grantha, 775Greek, 776Gujarati, 777Gurmukhi, 778Han, 779Hangul, 780Hanunoo, 781Hatran, 782Hebrew, 783Hiragana, 784Imperial_Aramaic, 785Inherited, 786Inscriptional_Pahlavi, 787Inscriptional_Parthian, 788Javanese, 789Kaithi, 790Kannada, 791Katakana, 792Kayah_Li, 793Kharoshthi, 794Khmer, 795Khojki, 796Khudawadi, 797Lao, 798Latin, 799Lepcha, 800Limbu, 801Linear_A, 802Linear_B, 803Lisu, 804Lycian, 805Lydian, 806Mahajani, 807Malayalam, 808Mandaic, 809Manichaean, 810Meetei_Mayek, 811Mende_Kikakui, 812Meroitic_Cursive, 813Meroitic_Hieroglyphs, 814Miao, 815Modi, 816Mongolian, 817Mro, 818Multani, 819Myanmar, 820Nabataean, 821New_Tai_Lue, 822Nko, 823Ogham, 824Ol_Chiki, 825Old_Hungarian, 826Old_Italic, 827Old_North_Arabian, 828Old_Permic, 829Old_Persian, 830Old_South_Arabian, 831Old_Turkic, 832Oriya, 833Osmanya, 834Pahawh_Hmong, 835Palmyrene, 836Pau_Cin_Hau, 837Phags_Pa, 838Phoenician, 839Psalter_Pahlavi, 840Rejang, 841Runic, 842Samaritan, 843Saurashtra, 844Sharada, 845Shavian, 846Siddham, 847SignWriting, 848Sinhala, 849Sora_Sompeng, 850Sundanese, 851Syloti_Nagri, 852Syriac, 853Tagalog, 854Tagbanwa, 855Tai_Le, 856Tai_Tham, 857Tai_Viet, 858Takri, 859Tamil, 860Telugu, 861Thaana, 862Thai, 863Tibetan, 864Tifinagh, 865Tirhuta, 866Ugaritic, 867Vai, 868Warang_Citi, 869Yi. 870</P> 871<P> 872Each character has exactly one Unicode general category property, specified by 873a two-letter abbreviation. For compatibility with Perl, negation can be 874specified by including a circumflex between the opening brace and the property 875name. For example, \p{^Lu} is the same as \P{Lu}. 876</P> 877<P> 878If only one letter is specified with \p or \P, it includes all the general 879category properties that start with that letter. In this case, in the absence 880of negation, the curly brackets in the escape sequence are optional; these two 881examples have the same effect: 882<pre> 883 \p{L} 884 \pL 885</pre> 886The following general category property codes are supported: 887<pre> 888 C Other 889 Cc Control 890 Cf Format 891 Cn Unassigned 892 Co Private use 893 Cs Surrogate 894 895 L Letter 896 Ll Lower case letter 897 Lm Modifier letter 898 Lo Other letter 899 Lt Title case letter 900 Lu Upper case letter 901 902 M Mark 903 Mc Spacing mark 904 Me Enclosing mark 905 Mn Non-spacing mark 906 907 N Number 908 Nd Decimal number 909 Nl Letter number 910 No Other number 911 912 P Punctuation 913 Pc Connector punctuation 914 Pd Dash punctuation 915 Pe Close punctuation 916 Pf Final punctuation 917 Pi Initial punctuation 918 Po Other punctuation 919 Ps Open punctuation 920 921 S Symbol 922 Sc Currency symbol 923 Sk Modifier symbol 924 Sm Mathematical symbol 925 So Other symbol 926 927 Z Separator 928 Zl Line separator 929 Zp Paragraph separator 930 Zs Space separator 931</pre> 932The special property L& is also supported: it matches a character that has 933the Lu, Ll, or Lt property, in other words, a letter that is not classified as 934a modifier or "other". 935</P> 936<P> 937The Cs (Surrogate) property applies only to characters in the range U+D800 to 938U+DFFF. Such characters are not valid in Unicode strings and so 939cannot be tested by PCRE2, unless UTF validity checking has been turned off 940(see the discussion of PCRE2_NO_UTF_CHECK in the 941<a href="pcre2api.html"><b>pcre2api</b></a> 942page). Perl does not support the Cs property. 943</P> 944<P> 945The long synonyms for property names that Perl supports (such as \p{Letter}) 946are not supported by PCRE2, nor is it permitted to prefix any of these 947properties with "Is". 948</P> 949<P> 950No character that is in the Unicode table has the Cn (unassigned) property. 951Instead, this property is assumed for any code point that is not in the 952Unicode table. 953</P> 954<P> 955Specifying caseless matching does not affect these escape sequences. For 956example, \p{Lu} always matches only upper case letters. This is different from 957the behaviour of current versions of Perl. 958</P> 959<P> 960Matching characters by Unicode property is not fast, because PCRE2 has to do a 961multistage table lookup in order to find a character's property. That is why 962the traditional escape sequences such as \d and \w do not use Unicode 963properties in PCRE2 by default, though you can make them do so by setting the 964PCRE2_UCP option or by starting the pattern with (*UCP). 965</P> 966<br><b> 967Extended grapheme clusters 968</b><br> 969<P> 970The \X escape matches any number of Unicode characters that form an "extended 971grapheme cluster", and treats the sequence as an atomic group 972<a href="#atomicgroup">(see below).</a> 973Unicode supports various kinds of composite character by giving each character 974a grapheme breaking property, and having rules that use these properties to 975define the boundaries of extended grapheme clusters. \X always matches at 976least one character. Then it decides whether to add additional characters 977according to the following rules for ending a cluster: 978</P> 979<P> 9801. End at the end of the subject string. 981</P> 982<P> 9832. Do not end between CR and LF; otherwise end after any control character. 984</P> 985<P> 9863. Do not break Hangul (a Korean script) syllable sequences. Hangul characters 987are of five types: L, V, T, LV, and LVT. An L character may be followed by an 988L, V, LV, or LVT character; an LV or V character may be followed by a V or T 989character; an LVT or T character may be follwed only by a T character. 990</P> 991<P> 9924. Do not end before extending characters or spacing marks. Characters with 993the "mark" property always have the "extend" grapheme breaking property. 994</P> 995<P> 9965. Do not end after prepend characters. 997</P> 998<P> 9996. Otherwise, end the cluster. 1000<a name="extraprops"></a></P> 1001<br><b> 1002PCRE2's additional properties 1003</b><br> 1004<P> 1005As well as the standard Unicode properties described above, PCRE2 supports four 1006more that make it possible to convert traditional escape sequences such as \w 1007and \s to use Unicode properties. PCRE2 uses these non-standard, non-Perl 1008properties internally when PCRE2_UCP is set. However, they may also be used 1009explicitly. These properties are: 1010<pre> 1011 Xan Any alphanumeric character 1012 Xps Any POSIX space character 1013 Xsp Any Perl space character 1014 Xwd Any Perl "word" character 1015</pre> 1016Xan matches characters that have either the L (letter) or the N (number) 1017property. Xps matches the characters tab, linefeed, vertical tab, form feed, or 1018carriage return, and any other character that has the Z (separator) property. 1019Xsp is the same as Xps; in PCRE1 it used to exclude vertical tab, for Perl 1020compatibility, but Perl changed. Xwd matches the same characters as Xan, plus 1021underscore. 1022</P> 1023<P> 1024There is another non-standard property, Xuc, which matches any character that 1025can be represented by a Universal Character Name in C++ and other programming 1026languages. These are the characters $, @, ` (grave accent), and all characters 1027with Unicode code points greater than or equal to U+00A0, except for the 1028surrogates U+D800 to U+DFFF. Note that most base (ASCII) characters are 1029excluded. (Universal Character Names are of the form \uHHHH or \UHHHHHHHH 1030where H is a hexadecimal digit. Note that the Xuc property does not match these 1031sequences but the characters that they represent.) 1032<a name="resetmatchstart"></a></P> 1033<br><b> 1034Resetting the match start 1035</b><br> 1036<P> 1037The escape sequence \K causes any previously matched characters not to be 1038included in the final matched sequence. For example, the pattern: 1039<pre> 1040 foo\Kbar 1041</pre> 1042matches "foobar", but reports that it has matched "bar". This feature is 1043similar to a lookbehind assertion 1044<a href="#lookbehind">(described below).</a> 1045However, in this case, the part of the subject before the real match does not 1046have to be of fixed length, as lookbehind assertions do. The use of \K does 1047not interfere with the setting of 1048<a href="#subpattern">captured substrings.</a> 1049For example, when the pattern 1050<pre> 1051 (foo)\Kbar 1052</pre> 1053matches "foobar", the first substring is still set to "foo". 1054</P> 1055<P> 1056Perl documents that the use of \K within assertions is "not well defined". In 1057PCRE2, \K is acted upon when it occurs inside positive assertions, but is 1058ignored in negative assertions. Note that when a pattern such as (?=ab\K) 1059matches, the reported start of the match can be greater than the end of the 1060match. 1061<a name="smallassertions"></a></P> 1062<br><b> 1063Simple assertions 1064</b><br> 1065<P> 1066The final use of backslash is for certain simple assertions. An assertion 1067specifies a condition that has to be met at a particular point in a match, 1068without consuming any characters from the subject string. The use of 1069subpatterns for more complicated assertions is described 1070<a href="#bigassertions">below.</a> 1071The backslashed assertions are: 1072<pre> 1073 \b matches at a word boundary 1074 \B matches when not at a word boundary 1075 \A matches at the start of the subject 1076 \Z matches at the end of the subject 1077 also matches before a newline at the end of the subject 1078 \z matches only at the end of the subject 1079 \G matches at the first matching position in the subject 1080</pre> 1081Inside a character class, \b has a different meaning; it matches the backspace 1082character. If any other of these assertions appears in a character class, an 1083"invalid escape sequence" error is generated. 1084</P> 1085<P> 1086A word boundary is a position in the subject string where the current character 1087and the previous character do not both match \w or \W (i.e. one matches 1088\w and the other matches \W), or the start or end of the string if the 1089first or last character matches \w, respectively. In a UTF mode, the meanings 1090of \w and \W can be changed by setting the PCRE2_UCP option. When this is 1091done, it also affects \b and \B. Neither PCRE2 nor Perl has a separate "start 1092of word" or "end of word" metasequence. However, whatever follows \b normally 1093determines which it is. For example, the fragment \ba matches "a" at the start 1094of a word. 1095</P> 1096<P> 1097The \A, \Z, and \z assertions differ from the traditional circumflex and 1098dollar (described in the next section) in that they only ever match at the very 1099start and end of the subject string, whatever options are set. Thus, they are 1100independent of multiline mode. These three assertions are not affected by the 1101PCRE2_NOTBOL or PCRE2_NOTEOL options, which affect only the behaviour of the 1102circumflex and dollar metacharacters. However, if the <i>startoffset</i> 1103argument of <b>pcre2_match()</b> is non-zero, indicating that matching is to 1104start at a point other than the beginning of the subject, \A can never match. 1105The difference between \Z and \z is that \Z matches before a newline at the 1106end of the string as well as at the very end, whereas \z matches only at the 1107end. 1108</P> 1109<P> 1110The \G assertion is true only when the current matching position is at the 1111start point of the match, as specified by the <i>startoffset</i> argument of 1112<b>pcre2_match()</b>. It differs from \A when the value of <i>startoffset</i> is 1113non-zero. By calling <b>pcre2_match()</b> multiple times with appropriate 1114arguments, you can mimic Perl's /g option, and it is in this kind of 1115implementation where \G can be useful. 1116</P> 1117<P> 1118Note, however, that PCRE2's interpretation of \G, as the start of the current 1119match, is subtly different from Perl's, which defines it as the end of the 1120previous match. In Perl, these can be different when the previously matched 1121string was empty. Because PCRE2 does just one match at a time, it cannot 1122reproduce this behaviour. 1123</P> 1124<P> 1125If all the alternatives of a pattern begin with \G, the expression is anchored 1126to the starting match position, and the "anchored" flag is set in the compiled 1127regular expression. 1128</P> 1129<br><a name="SEC6" href="#TOC1">CIRCUMFLEX AND DOLLAR</a><br> 1130<P> 1131The circumflex and dollar metacharacters are zero-width assertions. That is, 1132they test for a particular condition being true without consuming any 1133characters from the subject string. These two metacharacters are concerned with 1134matching the starts and ends of lines. If the newline convention is set so that 1135only the two-character sequence CRLF is recognized as a newline, isolated CR 1136and LF characters are treated as ordinary data characters, and are not 1137recognized as newlines. 1138</P> 1139<P> 1140Outside a character class, in the default matching mode, the circumflex 1141character is an assertion that is true only if the current matching point is at 1142the start of the subject string. If the <i>startoffset</i> argument of 1143<b>pcre2_match()</b> is non-zero, or if PCRE2_NOTBOL is set, circumflex can 1144never match if the PCRE2_MULTILINE option is unset. Inside a character class, 1145circumflex has an entirely different meaning 1146<a href="#characterclass">(see below).</a> 1147</P> 1148<P> 1149Circumflex need not be the first character of the pattern if a number of 1150alternatives are involved, but it should be the first thing in each alternative 1151in which it appears if the pattern is ever to match that branch. If all 1152possible alternatives start with a circumflex, that is, if the pattern is 1153constrained to match only at the start of the subject, it is said to be an 1154"anchored" pattern. (There are also other constructs that can cause a pattern 1155to be anchored.) 1156</P> 1157<P> 1158The dollar character is an assertion that is true only if the current matching 1159point is at the end of the subject string, or immediately before a newline at 1160the end of the string (by default), unless PCRE2_NOTEOL is set. Note, however, 1161that it does not actually match the newline. Dollar need not be the last 1162character of the pattern if a number of alternatives are involved, but it 1163should be the last item in any branch in which it appears. Dollar has no 1164special meaning in a character class. 1165</P> 1166<P> 1167The meaning of dollar can be changed so that it matches only at the very end of 1168the string, by setting the PCRE2_DOLLAR_ENDONLY option at compile time. This 1169does not affect the \Z assertion. 1170</P> 1171<P> 1172The meanings of the circumflex and dollar metacharacters are changed if the 1173PCRE2_MULTILINE option is set. When this is the case, a dollar character 1174matches before any newlines in the string, as well as at the very end, and a 1175circumflex matches immediately after internal newlines as well as at the start 1176of the subject string. It does not match after a newline that ends the string, 1177for compatibility with Perl. However, this can be changed by setting the 1178PCRE2_ALT_CIRCUMFLEX option. 1179</P> 1180<P> 1181For example, the pattern /^abc$/ matches the subject string "def\nabc" (where 1182\n represents a newline) in multiline mode, but not otherwise. Consequently, 1183patterns that are anchored in single line mode because all branches start with 1184^ are not anchored in multiline mode, and a match for circumflex is possible 1185when the <i>startoffset</i> argument of <b>pcre2_match()</b> is non-zero. The 1186PCRE2_DOLLAR_ENDONLY option is ignored if PCRE2_MULTILINE is set. 1187</P> 1188<P> 1189When the newline convention (see 1190<a href="#newlines">"Newline conventions"</a> 1191below) recognizes the two-character sequence CRLF as a newline, this is 1192preferred, even if the single characters CR and LF are also recognized as 1193newlines. For example, if the newline convention is "any", a multiline mode 1194circumflex matches before "xyz" in the string "abc\r\nxyz" rather than after 1195CR, even though CR on its own is a valid newline. (It also matches at the very 1196start of the string, of course.) 1197</P> 1198<P> 1199Note that the sequences \A, \Z, and \z can be used to match the start and 1200end of the subject in both modes, and if all branches of a pattern start with 1201\A it is always anchored, whether or not PCRE2_MULTILINE is set. 1202<a name="fullstopdot"></a></P> 1203<br><a name="SEC7" href="#TOC1">FULL STOP (PERIOD, DOT) AND \N</a><br> 1204<P> 1205Outside a character class, a dot in the pattern matches any one character in 1206the subject string except (by default) a character that signifies the end of a 1207line. 1208</P> 1209<P> 1210When a line ending is defined as a single character, dot never matches that 1211character; when the two-character sequence CRLF is used, dot does not match CR 1212if it is immediately followed by LF, but otherwise it matches all characters 1213(including isolated CRs and LFs). When any Unicode line endings are being 1214recognized, dot does not match CR or LF or any of the other line ending 1215characters. 1216</P> 1217<P> 1218The behaviour of dot with regard to newlines can be changed. If the 1219PCRE2_DOTALL option is set, a dot matches any one character, without exception. 1220If the two-character sequence CRLF is present in the subject string, it takes 1221two dots to match it. 1222</P> 1223<P> 1224The handling of dot is entirely independent of the handling of circumflex and 1225dollar, the only relationship being that they both involve newlines. Dot has no 1226special meaning in a character class. 1227</P> 1228<P> 1229The escape sequence \N behaves like a dot, except that it is not affected by 1230the PCRE2_DOTALL option. In other words, it matches any character except one 1231that signifies the end of a line. Perl also uses \N to match characters by 1232name; PCRE2 does not support this. 1233</P> 1234<br><a name="SEC8" href="#TOC1">MATCHING A SINGLE CODE UNIT</a><br> 1235<P> 1236Outside a character class, the escape sequence \C matches any one code unit, 1237whether or not a UTF mode is set. In the 8-bit library, one code unit is one 1238byte; in the 16-bit library it is a 16-bit unit; in the 32-bit library it is a 123932-bit unit. Unlike a dot, \C always matches line-ending characters. The 1240feature is provided in Perl in order to match individual bytes in UTF-8 mode, 1241but it is unclear how it can usefully be used. 1242</P> 1243<P> 1244Because \C breaks up characters into individual code units, matching one unit 1245with \C in UTF-8 or UTF-16 mode means that the rest of the string may start 1246with a malformed UTF character. This has undefined results, because PCRE2 1247assumes that it is matching character by character in a valid UTF string (by 1248default it checks the subject string's validity at the start of processing 1249unless the PCRE2_NO_UTF_CHECK option is used). 1250</P> 1251<P> 1252An application can lock out the use of \C by setting the 1253PCRE2_NEVER_BACKSLASH_C option when compiling a pattern. It is also possible to 1254build PCRE2 with the use of \C permanently disabled. 1255</P> 1256<P> 1257PCRE2 does not allow \C to appear in lookbehind assertions 1258<a href="#lookbehind">(described below)</a> 1259in UTF-8 or UTF-16 modes, because this would make it impossible to calculate 1260the length of the lookbehind. Neither the alternative matching function 1261<b>pcre2_dfa_match()</b> nor the JIT optimizer support \C in these UTF modes. 1262The former gives a match-time error; the latter fails to optimize and so the 1263match is always run using the interpreter. 1264</P> 1265<P> 1266In the 32-bit library, however, \C is always supported (when not explicitly 1267locked out) because it always matches a single code unit, whether or not UTF-32 1268is specified. 1269</P> 1270<P> 1271In general, the \C escape sequence is best avoided. However, one way of using 1272it that avoids the problem of malformed UTF-8 or UTF-16 characters is to use a 1273lookahead to check the length of the next character, as in this pattern, which 1274could be used with a UTF-8 string (ignore white space and line breaks): 1275<pre> 1276 (?| (?=[\x00-\x7f])(\C) | 1277 (?=[\x80-\x{7ff}])(\C)(\C) | 1278 (?=[\x{800}-\x{ffff}])(\C)(\C)(\C) | 1279 (?=[\x{10000}-\x{1fffff}])(\C)(\C)(\C)(\C)) 1280</pre> 1281In this example, a group that starts with (?| resets the capturing parentheses 1282numbers in each alternative (see 1283<a href="#dupsubpatternnumber">"Duplicate Subpattern Numbers"</a> 1284below). The assertions at the start of each branch check the next UTF-8 1285character for values whose encoding uses 1, 2, 3, or 4 bytes, respectively. The 1286character's individual bytes are then captured by the appropriate number of 1287\C groups. 1288<a name="characterclass"></a></P> 1289<br><a name="SEC9" href="#TOC1">SQUARE BRACKETS AND CHARACTER CLASSES</a><br> 1290<P> 1291An opening square bracket introduces a character class, terminated by a closing 1292square bracket. A closing square bracket on its own is not special by default. 1293If a closing square bracket is required as a member of the class, it should be 1294the first data character in the class (after an initial circumflex, if present) 1295or escaped with a backslash. This means that, by default, an empty class cannot 1296be defined. However, if the PCRE2_ALLOW_EMPTY_CLASS option is set, a closing 1297square bracket at the start does end the (empty) class. 1298</P> 1299<P> 1300A character class matches a single character in the subject. A matched 1301character must be in the set of characters defined by the class, unless the 1302first character in the class definition is a circumflex, in which case the 1303subject character must not be in the set defined by the class. If a circumflex 1304is actually required as a member of the class, ensure it is not the first 1305character, or escape it with a backslash. 1306</P> 1307<P> 1308For example, the character class [aeiou] matches any lower case vowel, while 1309[^aeiou] matches any character that is not a lower case vowel. Note that a 1310circumflex is just a convenient notation for specifying the characters that 1311are in the class by enumerating those that are not. A class that starts with a 1312circumflex is not an assertion; it still consumes a character from the subject 1313string, and therefore it fails if the current pointer is at the end of the 1314string. 1315</P> 1316<P> 1317When caseless matching is set, any letters in a class represent both their 1318upper case and lower case versions, so for example, a caseless [aeiou] matches 1319"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a 1320caseful version would. 1321</P> 1322<P> 1323Characters that might indicate line breaks are never treated in any special way 1324when matching character classes, whatever line-ending sequence is in use, and 1325whatever setting of the PCRE2_DOTALL and PCRE2_MULTILINE options is used. A 1326class such as [^a] always matches one of these characters. 1327</P> 1328<P> 1329The minus (hyphen) character can be used to specify a range of characters in a 1330character class. For example, [d-m] matches any letter between d and m, 1331inclusive. If a minus character is required in a class, it must be escaped with 1332a backslash or appear in a position where it cannot be interpreted as 1333indicating a range, typically as the first or last character in the class, or 1334immediately after a range. For example, [b-d-z] matches letters in the range b 1335to d, a hyphen character, or z. 1336</P> 1337<P> 1338It is not possible to have the literal character "]" as the end character of a 1339range. A pattern such as [W-]46] is interpreted as a class of two characters 1340("W" and "-") followed by a literal string "46]", so it would match "W46]" or 1341"-46]". However, if the "]" is escaped with a backslash it is interpreted as 1342the end of range, so [W-\]46] is interpreted as a class containing a range 1343followed by two other characters. The octal or hexadecimal representation of 1344"]" can also be used to end a range. 1345</P> 1346<P> 1347An error is generated if a POSIX character class (see below) or an escape 1348sequence other than one that defines a single character appears at a point 1349where a range ending character is expected. For example, [z-\xff] is valid, 1350but [A-\d] and [A-[:digit:]] are not. 1351</P> 1352<P> 1353Ranges normally include all code points between the start and end characters, 1354inclusive. They can also be used for code points specified numerically, for 1355example [\000-\037]. Ranges can include any characters that are valid for the 1356current mode. 1357</P> 1358<P> 1359There is a special case in EBCDIC environments for ranges whose end points are 1360both specified as literal letters in the same case. For compatibility with 1361Perl, EBCDIC code points within the range that are not letters are omitted. For 1362example, [h-k] matches only four characters, even though the codes for h and k 1363are 0x88 and 0x92, a range of 11 code points. However, if the range is 1364specified numerically, for example, [\x88-\x92] or [h-\x92], all code points 1365are included. 1366</P> 1367<P> 1368If a range that includes letters is used when caseless matching is set, it 1369matches the letters in either case. For example, [W-c] is equivalent to 1370[][\\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if character 1371tables for a French locale are in use, [\xc8-\xcb] matches accented E 1372characters in both cases. 1373</P> 1374<P> 1375The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, 1376\V, \w, and \W may appear in a character class, and add the characters that 1377they match to the class. For example, [\dABCDEF] matches any hexadecimal 1378digit. In UTF modes, the PCRE2_UCP option affects the meanings of \d, \s, \w 1379and their upper case partners, just as it does when they appear outside a 1380character class, as described in the section entitled 1381<a href="#genericchartypes">"Generic character types"</a> 1382above. The escape sequence \b has a different meaning inside a character 1383class; it matches the backspace character. The sequences \B, \N, \R, and \X 1384are not special inside a character class. Like any other unrecognized escape 1385sequences, they cause an error. 1386</P> 1387<P> 1388A circumflex can conveniently be used with the upper case character types to 1389specify a more restricted set of characters than the matching lower case type. 1390For example, the class [^\W_] matches any letter or digit, but not underscore, 1391whereas [\w] includes underscore. A positive character class should be read as 1392"something OR something OR ..." and a negative class as "NOT something AND NOT 1393something AND NOT ...". 1394</P> 1395<P> 1396The only metacharacters that are recognized in character classes are backslash, 1397hyphen (only where it can be interpreted as specifying a range), circumflex 1398(only at the start), opening square bracket (only when it can be interpreted as 1399introducing a POSIX class name, or for a special compatibility feature - see 1400the next two sections), and the terminating closing square bracket. However, 1401escaping other non-alphanumeric characters does no harm. 1402</P> 1403<br><a name="SEC10" href="#TOC1">POSIX CHARACTER CLASSES</a><br> 1404<P> 1405Perl supports the POSIX notation for character classes. This uses names 1406enclosed by [: and :] within the enclosing square brackets. PCRE2 also supports 1407this notation. For example, 1408<pre> 1409 [01[:alpha:]%] 1410</pre> 1411matches "0", "1", any alphabetic character, or "%". The supported class names 1412are: 1413<pre> 1414 alnum letters and digits 1415 alpha letters 1416 ascii character codes 0 - 127 1417 blank space or tab only 1418 cntrl control characters 1419 digit decimal digits (same as \d) 1420 graph printing characters, excluding space 1421 lower lower case letters 1422 print printing characters, including space 1423 punct printing characters, excluding letters and digits and space 1424 space white space (the same as \s from PCRE2 8.34) 1425 upper upper case letters 1426 word "word" characters (same as \w) 1427 xdigit hexadecimal digits 1428</pre> 1429The default "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), 1430and space (32). If locale-specific matching is taking place, the list of space 1431characters may be different; there may be fewer or more of them. "Space" and 1432\s match the same set of characters. 1433</P> 1434<P> 1435The name "word" is a Perl extension, and "blank" is a GNU extension from Perl 14365.8. Another Perl extension is negation, which is indicated by a ^ character 1437after the colon. For example, 1438<pre> 1439 [12[:^digit:]] 1440</pre> 1441matches "1", "2", or any non-digit. PCRE2 (and Perl) also recognize the POSIX 1442syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not 1443supported, and an error is given if they are encountered. 1444</P> 1445<P> 1446By default, characters with values greater than 127 do not match any of the 1447POSIX character classes, although this may be different for characters in the 1448range 128-255 when locale-specific matching is happening. However, if the 1449PCRE2_UCP option is passed to <b>pcre2_compile()</b>, some of the classes are 1450changed so that Unicode character properties are used. This is achieved by 1451replacing certain POSIX classes with other sequences, as follows: 1452<pre> 1453 [:alnum:] becomes \p{Xan} 1454 [:alpha:] becomes \p{L} 1455 [:blank:] becomes \h 1456 [:cntrl:] becomes \p{Cc} 1457 [:digit:] becomes \p{Nd} 1458 [:lower:] becomes \p{Ll} 1459 [:space:] becomes \p{Xps} 1460 [:upper:] becomes \p{Lu} 1461 [:word:] becomes \p{Xwd} 1462</pre> 1463Negated versions, such as [:^alpha:] use \P instead of \p. Three other POSIX 1464classes are handled specially in UCP mode: 1465</P> 1466<P> 1467[:graph:] 1468This matches characters that have glyphs that mark the page when printed. In 1469Unicode property terms, it matches all characters with the L, M, N, P, S, or Cf 1470properties, except for: 1471<pre> 1472 U+061C Arabic Letter Mark 1473 U+180E Mongolian Vowel Separator 1474 U+2066 - U+2069 Various "isolate"s 1475 1476</PRE> 1477</P> 1478<P> 1479[:print:] 1480This matches the same characters as [:graph:] plus space characters that are 1481not controls, that is, characters with the Zs property. 1482</P> 1483<P> 1484[:punct:] 1485This matches all characters that have the Unicode P (punctuation) property, 1486plus those characters with code points less than 256 that have the S (Symbol) 1487property. 1488</P> 1489<P> 1490The other POSIX classes are unchanged, and match only characters with code 1491points less than 256. 1492</P> 1493<br><a name="SEC11" href="#TOC1">COMPATIBILITY FEATURE FOR WORD BOUNDARIES</a><br> 1494<P> 1495In the POSIX.2 compliant library that was included in 4.4BSD Unix, the ugly 1496syntax [[:<:]] and [[:>:]] is used for matching "start of word" and "end of 1497word". PCRE2 treats these items as follows: 1498<pre> 1499 [[:<:]] is converted to \b(?=\w) 1500 [[:>:]] is converted to \b(?<=\w) 1501</pre> 1502Only these exact character sequences are recognized. A sequence such as 1503[a[:<:]b] provokes error for an unrecognized POSIX class name. This support is 1504not compatible with Perl. It is provided to help migrations from other 1505environments, and is best not used in any new patterns. Note that \b matches 1506at the start and the end of a word (see 1507<a href="#smallassertions">"Simple assertions"</a> 1508above), and in a Perl-style pattern the preceding or following character 1509normally shows which is wanted, without the need for the assertions that are 1510used above in order to give exactly the POSIX behaviour. 1511</P> 1512<br><a name="SEC12" href="#TOC1">VERTICAL BAR</a><br> 1513<P> 1514Vertical bar characters are used to separate alternative patterns. For example, 1515the pattern 1516<pre> 1517 gilbert|sullivan 1518</pre> 1519matches either "gilbert" or "sullivan". Any number of alternatives may appear, 1520and an empty alternative is permitted (matching the empty string). The matching 1521process tries each alternative in turn, from left to right, and the first one 1522that succeeds is used. If the alternatives are within a subpattern 1523<a href="#subpattern">(defined below),</a> 1524"succeeds" means matching the rest of the main pattern as well as the 1525alternative in the subpattern. 1526</P> 1527<br><a name="SEC13" href="#TOC1">INTERNAL OPTION SETTING</a><br> 1528<P> 1529The settings of the PCRE2_CASELESS, PCRE2_MULTILINE, PCRE2_DOTALL, and 1530PCRE2_EXTENDED options (which are Perl-compatible) can be changed from within 1531the pattern by a sequence of Perl option letters enclosed between "(?" and ")". 1532The option letters are 1533<pre> 1534 i for PCRE2_CASELESS 1535 m for PCRE2_MULTILINE 1536 s for PCRE2_DOTALL 1537 x for PCRE2_EXTENDED 1538</pre> 1539For example, (?im) sets caseless, multiline matching. It is also possible to 1540unset these options by preceding the letter with a hyphen, and a combined 1541setting and unsetting such as (?im-sx), which sets PCRE2_CASELESS and 1542PCRE2_MULTILINE while unsetting PCRE2_DOTALL and PCRE2_EXTENDED, is also 1543permitted. If a letter appears both before and after the hyphen, the option is 1544unset. An empty options setting "(?)" is allowed. Needless to say, it has no 1545effect. 1546</P> 1547<P> 1548The PCRE2-specific options PCRE2_DUPNAMES and PCRE2_UNGREEDY can be changed in 1549the same way as the Perl-compatible options by using the characters J and U 1550respectively. 1551</P> 1552<P> 1553When one of these option changes occurs at top level (that is, not inside 1554subpattern parentheses), the change applies to the remainder of the pattern 1555that follows. If the change is placed right at the start of a pattern, PCRE2 1556extracts it into the global options (and it will therefore show up in data 1557extracted by the <b>pcre2_pattern_info()</b> function). 1558</P> 1559<P> 1560An option change within a subpattern (see below for a description of 1561subpatterns) affects only that part of the subpattern that follows it, so 1562<pre> 1563 (a(?i)b)c 1564</pre> 1565matches abc and aBc and no other strings (assuming PCRE2_CASELESS is not used). 1566By this means, options can be made to have different settings in different 1567parts of the pattern. Any changes made in one alternative do carry on 1568into subsequent branches within the same subpattern. For example, 1569<pre> 1570 (a(?i)b|c) 1571</pre> 1572matches "ab", "aB", "c", and "C", even though when matching "C" the first 1573branch is abandoned before the option setting. This is because the effects of 1574option settings happen at compile time. There would be some very weird 1575behaviour otherwise. 1576</P> 1577<P> 1578As a convenient shorthand, if any option settings are required at the start of 1579a non-capturing subpattern (see the next section), the option letters may 1580appear between the "?" and the ":". Thus the two patterns 1581<pre> 1582 (?i:saturday|sunday) 1583 (?:(?i)saturday|sunday) 1584</pre> 1585match exactly the same set of strings. 1586</P> 1587<P> 1588<b>Note:</b> There are other PCRE2-specific options that can be set by the 1589application when the compiling function is called. The pattern can contain 1590special leading sequences such as (*CRLF) to override what the application has 1591set or what has been defaulted. Details are given in the section entitled 1592<a href="#newlineseq">"Newline sequences"</a> 1593above. There are also the (*UTF) and (*UCP) leading sequences that can be used 1594to set UTF and Unicode property modes; they are equivalent to setting the 1595PCRE2_UTF and PCRE2_UCP options, respectively. However, the application can set 1596the PCRE2_NEVER_UTF and PCRE2_NEVER_UCP options, which lock out the use of the 1597(*UTF) and (*UCP) sequences. 1598<a name="subpattern"></a></P> 1599<br><a name="SEC14" href="#TOC1">SUBPATTERNS</a><br> 1600<P> 1601Subpatterns are delimited by parentheses (round brackets), which can be nested. 1602Turning part of a pattern into a subpattern does two things: 1603<br> 1604<br> 16051. It localizes a set of alternatives. For example, the pattern 1606<pre> 1607 cat(aract|erpillar|) 1608</pre> 1609matches "cataract", "caterpillar", or "cat". Without the parentheses, it would 1610match "cataract", "erpillar" or an empty string. 1611<br> 1612<br> 16132. It sets up the subpattern as a capturing subpattern. This means that, when 1614the whole pattern matches, the portion of the subject string that matched the 1615subpattern is passed back to the caller, separately from the portion that 1616matched the whole pattern. (This applies only to the traditional matching 1617function; the DFA matching function does not support capturing.) 1618</P> 1619<P> 1620Opening parentheses are counted from left to right (starting from 1) to obtain 1621numbers for the capturing subpatterns. For example, if the string "the red 1622king" is matched against the pattern 1623<pre> 1624 the ((red|white) (king|queen)) 1625</pre> 1626the captured substrings are "red king", "red", and "king", and are numbered 1, 16272, and 3, respectively. 1628</P> 1629<P> 1630The fact that plain parentheses fulfil two functions is not always helpful. 1631There are often times when a grouping subpattern is required without a 1632capturing requirement. If an opening parenthesis is followed by a question mark 1633and a colon, the subpattern does not do any capturing, and is not counted when 1634computing the number of any subsequent capturing subpatterns. For example, if 1635the string "the white queen" is matched against the pattern 1636<pre> 1637 the ((?:red|white) (king|queen)) 1638</pre> 1639the captured substrings are "white queen" and "queen", and are numbered 1 and 16402. The maximum number of capturing subpatterns is 65535. 1641</P> 1642<P> 1643As a convenient shorthand, if any option settings are required at the start of 1644a non-capturing subpattern, the option letters may appear between the "?" and 1645the ":". Thus the two patterns 1646<pre> 1647 (?i:saturday|sunday) 1648 (?:(?i)saturday|sunday) 1649</pre> 1650match exactly the same set of strings. Because alternative branches are tried 1651from left to right, and options are not reset until the end of the subpattern 1652is reached, an option setting in one branch does affect subsequent branches, so 1653the above patterns match "SUNDAY" as well as "Saturday". 1654<a name="dupsubpatternnumber"></a></P> 1655<br><a name="SEC15" href="#TOC1">DUPLICATE SUBPATTERN NUMBERS</a><br> 1656<P> 1657Perl 5.10 introduced a feature whereby each alternative in a subpattern uses 1658the same numbers for its capturing parentheses. Such a subpattern starts with 1659(?| and is itself a non-capturing subpattern. For example, consider this 1660pattern: 1661<pre> 1662 (?|(Sat)ur|(Sun))day 1663</pre> 1664Because the two alternatives are inside a (?| group, both sets of capturing 1665parentheses are numbered one. Thus, when the pattern matches, you can look 1666at captured substring number one, whichever alternative matched. This construct 1667is useful when you want to capture part, but not all, of one of a number of 1668alternatives. Inside a (?| group, parentheses are numbered as usual, but the 1669number is reset at the start of each branch. The numbers of any capturing 1670parentheses that follow the subpattern start after the highest number used in 1671any branch. The following example is taken from the Perl documentation. The 1672numbers underneath show in which buffer the captured content will be stored. 1673<pre> 1674 # before ---------------branch-reset----------- after 1675 / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x 1676 # 1 2 2 3 2 3 4 1677</pre> 1678A back reference to a numbered subpattern uses the most recent value that is 1679set for that number by any subpattern. The following pattern matches "abcabc" 1680or "defdef": 1681<pre> 1682 /(?|(abc)|(def))\1/ 1683</pre> 1684In contrast, a subroutine call to a numbered subpattern always refers to the 1685first one in the pattern with the given number. The following pattern matches 1686"abcabc" or "defabc": 1687<pre> 1688 /(?|(abc)|(def))(?1)/ 1689</pre> 1690A relative reference such as (?-1) is no different: it is just a convenient way 1691of computing an absolute group number. 1692</P> 1693<P> 1694If a 1695<a href="#conditions">condition test</a> 1696for a subpattern's having matched refers to a non-unique number, the test is 1697true if any of the subpatterns of that number have matched. 1698</P> 1699<P> 1700An alternative approach to using this "branch reset" feature is to use 1701duplicate named subpatterns, as described in the next section. 1702</P> 1703<br><a name="SEC16" href="#TOC1">NAMED SUBPATTERNS</a><br> 1704<P> 1705Identifying capturing parentheses by number is simple, but it can be very hard 1706to keep track of the numbers in complicated regular expressions. Furthermore, 1707if an expression is modified, the numbers may change. To help with this 1708difficulty, PCRE2 supports the naming of subpatterns. This feature was not 1709added to Perl until release 5.10. Python had the feature earlier, and PCRE1 1710introduced it at release 4.0, using the Python syntax. PCRE2 supports both the 1711Perl and the Python syntax. Perl allows identically numbered subpatterns to 1712have different names, but PCRE2 does not. 1713</P> 1714<P> 1715In PCRE2, a subpattern can be named in one of three ways: (?<name>...) or 1716(?'name'...) as in Perl, or (?P<name>...) as in Python. References to capturing 1717parentheses from other parts of the pattern, such as 1718<a href="#backreferences">back references,</a> 1719<a href="#recursion">recursion,</a> 1720and 1721<a href="#conditions">conditions,</a> 1722can be made by name as well as by number. 1723</P> 1724<P> 1725Names consist of up to 32 alphanumeric characters and underscores, but must 1726start with a non-digit. Named capturing parentheses are still allocated numbers 1727as well as names, exactly as if the names were not present. The PCRE2 API 1728provides function calls for extracting the name-to-number translation table 1729from a compiled pattern. There are also convenience functions for extracting a 1730captured substring by name. 1731</P> 1732<P> 1733By default, a name must be unique within a pattern, but it is possible to relax 1734this constraint by setting the PCRE2_DUPNAMES option at compile time. 1735(Duplicate names are also always permitted for subpatterns with the same 1736number, set up as described in the previous section.) Duplicate names can be 1737useful for patterns where only one instance of the named parentheses can match. 1738Suppose you want to match the name of a weekday, either as a 3-letter 1739abbreviation or as the full name, and in both cases you want to extract the 1740abbreviation. This pattern (ignoring the line breaks) does the job: 1741<pre> 1742 (?<DN>Mon|Fri|Sun)(?:day)?| 1743 (?<DN>Tue)(?:sday)?| 1744 (?<DN>Wed)(?:nesday)?| 1745 (?<DN>Thu)(?:rsday)?| 1746 (?<DN>Sat)(?:urday)? 1747</pre> 1748There are five capturing substrings, but only one is ever set after a match. 1749(An alternative way of solving this problem is to use a "branch reset" 1750subpattern, as described in the previous section.) 1751</P> 1752<P> 1753The convenience functions for extracting the data by name returns the substring 1754for the first (and in this example, the only) subpattern of that name that 1755matched. This saves searching to find which numbered subpattern it was. 1756</P> 1757<P> 1758If you make a back reference to a non-unique named subpattern from elsewhere in 1759the pattern, the subpatterns to which the name refers are checked in the order 1760in which they appear in the overall pattern. The first one that is set is used 1761for the reference. For example, this pattern matches both "foofoo" and 1762"barbar" but not "foobar" or "barfoo": 1763<pre> 1764 (?:(?<n>foo)|(?<n>bar))\k<n> 1765 1766</PRE> 1767</P> 1768<P> 1769If you make a subroutine call to a non-unique named subpattern, the one that 1770corresponds to the first occurrence of the name is used. In the absence of 1771duplicate numbers (see the previous section) this is the one with the lowest 1772number. 1773</P> 1774<P> 1775If you use a named reference in a condition 1776test (see the 1777<a href="#conditions">section about conditions</a> 1778below), either to check whether a subpattern has matched, or to check for 1779recursion, all subpatterns with the same name are tested. If the condition is 1780true for any one of them, the overall condition is true. This is the same 1781behaviour as testing by number. For further details of the interfaces for 1782handling named subpatterns, see the 1783<a href="pcre2api.html"><b>pcre2api</b></a> 1784documentation. 1785</P> 1786<P> 1787<b>Warning:</b> You cannot use different names to distinguish between two 1788subpatterns with the same number because PCRE2 uses only the numbers when 1789matching. For this reason, an error is given at compile time if different names 1790are given to subpatterns with the same number. However, you can always give the 1791same name to subpatterns with the same number, even when PCRE2_DUPNAMES is not 1792set. 1793</P> 1794<br><a name="SEC17" href="#TOC1">REPETITION</a><br> 1795<P> 1796Repetition is specified by quantifiers, which can follow any of the following 1797items: 1798<pre> 1799 a literal data character 1800 the dot metacharacter 1801 the \C escape sequence 1802 the \X escape sequence 1803 the \R escape sequence 1804 an escape such as \d or \pL that matches a single character 1805 a character class 1806 a back reference 1807 a parenthesized subpattern (including most assertions) 1808 a subroutine call to a subpattern (recursive or otherwise) 1809</pre> 1810The general repetition quantifier specifies a minimum and maximum number of 1811permitted matches, by giving the two numbers in curly brackets (braces), 1812separated by a comma. The numbers must be less than 65536, and the first must 1813be less than or equal to the second. For example: 1814<pre> 1815 z{2,4} 1816</pre> 1817matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special 1818character. If the second number is omitted, but the comma is present, there is 1819no upper limit; if the second number and the comma are both omitted, the 1820quantifier specifies an exact number of required matches. Thus 1821<pre> 1822 [aeiou]{3,} 1823</pre> 1824matches at least 3 successive vowels, but may match many more, whereas 1825<pre> 1826 \d{8} 1827</pre> 1828matches exactly 8 digits. An opening curly bracket that appears in a position 1829where a quantifier is not allowed, or one that does not match the syntax of a 1830quantifier, is taken as a literal character. For example, {,6} is not a 1831quantifier, but a literal string of four characters. 1832</P> 1833<P> 1834In UTF modes, quantifiers apply to characters rather than to individual code 1835units. Thus, for example, \x{100}{2} matches two characters, each of 1836which is represented by a two-byte sequence in a UTF-8 string. Similarly, 1837\X{3} matches three Unicode extended grapheme clusters, each of which may be 1838several code units long (and they may be of different lengths). 1839</P> 1840<P> 1841The quantifier {0} is permitted, causing the expression to behave as if the 1842previous item and the quantifier were not present. This may be useful for 1843subpatterns that are referenced as 1844<a href="#subpatternsassubroutines">subroutines</a> 1845from elsewhere in the pattern (but see also the section entitled 1846<a href="#subdefine">"Defining subpatterns for use by reference only"</a> 1847below). Items other than subpatterns that have a {0} quantifier are omitted 1848from the compiled pattern. 1849</P> 1850<P> 1851For convenience, the three most common quantifiers have single-character 1852abbreviations: 1853<pre> 1854 * is equivalent to {0,} 1855 + is equivalent to {1,} 1856 ? is equivalent to {0,1} 1857</pre> 1858It is possible to construct infinite loops by following a subpattern that can 1859match no characters with a quantifier that has no upper limit, for example: 1860<pre> 1861 (a?)* 1862</pre> 1863Earlier versions of Perl and PCRE1 used to give an error at compile time for 1864such patterns. However, because there are cases where this can be useful, such 1865patterns are now accepted, but if any repetition of the subpattern does in fact 1866match no characters, the loop is forcibly broken. 1867</P> 1868<P> 1869By default, the quantifiers are "greedy", that is, they match as much as 1870possible (up to the maximum number of permitted times), without causing the 1871rest of the pattern to fail. The classic example of where this gives problems 1872is in trying to match comments in C programs. These appear between /* and */ 1873and within the comment, individual * and / characters may appear. An attempt to 1874match C comments by applying the pattern 1875<pre> 1876 /\*.*\*/ 1877</pre> 1878to the string 1879<pre> 1880 /* first comment */ not comment /* second comment */ 1881</pre> 1882fails, because it matches the entire string owing to the greediness of the .* 1883item. 1884</P> 1885<P> 1886If a quantifier is followed by a question mark, it ceases to be greedy, and 1887instead matches the minimum number of times possible, so the pattern 1888<pre> 1889 /\*.*?\*/ 1890</pre> 1891does the right thing with the C comments. The meaning of the various 1892quantifiers is not otherwise changed, just the preferred number of matches. 1893Do not confuse this use of question mark with its use as a quantifier in its 1894own right. Because it has two uses, it can sometimes appear doubled, as in 1895<pre> 1896 \d??\d 1897</pre> 1898which matches one digit by preference, but can match two if that is the only 1899way the rest of the pattern matches. 1900</P> 1901<P> 1902If the PCRE2_UNGREEDY option is set (an option that is not available in Perl), 1903the quantifiers are not greedy by default, but individual ones can be made 1904greedy by following them with a question mark. In other words, it inverts the 1905default behaviour. 1906</P> 1907<P> 1908When a parenthesized subpattern is quantified with a minimum repeat count that 1909is greater than 1 or with a limited maximum, more memory is required for the 1910compiled pattern, in proportion to the size of the minimum or maximum. 1911</P> 1912<P> 1913If a pattern starts with .* or .{0,} and the PCRE2_DOTALL option (equivalent 1914to Perl's /s) is set, thus allowing the dot to match newlines, the pattern is 1915implicitly anchored, because whatever follows will be tried against every 1916character position in the subject string, so there is no point in retrying the 1917overall match at any position after the first. PCRE2 normally treats such a 1918pattern as though it were preceded by \A. 1919</P> 1920<P> 1921In cases where it is known that the subject string contains no newlines, it is 1922worth setting PCRE2_DOTALL in order to obtain this optimization, or 1923alternatively, using ^ to indicate anchoring explicitly. 1924</P> 1925<P> 1926However, there are some cases where the optimization cannot be used. When .* 1927is inside capturing parentheses that are the subject of a back reference 1928elsewhere in the pattern, a match at the start may fail where a later one 1929succeeds. Consider, for example: 1930<pre> 1931 (.*)abc\1 1932</pre> 1933If the subject is "xyz123abc123" the match point is the fourth character. For 1934this reason, such a pattern is not implicitly anchored. 1935</P> 1936<P> 1937Another case where implicit anchoring is not applied is when the leading .* is 1938inside an atomic group. Once again, a match at the start may fail where a later 1939one succeeds. Consider this pattern: 1940<pre> 1941 (?>.*?a)b 1942</pre> 1943It matches "ab" in the subject "aab". The use of the backtracking control verbs 1944(*PRUNE) and (*SKIP) also disable this optimization, and there is an option, 1945PCRE2_NO_DOTSTAR_ANCHOR, to do so explicitly. 1946</P> 1947<P> 1948When a capturing subpattern is repeated, the value captured is the substring 1949that matched the final iteration. For example, after 1950<pre> 1951 (tweedle[dume]{3}\s*)+ 1952</pre> 1953has matched "tweedledum tweedledee" the value of the captured substring is 1954"tweedledee". However, if there are nested capturing subpatterns, the 1955corresponding captured values may have been set in previous iterations. For 1956example, after 1957<pre> 1958 (a|(b))+ 1959</pre> 1960matches "aba" the value of the second captured substring is "b". 1961<a name="atomicgroup"></a></P> 1962<br><a name="SEC18" href="#TOC1">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a><br> 1963<P> 1964With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") 1965repetition, failure of what follows normally causes the repeated item to be 1966re-evaluated to see if a different number of repeats allows the rest of the 1967pattern to match. Sometimes it is useful to prevent this, either to change the 1968nature of the match, or to cause it fail earlier than it otherwise might, when 1969the author of the pattern knows there is no point in carrying on. 1970</P> 1971<P> 1972Consider, for example, the pattern \d+foo when applied to the subject line 1973<pre> 1974 123456bar 1975</pre> 1976After matching all 6 digits and then failing to match "foo", the normal 1977action of the matcher is to try again with only 5 digits matching the \d+ 1978item, and then with 4, and so on, before ultimately failing. "Atomic grouping" 1979(a term taken from Jeffrey Friedl's book) provides the means for specifying 1980that once a subpattern has matched, it is not to be re-evaluated in this way. 1981</P> 1982<P> 1983If we use atomic grouping for the previous example, the matcher gives up 1984immediately on failing to match "foo" the first time. The notation is a kind of 1985special parenthesis, starting with (?> as in this example: 1986<pre> 1987 (?>\d+)foo 1988</pre> 1989This kind of parenthesis "locks up" the part of the pattern it contains once 1990it has matched, and a failure further into the pattern is prevented from 1991backtracking into it. Backtracking past it to previous items, however, works as 1992normal. 1993</P> 1994<P> 1995An alternative description is that a subpattern of this type matches exactly 1996the string of characters that an identical standalone pattern would match, if 1997anchored at the current point in the subject string. 1998</P> 1999<P> 2000Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as 2001the above example can be thought of as a maximizing repeat that must swallow 2002everything it can. So, while both \d+ and \d+? are prepared to adjust the 2003number of digits they match in order to make the rest of the pattern match, 2004(?>\d+) can only match an entire sequence of digits. 2005</P> 2006<P> 2007Atomic groups in general can of course contain arbitrarily complicated 2008subpatterns, and can be nested. However, when the subpattern for an atomic 2009group is just a single repeated item, as in the example above, a simpler 2010notation, called a "possessive quantifier" can be used. This consists of an 2011additional + character following a quantifier. Using this notation, the 2012previous example can be rewritten as 2013<pre> 2014 \d++foo 2015</pre> 2016Note that a possessive quantifier can be used with an entire group, for 2017example: 2018<pre> 2019 (abc|xyz){2,3}+ 2020</pre> 2021Possessive quantifiers are always greedy; the setting of the PCRE2_UNGREEDY 2022option is ignored. They are a convenient notation for the simpler forms of 2023atomic group. However, there is no difference in the meaning of a possessive 2024quantifier and the equivalent atomic group, though there may be a performance 2025difference; possessive quantifiers should be slightly faster. 2026</P> 2027<P> 2028The possessive quantifier syntax is an extension to the Perl 5.8 syntax. 2029Jeffrey Friedl originated the idea (and the name) in the first edition of his 2030book. Mike McCloskey liked it, so implemented it when he built Sun's Java 2031package, and PCRE1 copied it from there. It ultimately found its way into Perl 2032at release 5.10. 2033</P> 2034<P> 2035PCRE2 has an optimization that automatically "possessifies" certain simple 2036pattern constructs. For example, the sequence A+B is treated as A++B because 2037there is no point in backtracking into a sequence of A's when B must follow. 2038This feature can be disabled by the PCRE2_NO_AUTOPOSSESS option, or starting 2039the pattern with (*NO_AUTO_POSSESS). 2040</P> 2041<P> 2042When a pattern contains an unlimited repeat inside a subpattern that can itself 2043be repeated an unlimited number of times, the use of an atomic group is the 2044only way to avoid some failing matches taking a very long time indeed. The 2045pattern 2046<pre> 2047 (\D+|<\d+>)*[!?] 2048</pre> 2049matches an unlimited number of substrings that either consist of non-digits, or 2050digits enclosed in <>, followed by either ! or ?. When it matches, it runs 2051quickly. However, if it is applied to 2052<pre> 2053 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 2054</pre> 2055it takes a long time before reporting failure. This is because the string can 2056be divided between the internal \D+ repeat and the external * repeat in a 2057large number of ways, and all have to be tried. (The example uses [!?] rather 2058than a single character at the end, because both PCRE2 and Perl have an 2059optimization that allows for fast failure when a single character is used. They 2060remember the last single character that is required for a match, and fail early 2061if it is not present in the string.) If the pattern is changed so that it uses 2062an atomic group, like this: 2063<pre> 2064 ((?>\D+)|<\d+>)*[!?] 2065</pre> 2066sequences of non-digits cannot be broken, and failure happens quickly. 2067<a name="backreferences"></a></P> 2068<br><a name="SEC19" href="#TOC1">BACK REFERENCES</a><br> 2069<P> 2070Outside a character class, a backslash followed by a digit greater than 0 (and 2071possibly further digits) is a back reference to a capturing subpattern earlier 2072(that is, to its left) in the pattern, provided there have been that many 2073previous capturing left parentheses. 2074</P> 2075<P> 2076However, if the decimal number following the backslash is less than 8, it is 2077always taken as a back reference, and causes an error only if there are not 2078that many capturing left parentheses in the entire pattern. In other words, the 2079parentheses that are referenced need not be to the left of the reference for 2080numbers less than 8. A "forward back reference" of this type can make sense 2081when a repetition is involved and the subpattern to the right has participated 2082in an earlier iteration. 2083</P> 2084<P> 2085It is not possible to have a numerical "forward back reference" to a subpattern 2086whose number is 8 or more using this syntax because a sequence such as \50 is 2087interpreted as a character defined in octal. See the subsection entitled 2088"Non-printing characters" 2089<a href="#digitsafterbackslash">above</a> 2090for further details of the handling of digits following a backslash. There is 2091no such problem when named parentheses are used. A back reference to any 2092subpattern is possible using named parentheses (see below). 2093</P> 2094<P> 2095Another way of avoiding the ambiguity inherent in the use of digits following a 2096backslash is to use the \g escape sequence. This escape must be followed by an 2097unsigned number or a negative number, optionally enclosed in braces. These 2098examples are all identical: 2099<pre> 2100 (ring), \1 2101 (ring), \g1 2102 (ring), \g{1} 2103</pre> 2104An unsigned number specifies an absolute reference without the ambiguity that 2105is present in the older syntax. It is also useful when literal digits follow 2106the reference. A negative number is a relative reference. Consider this 2107example: 2108<pre> 2109 (abc(def)ghi)\g{-1} 2110</pre> 2111The sequence \g{-1} is a reference to the most recently started capturing 2112subpattern before \g, that is, is it equivalent to \2 in this example. 2113Similarly, \g{-2} would be equivalent to \1. The use of relative references 2114can be helpful in long patterns, and also in patterns that are created by 2115joining together fragments that contain references within themselves. 2116</P> 2117<P> 2118A back reference matches whatever actually matched the capturing subpattern in 2119the current subject string, rather than anything matching the subpattern 2120itself (see 2121<a href="#subpatternsassubroutines">"Subpatterns as subroutines"</a> 2122below for a way of doing that). So the pattern 2123<pre> 2124 (sens|respons)e and \1ibility 2125</pre> 2126matches "sense and sensibility" and "response and responsibility", but not 2127"sense and responsibility". If caseful matching is in force at the time of the 2128back reference, the case of letters is relevant. For example, 2129<pre> 2130 ((?i)rah)\s+\1 2131</pre> 2132matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original 2133capturing subpattern is matched caselessly. 2134</P> 2135<P> 2136There are several different ways of writing back references to named 2137subpatterns. The .NET syntax \k{name} and the Perl syntax \k<name> or 2138\k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified 2139back reference syntax, in which \g can be used for both numeric and named 2140references, is also supported. We could rewrite the above example in any of 2141the following ways: 2142<pre> 2143 (?<p1>(?i)rah)\s+\k<p1> 2144 (?'p1'(?i)rah)\s+\k{p1} 2145 (?P<p1>(?i)rah)\s+(?P=p1) 2146 (?<p1>(?i)rah)\s+\g{p1} 2147</pre> 2148A subpattern that is referenced by name may appear in the pattern before or 2149after the reference. 2150</P> 2151<P> 2152There may be more than one back reference to the same subpattern. If a 2153subpattern has not actually been used in a particular match, any back 2154references to it always fail by default. For example, the pattern 2155<pre> 2156 (a|(bc))\2 2157</pre> 2158always fails if it starts to match "a" rather than "bc". However, if the 2159PCRE2_MATCH_UNSET_BACKREF option is set at compile time, a back reference to an 2160unset value matches an empty string. 2161</P> 2162<P> 2163Because there may be many capturing parentheses in a pattern, all digits 2164following a backslash are taken as part of a potential back reference number. 2165If the pattern continues with a digit character, some delimiter must be used to 2166terminate the back reference. If the PCRE2_EXTENDED option is set, this can be 2167white space. Otherwise, the \g{ syntax or an empty comment (see 2168<a href="#comments">"Comments"</a> 2169below) can be used. 2170</P> 2171<br><b> 2172Recursive back references 2173</b><br> 2174<P> 2175A back reference that occurs inside the parentheses to which it refers fails 2176when the subpattern is first used, so, for example, (a\1) never matches. 2177However, such references can be useful inside repeated subpatterns. For 2178example, the pattern 2179<pre> 2180 (a|b\1)+ 2181</pre> 2182matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of 2183the subpattern, the back reference matches the character string corresponding 2184to the previous iteration. In order for this to work, the pattern must be such 2185that the first iteration does not need to match the back reference. This can be 2186done using alternation, as in the example above, or by a quantifier with a 2187minimum of zero. 2188</P> 2189<P> 2190Back references of this type cause the group that they reference to be treated 2191as an 2192<a href="#atomicgroup">atomic group.</a> 2193Once the whole group has been matched, a subsequent matching failure cannot 2194cause backtracking into the middle of the group. 2195<a name="bigassertions"></a></P> 2196<br><a name="SEC20" href="#TOC1">ASSERTIONS</a><br> 2197<P> 2198An assertion is a test on the characters following or preceding the current 2199matching point that does not consume any characters. The simple assertions 2200coded as \b, \B, \A, \G, \Z, \z, ^ and $ are described 2201<a href="#smallassertions">above.</a> 2202</P> 2203<P> 2204More complicated assertions are coded as subpatterns. There are two kinds: 2205those that look ahead of the current position in the subject string, and those 2206that look behind it. An assertion subpattern is matched in the normal way, 2207except that it does not cause the current matching position to be changed. 2208</P> 2209<P> 2210Assertion subpatterns are not capturing subpatterns. If such an assertion 2211contains capturing subpatterns within it, these are counted for the purposes of 2212numbering the capturing subpatterns in the whole pattern. However, substring 2213capturing is carried out only for positive assertions. (Perl sometimes, but not 2214always, does do capturing in negative assertions.) 2215</P> 2216<P> 2217For compatibility with Perl, most assertion subpatterns may be repeated; though 2218it makes no sense to assert the same thing several times, the side effect of 2219capturing parentheses may occasionally be useful. However, an assertion that 2220forms the condition for a conditional subpattern may not be quantified. In 2221practice, for other assertions, there only three cases: 2222<br> 2223<br> 2224(1) If the quantifier is {0}, the assertion is never obeyed during matching. 2225However, it may contain internal capturing parenthesized groups that are called 2226from elsewhere via the 2227<a href="#subpatternsassubroutines">subroutine mechanism.</a> 2228<br> 2229<br> 2230(2) If quantifier is {0,n} where n is greater than zero, it is treated as if it 2231were {0,1}. At run time, the rest of the pattern match is tried with and 2232without the assertion, the order depending on the greediness of the quantifier. 2233<br> 2234<br> 2235(3) If the minimum repetition is greater than zero, the quantifier is ignored. 2236The assertion is obeyed just once when encountered during matching. 2237</P> 2238<br><b> 2239Lookahead assertions 2240</b><br> 2241<P> 2242Lookahead assertions start with (?= for positive assertions and (?! for 2243negative assertions. For example, 2244<pre> 2245 \w+(?=;) 2246</pre> 2247matches a word followed by a semicolon, but does not include the semicolon in 2248the match, and 2249<pre> 2250 foo(?!bar) 2251</pre> 2252matches any occurrence of "foo" that is not followed by "bar". Note that the 2253apparently similar pattern 2254<pre> 2255 (?!foo)bar 2256</pre> 2257does not find an occurrence of "bar" that is preceded by something other than 2258"foo"; it finds any occurrence of "bar" whatsoever, because the assertion 2259(?!foo) is always true when the next three characters are "bar". A 2260lookbehind assertion is needed to achieve the other effect. 2261</P> 2262<P> 2263If you want to force a matching failure at some point in a pattern, the most 2264convenient way to do it is with (?!) because an empty string always matches, so 2265an assertion that requires there not to be an empty string must always fail. 2266The backtracking control verb (*FAIL) or (*F) is a synonym for (?!). 2267<a name="lookbehind"></a></P> 2268<br><b> 2269Lookbehind assertions 2270</b><br> 2271<P> 2272Lookbehind assertions start with (?<= for positive assertions and (?<! for 2273negative assertions. For example, 2274<pre> 2275 (?<!foo)bar 2276</pre> 2277does find an occurrence of "bar" that is not preceded by "foo". The contents of 2278a lookbehind assertion are restricted such that all the strings it matches must 2279have a fixed length. However, if there are several top-level alternatives, they 2280do not all have to have the same fixed length. Thus 2281<pre> 2282 (?<=bullock|donkey) 2283</pre> 2284is permitted, but 2285<pre> 2286 (?<!dogs?|cats?) 2287</pre> 2288causes an error at compile time. Branches that match different length strings 2289are permitted only at the top level of a lookbehind assertion. This is an 2290extension compared with Perl, which requires all branches to match the same 2291length of string. An assertion such as 2292<pre> 2293 (?<=ab(c|de)) 2294</pre> 2295is not permitted, because its single top-level branch can match two different 2296lengths, but it is acceptable to PCRE2 if rewritten to use two top-level 2297branches: 2298<pre> 2299 (?<=abc|abde) 2300</pre> 2301In some cases, the escape sequence \K 2302<a href="#resetmatchstart">(see above)</a> 2303can be used instead of a lookbehind assertion to get round the fixed-length 2304restriction. 2305</P> 2306<P> 2307The implementation of lookbehind assertions is, for each alternative, to 2308temporarily move the current position back by the fixed length and then try to 2309match. If there are insufficient characters before the current position, the 2310assertion fails. 2311</P> 2312<P> 2313In a UTF mode, PCRE2 does not allow the \C escape (which matches a single code 2314unit even in a UTF mode) to appear in lookbehind assertions, because it makes 2315it impossible to calculate the length of the lookbehind. The \X and \R 2316escapes, which can match different numbers of code units, are also not 2317permitted. 2318</P> 2319<P> 2320<a href="#subpatternsassubroutines">"Subroutine"</a> 2321calls (see below) such as (?2) or (?&X) are permitted in lookbehinds, as long 2322as the subpattern matches a fixed-length string. 2323<a href="#recursion">Recursion,</a> 2324however, is not supported. 2325</P> 2326<P> 2327Possessive quantifiers can be used in conjunction with lookbehind assertions to 2328specify efficient matching of fixed-length strings at the end of subject 2329strings. Consider a simple pattern such as 2330<pre> 2331 abcd$ 2332</pre> 2333when applied to a long string that does not match. Because matching proceeds 2334from left to right, PCRE2 will look for each "a" in the subject and then see if 2335what follows matches the rest of the pattern. If the pattern is specified as 2336<pre> 2337 ^.*abcd$ 2338</pre> 2339the initial .* matches the entire string at first, but when this fails (because 2340there is no following "a"), it backtracks to match all but the last character, 2341then all but the last two characters, and so on. Once again the search for "a" 2342covers the entire string, from right to left, so we are no better off. However, 2343if the pattern is written as 2344<pre> 2345 ^.*+(?<=abcd) 2346</pre> 2347there can be no backtracking for the .*+ item because of the possessive 2348quantifier; it can match only the entire string. The subsequent lookbehind 2349assertion does a single test on the last four characters. If it fails, the 2350match fails immediately. For long strings, this approach makes a significant 2351difference to the processing time. 2352</P> 2353<br><b> 2354Using multiple assertions 2355</b><br> 2356<P> 2357Several assertions (of any sort) may occur in succession. For example, 2358<pre> 2359 (?<=\d{3})(?<!999)foo 2360</pre> 2361matches "foo" preceded by three digits that are not "999". Notice that each of 2362the assertions is applied independently at the same point in the subject 2363string. First there is a check that the previous three characters are all 2364digits, and then there is a check that the same three characters are not "999". 2365This pattern does <i>not</i> match "foo" preceded by six characters, the first 2366of which are digits and the last three of which are not "999". For example, it 2367doesn't match "123abcfoo". A pattern to do that is 2368<pre> 2369 (?<=\d{3}...)(?<!999)foo 2370</pre> 2371This time the first assertion looks at the preceding six characters, checking 2372that the first three are digits, and then the second assertion checks that the 2373preceding three characters are not "999". 2374</P> 2375<P> 2376Assertions can be nested in any combination. For example, 2377<pre> 2378 (?<=(?<!foo)bar)baz 2379</pre> 2380matches an occurrence of "baz" that is preceded by "bar" which in turn is not 2381preceded by "foo", while 2382<pre> 2383 (?<=\d{3}(?!999)...)foo 2384</pre> 2385is another pattern that matches "foo" preceded by three digits and any three 2386characters that are not "999". 2387<a name="conditions"></a></P> 2388<br><a name="SEC21" href="#TOC1">CONDITIONAL SUBPATTERNS</a><br> 2389<P> 2390It is possible to cause the matching process to obey a subpattern 2391conditionally or to choose between two alternative subpatterns, depending on 2392the result of an assertion, or whether a specific capturing subpattern has 2393already been matched. The two possible forms of conditional subpattern are: 2394<pre> 2395 (?(condition)yes-pattern) 2396 (?(condition)yes-pattern|no-pattern) 2397</pre> 2398If the condition is satisfied, the yes-pattern is used; otherwise the 2399no-pattern (if present) is used. If there are more than two alternatives in the 2400subpattern, a compile-time error occurs. Each of the two alternatives may 2401itself contain nested subpatterns of any form, including conditional 2402subpatterns; the restriction to two alternatives applies only at the level of 2403the condition. This pattern fragment is an example where the alternatives are 2404complex: 2405<pre> 2406 (?(1) (A|B|C) | (D | (?(2)E|F) | E) ) 2407 2408</PRE> 2409</P> 2410<P> 2411There are five kinds of condition: references to subpatterns, references to 2412recursion, two pseudo-conditions called DEFINE and VERSION, and assertions. 2413</P> 2414<br><b> 2415Checking for a used subpattern by number 2416</b><br> 2417<P> 2418If the text between the parentheses consists of a sequence of digits, the 2419condition is true if a capturing subpattern of that number has previously 2420matched. If there is more than one capturing subpattern with the same number 2421(see the earlier 2422<a href="#recursion">section about duplicate subpattern numbers),</a> 2423the condition is true if any of them have matched. An alternative notation is 2424to precede the digits with a plus or minus sign. In this case, the subpattern 2425number is relative rather than absolute. The most recently opened parentheses 2426can be referenced by (?(-1), the next most recent by (?(-2), and so on. Inside 2427loops it can also make sense to refer to subsequent groups. The next 2428parentheses to be opened can be referenced as (?(+1), and so on. (The value 2429zero in any of these forms is not used; it provokes a compile-time error.) 2430</P> 2431<P> 2432Consider the following pattern, which contains non-significant white space to 2433make it more readable (assume the PCRE2_EXTENDED option) and to divide it into 2434three parts for ease of discussion: 2435<pre> 2436 ( \( )? [^()]+ (?(1) \) ) 2437</pre> 2438The first part matches an optional opening parenthesis, and if that 2439character is present, sets it as the first captured substring. The second part 2440matches one or more characters that are not parentheses. The third part is a 2441conditional subpattern that tests whether or not the first set of parentheses 2442matched. If they did, that is, if subject started with an opening parenthesis, 2443the condition is true, and so the yes-pattern is executed and a closing 2444parenthesis is required. Otherwise, since no-pattern is not present, the 2445subpattern matches nothing. In other words, this pattern matches a sequence of 2446non-parentheses, optionally enclosed in parentheses. 2447</P> 2448<P> 2449If you were embedding this pattern in a larger one, you could use a relative 2450reference: 2451<pre> 2452 ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... 2453</pre> 2454This makes the fragment independent of the parentheses in the larger pattern. 2455</P> 2456<br><b> 2457Checking for a used subpattern by name 2458</b><br> 2459<P> 2460Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used 2461subpattern by name. For compatibility with earlier versions of PCRE1, which had 2462this facility before Perl, the syntax (?(name)...) is also recognized. 2463</P> 2464<P> 2465Rewriting the above example to use a named subpattern gives this: 2466<pre> 2467 (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) 2468</pre> 2469If the name used in a condition of this kind is a duplicate, the test is 2470applied to all subpatterns of the same name, and is true if any one of them has 2471matched. 2472</P> 2473<br><b> 2474Checking for pattern recursion 2475</b><br> 2476<P> 2477If the condition is the string (R), and there is no subpattern with the name R, 2478the condition is true if a recursive call to the whole pattern or any 2479subpattern has been made. If digits or a name preceded by ampersand follow the 2480letter R, for example: 2481<pre> 2482 (?(R3)...) or (?(R&name)...) 2483</pre> 2484the condition is true if the most recent recursion is into a subpattern whose 2485number or name is given. This condition does not check the entire recursion 2486stack. If the name used in a condition of this kind is a duplicate, the test is 2487applied to all subpatterns of the same name, and is true if any one of them is 2488the most recent recursion. 2489</P> 2490<P> 2491At "top level", all these recursion test conditions are false. 2492<a href="#recursion">The syntax for recursive patterns</a> 2493is described below. 2494<a name="subdefine"></a></P> 2495<br><b> 2496Defining subpatterns for use by reference only 2497</b><br> 2498<P> 2499If the condition is the string (DEFINE), and there is no subpattern with the 2500name DEFINE, the condition is always false. In this case, there may be only one 2501alternative in the subpattern. It is always skipped if control reaches this 2502point in the pattern; the idea of DEFINE is that it can be used to define 2503subroutines that can be referenced from elsewhere. (The use of 2504<a href="#subpatternsassubroutines">subroutines</a> 2505is described below.) For example, a pattern to match an IPv4 address such as 2506"192.168.23.245" could be written like this (ignore white space and line 2507breaks): 2508<pre> 2509 (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) 2510 \b (?&byte) (\.(?&byte)){3} \b 2511</pre> 2512The first part of the pattern is a DEFINE group inside which a another group 2513named "byte" is defined. This matches an individual component of an IPv4 2514address (a number less than 256). When matching takes place, this part of the 2515pattern is skipped because DEFINE acts like a false condition. The rest of the 2516pattern uses references to the named group to match the four dot-separated 2517components of an IPv4 address, insisting on a word boundary at each end. 2518</P> 2519<br><b> 2520Checking the PCRE2 version 2521</b><br> 2522<P> 2523Programs that link with a PCRE2 library can check the version by calling 2524<b>pcre2_config()</b> with appropriate arguments. Users of applications that do 2525not have access to the underlying code cannot do this. A special "condition" 2526called VERSION exists to allow such users to discover which version of PCRE2 2527they are dealing with by using this condition to match a string such as 2528"yesno". VERSION must be followed either by "=" or ">=" and a version number. 2529For example: 2530<pre> 2531 (?(VERSION>=10.4)yes|no) 2532</pre> 2533This pattern matches "yes" if the PCRE2 version is greater or equal to 10.4, or 2534"no" otherwise. The fractional part of the version number may not contain more 2535than two digits. 2536</P> 2537<br><b> 2538Assertion conditions 2539</b><br> 2540<P> 2541If the condition is not in any of the above formats, it must be an assertion. 2542This may be a positive or negative lookahead or lookbehind assertion. Consider 2543this pattern, again containing non-significant white space, and with the two 2544alternatives on the second line: 2545<pre> 2546 (?(?=[^a-z]*[a-z]) 2547 \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) 2548</pre> 2549The condition is a positive lookahead assertion that matches an optional 2550sequence of non-letters followed by a letter. In other words, it tests for the 2551presence of at least one letter in the subject. If a letter is found, the 2552subject is matched against the first alternative; otherwise it is matched 2553against the second. This pattern matches strings in one of the two forms 2554dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. 2555<a name="comments"></a></P> 2556<br><a name="SEC22" href="#TOC1">COMMENTS</a><br> 2557<P> 2558There are two ways of including comments in patterns that are processed by 2559PCRE2. In both cases, the start of the comment must not be in a character 2560class, nor in the middle of any other sequence of related characters such as 2561(?: or a subpattern name or number. The characters that make up a comment play 2562no part in the pattern matching. 2563</P> 2564<P> 2565The sequence (?# marks the start of a comment that continues up to the next 2566closing parenthesis. Nested parentheses are not permitted. If the 2567PCRE2_EXTENDED option is set, an unescaped # character also introduces a 2568comment, which in this case continues to immediately after the next newline 2569character or character sequence in the pattern. Which characters are 2570interpreted as newlines is controlled by an option passed to the compiling 2571function or by a special sequence at the start of the pattern, as described in 2572the section entitled 2573<a href="#newlines">"Newline conventions"</a> 2574above. Note that the end of this type of comment is a literal newline sequence 2575in the pattern; escape sequences that happen to represent a newline do not 2576count. For example, consider this pattern when PCRE2_EXTENDED is set, and the 2577default newline convention (a single linefeed character) is in force: 2578<pre> 2579 abc #comment \n still comment 2580</pre> 2581On encountering the # character, <b>pcre2_compile()</b> skips along, looking for 2582a newline in the pattern. The sequence \n is still literal at this stage, so 2583it does not terminate the comment. Only an actual character with the code value 25840x0a (the default newline) does so. 2585<a name="recursion"></a></P> 2586<br><a name="SEC23" href="#TOC1">RECURSIVE PATTERNS</a><br> 2587<P> 2588Consider the problem of matching a string in parentheses, allowing for 2589unlimited nested parentheses. Without the use of recursion, the best that can 2590be done is to use a pattern that matches up to some fixed depth of nesting. It 2591is not possible to handle an arbitrary nesting depth. 2592</P> 2593<P> 2594For some time, Perl has provided a facility that allows regular expressions to 2595recurse (amongst other things). It does this by interpolating Perl code in the 2596expression at run time, and the code can refer to the expression itself. A Perl 2597pattern using code interpolation to solve the parentheses problem can be 2598created like this: 2599<pre> 2600 $re = qr{\( (?: (?>[^()]+) | (?p{$re}) )* \)}x; 2601</pre> 2602The (?p{...}) item interpolates Perl code at run time, and in this case refers 2603recursively to the pattern in which it appears. 2604</P> 2605<P> 2606Obviously, PCRE2 cannot support the interpolation of Perl code. Instead, it 2607supports special syntax for recursion of the entire pattern, and also for 2608individual subpattern recursion. After its introduction in PCRE1 and Python, 2609this kind of recursion was subsequently introduced into Perl at release 5.10. 2610</P> 2611<P> 2612A special item that consists of (? followed by a number greater than zero and a 2613closing parenthesis is a recursive subroutine call of the subpattern of the 2614given number, provided that it occurs inside that subpattern. (If not, it is a 2615<a href="#subpatternsassubroutines">non-recursive subroutine</a> 2616call, which is described in the next section.) The special item (?R) or (?0) is 2617a recursive call of the entire regular expression. 2618</P> 2619<P> 2620This PCRE2 pattern solves the nested parentheses problem (assume the 2621PCRE2_EXTENDED option is set so that white space is ignored): 2622<pre> 2623 \( ( [^()]++ | (?R) )* \) 2624</pre> 2625First it matches an opening parenthesis. Then it matches any number of 2626substrings which can either be a sequence of non-parentheses, or a recursive 2627match of the pattern itself (that is, a correctly parenthesized substring). 2628Finally there is a closing parenthesis. Note the use of a possessive quantifier 2629to avoid backtracking into sequences of non-parentheses. 2630</P> 2631<P> 2632If this were part of a larger pattern, you would not want to recurse the entire 2633pattern, so instead you could use this: 2634<pre> 2635 ( \( ( [^()]++ | (?1) )* \) ) 2636</pre> 2637We have put the pattern into parentheses, and caused the recursion to refer to 2638them instead of the whole pattern. 2639</P> 2640<P> 2641In a larger pattern, keeping track of parenthesis numbers can be tricky. This 2642is made easier by the use of relative references. Instead of (?1) in the 2643pattern above you can write (?-2) to refer to the second most recently opened 2644parentheses preceding the recursion. In other words, a negative number counts 2645capturing parentheses leftwards from the point at which it is encountered. 2646</P> 2647<P> 2648Be aware however, that if 2649<a href="#dupsubpatternnumber">duplicate subpattern numbers</a> 2650are in use, relative references refer to the earliest subpattern with the 2651appropriate number. Consider, for example: 2652<pre> 2653 (?|(a)|(b)) (c) (?-2) 2654</pre> 2655The first two capturing groups (a) and (b) are both numbered 1, and group (c) 2656is number 2. When the reference (?-2) is encountered, the second most recently 2657opened parentheses has the number 1, but it is the first such group (the (a) 2658group) to which the recursion refers. This would be the same if an absolute 2659reference (?1) was used. In other words, relative references are just a 2660shorthand for computing a group number. 2661</P> 2662<P> 2663It is also possible to refer to subsequently opened parentheses, by writing 2664references such as (?+2). However, these cannot be recursive because the 2665reference is not inside the parentheses that are referenced. They are always 2666<a href="#subpatternsassubroutines">non-recursive subroutine</a> 2667calls, as described in the next section. 2668</P> 2669<P> 2670An alternative approach is to use named parentheses. The Perl syntax for this 2671is (?&name); PCRE1's earlier syntax (?P>name) is also supported. We could 2672rewrite the above example as follows: 2673<pre> 2674 (?<pn> \( ( [^()]++ | (?&pn) )* \) ) 2675</pre> 2676If there is more than one subpattern with the same name, the earliest one is 2677used. 2678</P> 2679<P> 2680The example pattern that we have been looking at contains nested unlimited 2681repeats, and so the use of a possessive quantifier for matching strings of 2682non-parentheses is important when applying the pattern to strings that do not 2683match. For example, when this pattern is applied to 2684<pre> 2685 (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() 2686</pre> 2687it yields "no match" quickly. However, if a possessive quantifier is not used, 2688the match runs for a very long time indeed because there are so many different 2689ways the + and * repeats can carve up the subject, and all have to be tested 2690before failure can be reported. 2691</P> 2692<P> 2693At the end of a match, the values of capturing parentheses are those from 2694the outermost level. If you want to obtain intermediate values, a callout 2695function can be used (see below and the 2696<a href="pcre2callout.html"><b>pcre2callout</b></a> 2697documentation). If the pattern above is matched against 2698<pre> 2699 (ab(cd)ef) 2700</pre> 2701the value for the inner capturing parentheses (numbered 2) is "ef", which is 2702the last value taken on at the top level. If a capturing subpattern is not 2703matched at the top level, its final captured value is unset, even if it was 2704(temporarily) set at a deeper level during the matching process. 2705</P> 2706<P> 2707If there are more than 15 capturing parentheses in a pattern, PCRE2 has to 2708obtain extra memory from the heap to store data during a recursion. If no 2709memory can be obtained, the match fails with the PCRE2_ERROR_NOMEMORY error. 2710</P> 2711<P> 2712Do not confuse the (?R) item with the condition (R), which tests for recursion. 2713Consider this pattern, which matches text in angle brackets, allowing for 2714arbitrary nesting. Only digits are allowed in nested brackets (that is, when 2715recursing), whereas any characters are permitted at the outer level. 2716<pre> 2717 < (?: (?(R) \d++ | [^<>]*+) | (?R)) * > 2718</pre> 2719In this pattern, (?(R) is the start of a conditional subpattern, with two 2720different alternatives for the recursive and non-recursive cases. The (?R) item 2721is the actual recursive call. 2722<a name="recursiondifference"></a></P> 2723<br><b> 2724Differences in recursion processing between PCRE2 and Perl 2725</b><br> 2726<P> 2727Recursion processing in PCRE2 differs from Perl in two important ways. In PCRE2 2728(like Python, but unlike Perl), a recursive subpattern call is always treated 2729as an atomic group. That is, once it has matched some of the subject string, it 2730is never re-entered, even if it contains untried alternatives and there is a 2731subsequent matching failure. This can be illustrated by the following pattern, 2732which purports to match a palindromic string that contains an odd number of 2733characters (for example, "a", "aba", "abcba", "abcdcba"): 2734<pre> 2735 ^(.|(.)(?1)\2)$ 2736</pre> 2737The idea is that it either matches a single character, or two identical 2738characters surrounding a sub-palindrome. In Perl, this pattern works; in PCRE2 2739it does not if the pattern is longer than three characters. Consider the 2740subject string "abcba": 2741</P> 2742<P> 2743At the top level, the first character is matched, but as it is not at the end 2744of the string, the first alternative fails; the second alternative is taken 2745and the recursion kicks in. The recursive call to subpattern 1 successfully 2746matches the next character ("b"). (Note that the beginning and end of line 2747tests are not part of the recursion). 2748</P> 2749<P> 2750Back at the top level, the next character ("c") is compared with what 2751subpattern 2 matched, which was "a". This fails. Because the recursion is 2752treated as an atomic group, there are now no backtracking points, and so the 2753entire match fails. (Perl is able, at this point, to re-enter the recursion and 2754try the second alternative.) However, if the pattern is written with the 2755alternatives in the other order, things are different: 2756<pre> 2757 ^((.)(?1)\2|.)$ 2758</pre> 2759This time, the recursing alternative is tried first, and continues to recurse 2760until it runs out of characters, at which point the recursion fails. But this 2761time we do have another alternative to try at the higher level. That is the big 2762difference: in the previous case the remaining alternative is at a deeper 2763recursion level, which PCRE2 cannot use. 2764</P> 2765<P> 2766To change the pattern so that it matches all palindromic strings, not just 2767those with an odd number of characters, it is tempting to change the pattern to 2768this: 2769<pre> 2770 ^((.)(?1)\2|.?)$ 2771</pre> 2772Again, this works in Perl, but not in PCRE2, and for the same reason. When a 2773deeper recursion has matched a single character, it cannot be entered again in 2774order to match an empty string. The solution is to separate the two cases, and 2775write out the odd and even cases as alternatives at the higher level: 2776<pre> 2777 ^(?:((.)(?1)\2|)|((.)(?3)\4|.)) 2778</pre> 2779If you want to match typical palindromic phrases, the pattern has to ignore all 2780non-word characters, which can be done like this: 2781<pre> 2782 ^\W*+(?:((.)\W*+(?1)\W*+\2|)|((.)\W*+(?3)\W*+\4|\W*+.\W*+))\W*+$ 2783</pre> 2784If run with the PCRE2_CASELESS option, this pattern matches phrases such as "A 2785man, a plan, a canal: Panama!" and it works in both PCRE2 and Perl. Note the 2786use of the possessive quantifier *+ to avoid backtracking into sequences of 2787non-word characters. Without this, PCRE2 takes a great deal longer (ten times 2788or more) to match typical phrases, and Perl takes so long that you think it has 2789gone into a loop. 2790</P> 2791<P> 2792<b>WARNING</b>: The palindrome-matching patterns above work only if the subject 2793string does not start with a palindrome that is shorter than the entire string. 2794For example, although "abcba" is correctly matched, if the subject is "ababa", 2795PCRE2 finds the palindrome "aba" at the start, then fails at top level because 2796the end of the string does not follow. Once again, it cannot jump back into the 2797recursion to try other alternatives, so the entire match fails. 2798</P> 2799<P> 2800The second way in which PCRE2 and Perl differ in their recursion processing is 2801in the handling of captured values. In Perl, when a subpattern is called 2802recursively or as a subpattern (see the next section), it has no access to any 2803values that were captured outside the recursion, whereas in PCRE2 these values 2804can be referenced. Consider this pattern: 2805<pre> 2806 ^(.)(\1|a(?2)) 2807</pre> 2808In PCRE2, this pattern matches "bab". The first capturing parentheses match "b", 2809then in the second group, when the back reference \1 fails to match "b", the 2810second alternative matches "a" and then recurses. In the recursion, \1 does 2811now match "b" and so the whole match succeeds. In Perl, the pattern fails to 2812match because inside the recursive call \1 cannot access the externally set 2813value. 2814<a name="subpatternsassubroutines"></a></P> 2815<br><a name="SEC24" href="#TOC1">SUBPATTERNS AS SUBROUTINES</a><br> 2816<P> 2817If the syntax for a recursive subpattern call (either by number or by 2818name) is used outside the parentheses to which it refers, it operates like a 2819subroutine in a programming language. The called subpattern may be defined 2820before or after the reference. A numbered reference can be absolute or 2821relative, as in these examples: 2822<pre> 2823 (...(absolute)...)...(?2)... 2824 (...(relative)...)...(?-1)... 2825 (...(?+1)...(relative)... 2826</pre> 2827An earlier example pointed out that the pattern 2828<pre> 2829 (sens|respons)e and \1ibility 2830</pre> 2831matches "sense and sensibility" and "response and responsibility", but not 2832"sense and responsibility". If instead the pattern 2833<pre> 2834 (sens|respons)e and (?1)ibility 2835</pre> 2836is used, it does match "sense and responsibility" as well as the other two 2837strings. Another example is given in the discussion of DEFINE above. 2838</P> 2839<P> 2840All subroutine calls, whether recursive or not, are always treated as atomic 2841groups. That is, once a subroutine has matched some of the subject string, it 2842is never re-entered, even if it contains untried alternatives and there is a 2843subsequent matching failure. Any capturing parentheses that are set during the 2844subroutine call revert to their previous values afterwards. 2845</P> 2846<P> 2847Processing options such as case-independence are fixed when a subpattern is 2848defined, so if it is used as a subroutine, such options cannot be changed for 2849different calls. For example, consider this pattern: 2850<pre> 2851 (abc)(?i:(?-1)) 2852</pre> 2853It matches "abcabc". It does not match "abcABC" because the change of 2854processing option does not affect the called subpattern. 2855<a name="onigurumasubroutines"></a></P> 2856<br><a name="SEC25" href="#TOC1">ONIGURUMA SUBROUTINE SYNTAX</a><br> 2857<P> 2858For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or 2859a number enclosed either in angle brackets or single quotes, is an alternative 2860syntax for referencing a subpattern as a subroutine, possibly recursively. Here 2861are two of the examples used above, rewritten using this syntax: 2862<pre> 2863 (?<pn> \( ( (?>[^()]+) | \g<pn> )* \) ) 2864 (sens|respons)e and \g'1'ibility 2865</pre> 2866PCRE2 supports an extension to Oniguruma: if a number is preceded by a 2867plus or a minus sign it is taken as a relative reference. For example: 2868<pre> 2869 (abc)(?i:\g<-1>) 2870</pre> 2871Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> 2872synonymous. The former is a back reference; the latter is a subroutine call. 2873</P> 2874<br><a name="SEC26" href="#TOC1">CALLOUTS</a><br> 2875<P> 2876Perl has a feature whereby using the sequence (?{...}) causes arbitrary Perl 2877code to be obeyed in the middle of matching a regular expression. This makes it 2878possible, amongst other things, to extract different substrings that match the 2879same pair of parentheses when there is a repetition. 2880</P> 2881<P> 2882PCRE2 provides a similar feature, but of course it cannot obey arbitrary Perl 2883code. The feature is called "callout". The caller of PCRE2 provides an external 2884function by putting its entry point in a match context using the function 2885<b>pcre2_set_callout()</b>, and then passing that context to <b>pcre2_match()</b> 2886or <b>pcre2_dfa_match()</b>. If no match context is passed, or if the callout 2887entry point is set to NULL, callouts are disabled. 2888</P> 2889<P> 2890Within a regular expression, (?C<arg>) indicates a point at which the external 2891function is to be called. There are two kinds of callout: those with a 2892numerical argument and those with a string argument. (?C) on its own with no 2893argument is treated as (?C0). A numerical argument allows the application to 2894distinguish between different callouts. String arguments were added for release 289510.20 to make it possible for script languages that use PCRE2 to embed short 2896scripts within patterns in a similar way to Perl. 2897</P> 2898<P> 2899During matching, when PCRE2 reaches a callout point, the external function is 2900called. It is provided with the number or string argument of the callout, the 2901position in the pattern, and one item of data that is also set in the match 2902block. The callout function may cause matching to proceed, to backtrack, or to 2903fail. 2904</P> 2905<P> 2906By default, PCRE2 implements a number of optimizations at matching time, and 2907one side-effect is that sometimes callouts are skipped. If you need all 2908possible callouts to happen, you need to set options that disable the relevant 2909optimizations. More details, including a complete description of the 2910programming interface to the callout function, are given in the 2911<a href="pcre2callout.html"><b>pcre2callout</b></a> 2912documentation. 2913</P> 2914<br><b> 2915Callouts with numerical arguments 2916</b><br> 2917<P> 2918If you just want to have a means of identifying different callout points, put a 2919number less than 256 after the letter C. For example, this pattern has two 2920callout points: 2921<pre> 2922 (?C1)abc(?C2)def 2923</pre> 2924If the PCRE2_AUTO_CALLOUT flag is passed to <b>pcre2_compile()</b>, numerical 2925callouts are automatically installed before each item in the pattern. They are 2926all numbered 255. If there is a conditional group in the pattern whose 2927condition is an assertion, an additional callout is inserted just before the 2928condition. An explicit callout may also be set at this position, as in this 2929example: 2930<pre> 2931 (?(?C9)(?=a)abc|def) 2932</pre> 2933Note that this applies only to assertion conditions, not to other types of 2934condition. 2935</P> 2936<br><b> 2937Callouts with string arguments 2938</b><br> 2939<P> 2940A delimited string may be used instead of a number as a callout argument. The 2941starting delimiter must be one of ` ' " ^ % # $ { and the ending delimiter is 2942the same as the start, except for {, where the ending delimiter is }. If the 2943ending delimiter is needed within the string, it must be doubled. For 2944example: 2945<pre> 2946 (?C'ab ''c'' d')xyz(?C{any text})pqr 2947</pre> 2948The doubling is removed before the string is passed to the callout function. 2949<a name="backtrackcontrol"></a></P> 2950<br><a name="SEC27" href="#TOC1">BACKTRACKING CONTROL</a><br> 2951<P> 2952Perl 5.10 introduced a number of "Special Backtracking Control Verbs", which 2953are still described in the Perl documentation as "experimental and subject to 2954change or removal in a future version of Perl". It goes on to say: "Their usage 2955in production code should be noted to avoid problems during upgrades." The same 2956remarks apply to the PCRE2 features described in this section. 2957</P> 2958<P> 2959The new verbs make use of what was previously invalid syntax: an opening 2960parenthesis followed by an asterisk. They are generally of the form (*VERB) or 2961(*VERB:NAME). Some verbs take either form, possibly behaving differently 2962depending on whether or not a name is present. 2963</P> 2964<P> 2965By default, for compatibility with Perl, a name is any sequence of characters 2966that does not include a closing parenthesis. The name is not processed in 2967any way, and it is not possible to include a closing parenthesis in the name. 2968However, if the PCRE2_ALT_VERBNAMES option is set, normal backslash processing 2969is applied to verb names and only an unescaped closing parenthesis terminates 2970the name. A closing parenthesis can be included in a name either as \) or 2971between \Q and \E. If the PCRE2_EXTENDED option is set, unescaped whitespace 2972in verb names is skipped and #-comments are recognized, exactly as in the rest 2973of the pattern. 2974</P> 2975<P> 2976The maximum length of a name is 255 in the 8-bit library and 65535 in the 297716-bit and 32-bit libraries. If the name is empty, that is, if the closing 2978parenthesis immediately follows the colon, the effect is as if the colon were 2979not there. Any number of these verbs may occur in a pattern. 2980</P> 2981<P> 2982Since these verbs are specifically related to backtracking, most of them can be 2983used only when the pattern is to be matched using the traditional matching 2984function, because these use a backtracking algorithm. With the exception of 2985(*FAIL), which behaves like a failing negative assertion, the backtracking 2986control verbs cause an error if encountered by the DFA matching function. 2987</P> 2988<P> 2989The behaviour of these verbs in 2990<a href="#btrepeat">repeated groups,</a> 2991<a href="#btassert">assertions,</a> 2992and in 2993<a href="#btsub">subpatterns called as subroutines</a> 2994(whether or not recursively) is documented below. 2995<a name="nooptimize"></a></P> 2996<br><b> 2997Optimizations that affect backtracking verbs 2998</b><br> 2999<P> 3000PCRE2 contains some optimizations that are used to speed up matching by running 3001some checks at the start of each match attempt. For example, it may know the 3002minimum length of matching subject, or that a particular character must be 3003present. When one of these optimizations bypasses the running of a match, any 3004included backtracking verbs will not, of course, be processed. You can suppress 3005the start-of-match optimizations by setting the PCRE2_NO_START_OPTIMIZE option 3006when calling <b>pcre2_compile()</b>, or by starting the pattern with 3007(*NO_START_OPT). There is more discussion of this option in the section 3008entitled 3009<a href="pcre2api.html#compiling">"Compiling a pattern"</a> 3010in the 3011<a href="pcre2api.html"><b>pcre2api</b></a> 3012documentation. 3013</P> 3014<P> 3015Experiments with Perl suggest that it too has similar optimizations, sometimes 3016leading to anomalous results. 3017</P> 3018<br><b> 3019Verbs that act immediately 3020</b><br> 3021<P> 3022The following verbs act as soon as they are encountered. They may not be 3023followed by a name. 3024<pre> 3025 (*ACCEPT) 3026</pre> 3027This verb causes the match to end successfully, skipping the remainder of the 3028pattern. However, when it is inside a subpattern that is called as a 3029subroutine, only that subpattern is ended successfully. Matching then continues 3030at the outer level. If (*ACCEPT) in triggered in a positive assertion, the 3031assertion succeeds; in a negative assertion, the assertion fails. 3032</P> 3033<P> 3034If (*ACCEPT) is inside capturing parentheses, the data so far is captured. For 3035example: 3036<pre> 3037 A((?:A|B(*ACCEPT)|C)D) 3038</pre> 3039This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is captured by 3040the outer parentheses. 3041<pre> 3042 (*FAIL) or (*F) 3043</pre> 3044This verb causes a matching failure, forcing backtracking to occur. It is 3045equivalent to (?!) but easier to read. The Perl documentation notes that it is 3046probably useful only when combined with (?{}) or (??{}). Those are, of course, 3047Perl features that are not present in PCRE2. The nearest equivalent is the 3048callout feature, as for example in this pattern: 3049<pre> 3050 a+(?C)(*FAIL) 3051</pre> 3052A match with the string "aaaa" always fails, but the callout is taken before 3053each backtrack happens (in this example, 10 times). 3054</P> 3055<br><b> 3056Recording which path was taken 3057</b><br> 3058<P> 3059There is one verb whose main purpose is to track how a match was arrived at, 3060though it also has a secondary use in conjunction with advancing the match 3061starting point (see (*SKIP) below). 3062<pre> 3063 (*MARK:NAME) or (*:NAME) 3064</pre> 3065A name is always required with this verb. There may be as many instances of 3066(*MARK) as you like in a pattern, and their names do not have to be unique. 3067</P> 3068<P> 3069When a match succeeds, the name of the last-encountered (*MARK:NAME), 3070(*PRUNE:NAME), or (*THEN:NAME) on the matching path is passed back to the 3071caller as described in the section entitled 3072<a href="pcre2api.html#matchotherdata">"Other information about the match"</a> 3073in the 3074<a href="pcre2api.html"><b>pcre2api</b></a> 3075documentation. Here is an example of <b>pcre2test</b> output, where the "mark" 3076modifier requests the retrieval and outputting of (*MARK) data: 3077<pre> 3078 re> /X(*MARK:A)Y|X(*MARK:B)Z/mark 3079 data> XY 3080 0: XY 3081 MK: A 3082 XZ 3083 0: XZ 3084 MK: B 3085</pre> 3086The (*MARK) name is tagged with "MK:" in this output, and in this example it 3087indicates which of the two alternatives matched. This is a more efficient way 3088of obtaining this information than putting each alternative in its own 3089capturing parentheses. 3090</P> 3091<P> 3092If a verb with a name is encountered in a positive assertion that is true, the 3093name is recorded and passed back if it is the last-encountered. This does not 3094happen for negative assertions or failing positive assertions. 3095</P> 3096<P> 3097After a partial match or a failed match, the last encountered name in the 3098entire match process is returned. For example: 3099<pre> 3100 re> /X(*MARK:A)Y|X(*MARK:B)Z/mark 3101 data> XP 3102 No match, mark = B 3103</pre> 3104Note that in this unanchored example the mark is retained from the match 3105attempt that started at the letter "X" in the subject. Subsequent match 3106attempts starting at "P" and then with an empty string do not get as far as the 3107(*MARK) item, but nevertheless do not reset it. 3108</P> 3109<P> 3110If you are interested in (*MARK) values after failed matches, you should 3111probably set the PCRE2_NO_START_OPTIMIZE option 3112<a href="#nooptimize">(see above)</a> 3113to ensure that the match is always attempted. 3114</P> 3115<br><b> 3116Verbs that act after backtracking 3117</b><br> 3118<P> 3119The following verbs do nothing when they are encountered. Matching continues 3120with what follows, but if there is no subsequent match, causing a backtrack to 3121the verb, a failure is forced. That is, backtracking cannot pass to the left of 3122the verb. However, when one of these verbs appears inside an atomic group 3123(which includes any group that is called as a subroutine) or in an assertion 3124that is true, its effect is confined to that group, because once the group has 3125been matched, there is never any backtracking into it. In this situation, 3126backtracking has to jump to the left of the entire atomic group or assertion. 3127</P> 3128<P> 3129These verbs differ in exactly what kind of failure occurs when backtracking 3130reaches them. The behaviour described below is what happens when the verb is 3131not in a subroutine or an assertion. Subsequent sections cover these special 3132cases. 3133<pre> 3134 (*COMMIT) 3135</pre> 3136This verb, which may not be followed by a name, causes the whole match to fail 3137outright if there is a later matching failure that causes backtracking to reach 3138it. Even if the pattern is unanchored, no further attempts to find a match by 3139advancing the starting point take place. If (*COMMIT) is the only backtracking 3140verb that is encountered, once it has been passed <b>pcre2_match()</b> is 3141committed to finding a match at the current starting point, or not at all. For 3142example: 3143<pre> 3144 a+(*COMMIT)b 3145</pre> 3146This matches "xxaab" but not "aacaab". It can be thought of as a kind of 3147dynamic anchor, or "I've started, so I must finish." The name of the most 3148recently passed (*MARK) in the path is passed back when (*COMMIT) forces a 3149match failure. 3150</P> 3151<P> 3152If there is more than one backtracking verb in a pattern, a different one that 3153follows (*COMMIT) may be triggered first, so merely passing (*COMMIT) during a 3154match does not always guarantee that a match must be at this starting point. 3155</P> 3156<P> 3157Note that (*COMMIT) at the start of a pattern is not the same as an anchor, 3158unless PCRE2's start-of-match optimizations are turned off, as shown in this 3159output from <b>pcre2test</b>: 3160<pre> 3161 re> /(*COMMIT)abc/ 3162 data> xyzabc 3163 0: abc 3164 data> 3165 re> /(*COMMIT)abc/no_start_optimize 3166 data> xyzabc 3167 No match 3168</pre> 3169For the first pattern, PCRE2 knows that any match must start with "a", so the 3170optimization skips along the subject to "a" before applying the pattern to the 3171first set of data. The match attempt then succeeds. The second pattern disables 3172the optimization that skips along to the first character. The pattern is now 3173applied starting at "x", and so the (*COMMIT) causes the match to fail without 3174trying any other starting points. 3175<pre> 3176 (*PRUNE) or (*PRUNE:NAME) 3177</pre> 3178This verb causes the match to fail at the current starting position in the 3179subject if there is a later matching failure that causes backtracking to reach 3180it. If the pattern is unanchored, the normal "bumpalong" advance to the next 3181starting character then happens. Backtracking can occur as usual to the left of 3182(*PRUNE), before it is reached, or when matching to the right of (*PRUNE), but 3183if there is no match to the right, backtracking cannot cross (*PRUNE). In 3184simple cases, the use of (*PRUNE) is just an alternative to an atomic group or 3185possessive quantifier, but there are some uses of (*PRUNE) that cannot be 3186expressed in any other way. In an anchored pattern (*PRUNE) has the same effect 3187as (*COMMIT). 3188</P> 3189<P> 3190The behaviour of (*PRUNE:NAME) is the not the same as (*MARK:NAME)(*PRUNE). 3191It is like (*MARK:NAME) in that the name is remembered for passing back to the 3192caller. However, (*SKIP:NAME) searches only for names set with (*MARK), 3193ignoring those set by (*PRUNE) or (*THEN). 3194<pre> 3195 (*SKIP) 3196</pre> 3197This verb, when given without a name, is like (*PRUNE), except that if the 3198pattern is unanchored, the "bumpalong" advance is not to the next character, 3199but to the position in the subject where (*SKIP) was encountered. (*SKIP) 3200signifies that whatever text was matched leading up to it cannot be part of a 3201successful match. Consider: 3202<pre> 3203 a+(*SKIP)b 3204</pre> 3205If the subject is "aaaac...", after the first match attempt fails (starting at 3206the first character in the string), the starting point skips on to start the 3207next attempt at "c". Note that a possessive quantifer does not have the same 3208effect as this example; although it would suppress backtracking during the 3209first match attempt, the second attempt would start at the second character 3210instead of skipping on to "c". 3211<pre> 3212 (*SKIP:NAME) 3213</pre> 3214When (*SKIP) has an associated name, its behaviour is modified. When it is 3215triggered, the previous path through the pattern is searched for the most 3216recent (*MARK) that has the same name. If one is found, the "bumpalong" advance 3217is to the subject position that corresponds to that (*MARK) instead of to where 3218(*SKIP) was encountered. If no (*MARK) with a matching name is found, the 3219(*SKIP) is ignored. 3220</P> 3221<P> 3222Note that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It ignores 3223names that are set by (*PRUNE:NAME) or (*THEN:NAME). 3224<pre> 3225 (*THEN) or (*THEN:NAME) 3226</pre> 3227This verb causes a skip to the next innermost alternative when backtracking 3228reaches it. That is, it cancels any further backtracking within the current 3229alternative. Its name comes from the observation that it can be used for a 3230pattern-based if-then-else block: 3231<pre> 3232 ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... 3233</pre> 3234If the COND1 pattern matches, FOO is tried (and possibly further items after 3235the end of the group if FOO succeeds); on failure, the matcher skips to the 3236second alternative and tries COND2, without backtracking into COND1. If that 3237succeeds and BAR fails, COND3 is tried. If subsequently BAZ fails, there are no 3238more alternatives, so there is a backtrack to whatever came before the entire 3239group. If (*THEN) is not inside an alternation, it acts like (*PRUNE). 3240</P> 3241<P> 3242The behaviour of (*THEN:NAME) is the not the same as (*MARK:NAME)(*THEN). 3243It is like (*MARK:NAME) in that the name is remembered for passing back to the 3244caller. However, (*SKIP:NAME) searches only for names set with (*MARK), 3245ignoring those set by (*PRUNE) and (*THEN). 3246</P> 3247<P> 3248A subpattern that does not contain a | character is just a part of the 3249enclosing alternative; it is not a nested alternation with only one 3250alternative. The effect of (*THEN) extends beyond such a subpattern to the 3251enclosing alternative. Consider this pattern, where A, B, etc. are complex 3252pattern fragments that do not contain any | characters at this level: 3253<pre> 3254 A (B(*THEN)C) | D 3255</pre> 3256If A and B are matched, but there is a failure in C, matching does not 3257backtrack into A; instead it moves to the next alternative, that is, D. 3258However, if the subpattern containing (*THEN) is given an alternative, it 3259behaves differently: 3260<pre> 3261 A (B(*THEN)C | (*FAIL)) | D 3262</pre> 3263The effect of (*THEN) is now confined to the inner subpattern. After a failure 3264in C, matching moves to (*FAIL), which causes the whole subpattern to fail 3265because there are no more alternatives to try. In this case, matching does now 3266backtrack into A. 3267</P> 3268<P> 3269Note that a conditional subpattern is not considered as having two 3270alternatives, because only one is ever used. In other words, the | character in 3271a conditional subpattern has a different meaning. Ignoring white space, 3272consider: 3273<pre> 3274 ^.*? (?(?=a) a | b(*THEN)c ) 3275</pre> 3276If the subject is "ba", this pattern does not match. Because .*? is ungreedy, 3277it initially matches zero characters. The condition (?=a) then fails, the 3278character "b" is matched, but "c" is not. At this point, matching does not 3279backtrack to .*? as might perhaps be expected from the presence of the | 3280character. The conditional subpattern is part of the single alternative that 3281comprises the whole pattern, and so the match fails. (If there was a backtrack 3282into .*?, allowing it to match "b", the match would succeed.) 3283</P> 3284<P> 3285The verbs just described provide four different "strengths" of control when 3286subsequent matching fails. (*THEN) is the weakest, carrying on the match at the 3287next alternative. (*PRUNE) comes next, failing the match at the current 3288starting position, but allowing an advance to the next character (for an 3289unanchored pattern). (*SKIP) is similar, except that the advance may be more 3290than one character. (*COMMIT) is the strongest, causing the entire match to 3291fail. 3292</P> 3293<br><b> 3294More than one backtracking verb 3295</b><br> 3296<P> 3297If more than one backtracking verb is present in a pattern, the one that is 3298backtracked onto first acts. For example, consider this pattern, where A, B, 3299etc. are complex pattern fragments: 3300<pre> 3301 (A(*COMMIT)B(*THEN)C|ABD) 3302</pre> 3303If A matches but B fails, the backtrack to (*COMMIT) causes the entire match to 3304fail. However, if A and B match, but C fails, the backtrack to (*THEN) causes 3305the next alternative (ABD) to be tried. This behaviour is consistent, but is 3306not always the same as Perl's. It means that if two or more backtracking verbs 3307appear in succession, all the the last of them has no effect. Consider this 3308example: 3309<pre> 3310 ...(*COMMIT)(*PRUNE)... 3311</pre> 3312If there is a matching failure to the right, backtracking onto (*PRUNE) causes 3313it to be triggered, and its action is taken. There can never be a backtrack 3314onto (*COMMIT). 3315<a name="btrepeat"></a></P> 3316<br><b> 3317Backtracking verbs in repeated groups 3318</b><br> 3319<P> 3320PCRE2 differs from Perl in its handling of backtracking verbs in repeated 3321groups. For example, consider: 3322<pre> 3323 /(a(*COMMIT)b)+ac/ 3324</pre> 3325If the subject is "abac", Perl matches, but PCRE2 fails because the (*COMMIT) 3326in the second repeat of the group acts. 3327<a name="btassert"></a></P> 3328<br><b> 3329Backtracking verbs in assertions 3330</b><br> 3331<P> 3332(*FAIL) in an assertion has its normal effect: it forces an immediate 3333backtrack. 3334</P> 3335<P> 3336(*ACCEPT) in a positive assertion causes the assertion to succeed without any 3337further processing. In a negative assertion, (*ACCEPT) causes the assertion to 3338fail without any further processing. 3339</P> 3340<P> 3341The other backtracking verbs are not treated specially if they appear in a 3342positive assertion. In particular, (*THEN) skips to the next alternative in the 3343innermost enclosing group that has alternations, whether or not this is within 3344the assertion. 3345</P> 3346<P> 3347Negative assertions are, however, different, in order to ensure that changing a 3348positive assertion into a negative assertion changes its result. Backtracking 3349into (*COMMIT), (*SKIP), or (*PRUNE) causes a negative assertion to be true, 3350without considering any further alternative branches in the assertion. 3351Backtracking into (*THEN) causes it to skip to the next enclosing alternative 3352within the assertion (the normal behaviour), but if the assertion does not have 3353such an alternative, (*THEN) behaves like (*PRUNE). 3354<a name="btsub"></a></P> 3355<br><b> 3356Backtracking verbs in subroutines 3357</b><br> 3358<P> 3359These behaviours occur whether or not the subpattern is called recursively. 3360Perl's treatment of subroutines is different in some cases. 3361</P> 3362<P> 3363(*FAIL) in a subpattern called as a subroutine has its normal effect: it forces 3364an immediate backtrack. 3365</P> 3366<P> 3367(*ACCEPT) in a subpattern called as a subroutine causes the subroutine match to 3368succeed without any further processing. Matching then continues after the 3369subroutine call. 3370</P> 3371<P> 3372(*COMMIT), (*SKIP), and (*PRUNE) in a subpattern called as a subroutine cause 3373the subroutine match to fail. 3374</P> 3375<P> 3376(*THEN) skips to the next alternative in the innermost enclosing group within 3377the subpattern that has alternatives. If there is no such group within the 3378subpattern, (*THEN) causes the subroutine match to fail. 3379</P> 3380<br><a name="SEC28" href="#TOC1">SEE ALSO</a><br> 3381<P> 3382<b>pcre2api</b>(3), <b>pcre2callout</b>(3), <b>pcre2matching</b>(3), 3383<b>pcre2syntax</b>(3), <b>pcre2</b>(3). 3384</P> 3385<br><a name="SEC29" href="#TOC1">AUTHOR</a><br> 3386<P> 3387Philip Hazel 3388<br> 3389University Computing Service 3390<br> 3391Cambridge, England. 3392<br> 3393</P> 3394<br><a name="SEC30" href="#TOC1">REVISION</a><br> 3395<P> 3396Last updated: 20 June 2016 3397<br> 3398Copyright © 1997-2016 University of Cambridge. 3399<br> 3400<p> 3401Return to the <a href="index.html">PCRE2 index page</a>. 3402</p> 3403