• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Advanced googletest Topics
2
3## Introduction
4
5Now that you have read the [googletest Primer](primer.md) and learned how to
6write tests using googletest, it's time to learn some new tricks. This document
7will show you more assertions as well as how to construct complex failure
8messages, propagate fatal failures, reuse and speed up your test fixtures, and
9use various flags with your tests.
10
11## More Assertions
12
13This section covers some less frequently used, but still significant,
14assertions.
15
16### Explicit Success and Failure
17
18See [Explicit Success and Failure](reference/assertions.md#success-failure) in
19the Assertions Reference.
20
21### Exception Assertions
22
23See [Exception Assertions](reference/assertions.md#exceptions) in the Assertions
24Reference.
25
26### Predicate Assertions for Better Error Messages
27
28Even though googletest has a rich set of assertions, they can never be complete,
29as it's impossible (nor a good idea) to anticipate all scenarios a user might
30run into. Therefore, sometimes a user has to use `EXPECT_TRUE()` to check a
31complex expression, for lack of a better macro. This has the problem of not
32showing you the values of the parts of the expression, making it hard to
33understand what went wrong. As a workaround, some users choose to construct the
34failure message by themselves, streaming it into `EXPECT_TRUE()`. However, this
35is awkward especially when the expression has side-effects or is expensive to
36evaluate.
37
38googletest gives you three different options to solve this problem:
39
40#### Using an Existing Boolean Function
41
42If you already have a function or functor that returns `bool` (or a type that
43can be implicitly converted to `bool`), you can use it in a *predicate
44assertion* to get the function arguments printed for free. See
45[`EXPECT_PRED*`](reference/assertions.md#EXPECT_PRED) in the Assertions
46Reference for details.
47
48#### Using a Function That Returns an AssertionResult
49
50While `EXPECT_PRED*()` and friends are handy for a quick job, the syntax is not
51satisfactory: you have to use different macros for different arities, and it
52feels more like Lisp than C++. The `::testing::AssertionResult` class solves
53this problem.
54
55An `AssertionResult` object represents the result of an assertion (whether it's
56a success or a failure, and an associated message). You can create an
57`AssertionResult` using one of these factory functions:
58
59```c++
60namespace testing {
61
62// Returns an AssertionResult object to indicate that an assertion has
63// succeeded.
64AssertionResult AssertionSuccess();
65
66// Returns an AssertionResult object to indicate that an assertion has
67// failed.
68AssertionResult AssertionFailure();
69
70}
71```
72
73You can then use the `<<` operator to stream messages to the `AssertionResult`
74object.
75
76To provide more readable messages in Boolean assertions (e.g. `EXPECT_TRUE()`),
77write a predicate function that returns `AssertionResult` instead of `bool`. For
78example, if you define `IsEven()` as:
79
80```c++
81testing::AssertionResult IsEven(int n) {
82  if ((n % 2) == 0)
83    return testing::AssertionSuccess();
84  else
85    return testing::AssertionFailure() << n << " is odd";
86}
87```
88
89instead of:
90
91```c++
92bool IsEven(int n) {
93  return (n % 2) == 0;
94}
95```
96
97the failed assertion `EXPECT_TRUE(IsEven(Fib(4)))` will print:
98
99```none
100Value of: IsEven(Fib(4))
101  Actual: false (3 is odd)
102Expected: true
103```
104
105instead of a more opaque
106
107```none
108Value of: IsEven(Fib(4))
109  Actual: false
110Expected: true
111```
112
113If you want informative messages in `EXPECT_FALSE` and `ASSERT_FALSE` as well
114(one third of Boolean assertions in the Google code base are negative ones), and
115are fine with making the predicate slower in the success case, you can supply a
116success message:
117
118```c++
119testing::AssertionResult IsEven(int n) {
120  if ((n % 2) == 0)
121    return testing::AssertionSuccess() << n << " is even";
122  else
123    return testing::AssertionFailure() << n << " is odd";
124}
125```
126
127Then the statement `EXPECT_FALSE(IsEven(Fib(6)))` will print
128
129```none
130  Value of: IsEven(Fib(6))
131     Actual: true (8 is even)
132  Expected: false
133```
134
135#### Using a Predicate-Formatter
136
137If you find the default message generated by
138[`EXPECT_PRED*`](reference/assertions.md#EXPECT_PRED) and
139[`EXPECT_TRUE`](reference/assertions.md#EXPECT_TRUE) unsatisfactory, or some
140arguments to your predicate do not support streaming to `ostream`, you can
141instead use *predicate-formatter assertions* to *fully* customize how the
142message is formatted. See
143[`EXPECT_PRED_FORMAT*`](reference/assertions.md#EXPECT_PRED_FORMAT) in the
144Assertions Reference for details.
145
146### Floating-Point Comparison
147
148See [Floating-Point Comparison](reference/assertions.md#floating-point) in the
149Assertions Reference.
150
151#### Floating-Point Predicate-Format Functions
152
153Some floating-point operations are useful, but not that often used. In order to
154avoid an explosion of new macros, we provide them as predicate-format functions
155that can be used in the predicate assertion macro
156[`EXPECT_PRED_FORMAT2`](reference/assertions.md#EXPECT_PRED_FORMAT), for
157example:
158
159```c++
160EXPECT_PRED_FORMAT2(testing::FloatLE, val1, val2);
161EXPECT_PRED_FORMAT2(testing::DoubleLE, val1, val2);
162```
163
164The above code verifies that `val1` is less than, or approximately equal to,
165`val2`.
166
167### Asserting Using gMock Matchers
168
169See [`EXPECT_THAT`](reference/assertions.md#EXPECT_THAT) in the Assertions
170Reference.
171
172### More String Assertions
173
174(Please read the [previous](#asserting-using-gmock-matchers) section first if
175you haven't.)
176
177You can use the gMock [string matchers](reference/matchers.md#string-matchers)
178with [`EXPECT_THAT`](reference/assertions.md#EXPECT_THAT) to do more string
179comparison tricks (sub-string, prefix, suffix, regular expression, and etc). For
180example,
181
182```c++
183using ::testing::HasSubstr;
184using ::testing::MatchesRegex;
185...
186  ASSERT_THAT(foo_string, HasSubstr("needle"));
187  EXPECT_THAT(bar_string, MatchesRegex("\\w*\\d+"));
188```
189
190### Windows HRESULT assertions
191
192See [Windows HRESULT Assertions](reference/assertions.md#HRESULT) in the
193Assertions Reference.
194
195### Type Assertions
196
197You can call the function
198
199```c++
200::testing::StaticAssertTypeEq<T1, T2>();
201```
202
203to assert that types `T1` and `T2` are the same. The function does nothing if
204the assertion is satisfied. If the types are different, the function call will
205fail to compile, the compiler error message will say that
206`T1 and T2 are not the same type` and most likely (depending on the compiler)
207show you the actual values of `T1` and `T2`. This is mainly useful inside
208template code.
209
210**Caveat**: When used inside a member function of a class template or a function
211template, `StaticAssertTypeEq<T1, T2>()` is effective only if the function is
212instantiated. For example, given:
213
214```c++
215template <typename T> class Foo {
216 public:
217  void Bar() { testing::StaticAssertTypeEq<int, T>(); }
218};
219```
220
221the code:
222
223```c++
224void Test1() { Foo<bool> foo; }
225```
226
227will not generate a compiler error, as `Foo<bool>::Bar()` is never actually
228instantiated. Instead, you need:
229
230```c++
231void Test2() { Foo<bool> foo; foo.Bar(); }
232```
233
234to cause a compiler error.
235
236### Assertion Placement
237
238You can use assertions in any C++ function. In particular, it doesn't have to be
239a method of the test fixture class. The one constraint is that assertions that
240generate a fatal failure (`FAIL*` and `ASSERT_*`) can only be used in
241void-returning functions. This is a consequence of Google's not using
242exceptions. By placing it in a non-void function you'll get a confusing compile
243error like `"error: void value not ignored as it ought to be"` or `"cannot
244initialize return object of type 'bool' with an rvalue of type 'void'"` or
245`"error: no viable conversion from 'void' to 'string'"`.
246
247If you need to use fatal assertions in a function that returns non-void, one
248option is to make the function return the value in an out parameter instead. For
249example, you can rewrite `T2 Foo(T1 x)` to `void Foo(T1 x, T2* result)`. You
250need to make sure that `*result` contains some sensible value even when the
251function returns prematurely. As the function now returns `void`, you can use
252any assertion inside of it.
253
254If changing the function's type is not an option, you should just use assertions
255that generate non-fatal failures, such as `ADD_FAILURE*` and `EXPECT_*`.
256
257{: .callout .note}
258NOTE: Constructors and destructors are not considered void-returning functions,
259according to the C++ language specification, and so you may not use fatal
260assertions in them; you'll get a compilation error if you try. Instead, either
261call `abort` and crash the entire test executable, or put the fatal assertion in
262a `SetUp`/`TearDown` function; see
263[constructor/destructor vs. `SetUp`/`TearDown`](faq.md#CtorVsSetUp)
264
265{: .callout .warning}
266WARNING: A fatal assertion in a helper function (private void-returning method)
267called from a constructor or destructor does not terminate the current test, as
268your intuition might suggest: it merely returns from the constructor or
269destructor early, possibly leaving your object in a partially-constructed or
270partially-destructed state! You almost certainly want to `abort` or use
271`SetUp`/`TearDown` instead.
272
273## Skipping test execution
274
275Related to the assertions `SUCCEED()` and `FAIL()`, you can prevent further test
276execution at runtime with the `GTEST_SKIP()` macro. This is useful when you need
277to check for preconditions of the system under test during runtime and skip
278tests in a meaningful way.
279
280`GTEST_SKIP()` can be used in individual test cases or in the `SetUp()` methods
281of classes derived from either `::testing::Environment` or `::testing::Test`.
282For example:
283
284```c++
285TEST(SkipTest, DoesSkip) {
286  GTEST_SKIP() << "Skipping single test";
287  EXPECT_EQ(0, 1);  // Won't fail; it won't be executed
288}
289
290class SkipFixture : public ::testing::Test {
291 protected:
292  void SetUp() override {
293    GTEST_SKIP() << "Skipping all tests for this fixture";
294  }
295};
296
297// Tests for SkipFixture won't be executed.
298TEST_F(SkipFixture, SkipsOneTest) {
299  EXPECT_EQ(5, 7);  // Won't fail
300}
301```
302
303As with assertion macros, you can stream a custom message into `GTEST_SKIP()`.
304
305## Teaching googletest How to Print Your Values
306
307When a test assertion such as `EXPECT_EQ` fails, googletest prints the argument
308values to help you debug. It does this using a user-extensible value printer.
309
310This printer knows how to print built-in C++ types, native arrays, STL
311containers, and any type that supports the `<<` operator. For other types, it
312prints the raw bytes in the value and hopes that you the user can figure it out.
313
314As mentioned earlier, the printer is *extensible*. That means you can teach it
315to do a better job at printing your particular type than to dump the bytes. To
316do that, define `<<` for your type:
317
318```c++
319#include <ostream>
320
321namespace foo {
322
323class Bar {  // We want googletest to be able to print instances of this.
324...
325  // Create a free inline friend function.
326  friend std::ostream& operator<<(std::ostream& os, const Bar& bar) {
327    return os << bar.DebugString();  // whatever needed to print bar to os
328  }
329};
330
331// If you can't declare the function in the class it's important that the
332// << operator is defined in the SAME namespace that defines Bar.  C++'s look-up
333// rules rely on that.
334std::ostream& operator<<(std::ostream& os, const Bar& bar) {
335  return os << bar.DebugString();  // whatever needed to print bar to os
336}
337
338}  // namespace foo
339```
340
341Sometimes, this might not be an option: your team may consider it bad style to
342have a `<<` operator for `Bar`, or `Bar` may already have a `<<` operator that
343doesn't do what you want (and you cannot change it). If so, you can instead
344define a `PrintTo()` function like this:
345
346```c++
347#include <ostream>
348
349namespace foo {
350
351class Bar {
352  ...
353  friend void PrintTo(const Bar& bar, std::ostream* os) {
354    *os << bar.DebugString();  // whatever needed to print bar to os
355  }
356};
357
358// If you can't declare the function in the class it's important that PrintTo()
359// is defined in the SAME namespace that defines Bar.  C++'s look-up rules rely
360// on that.
361void PrintTo(const Bar& bar, std::ostream* os) {
362  *os << bar.DebugString();  // whatever needed to print bar to os
363}
364
365}  // namespace foo
366```
367
368If you have defined both `<<` and `PrintTo()`, the latter will be used when
369googletest is concerned. This allows you to customize how the value appears in
370googletest's output without affecting code that relies on the behavior of its
371`<<` operator.
372
373If you want to print a value `x` using googletest's value printer yourself, just
374call `::testing::PrintToString(x)`, which returns an `std::string`:
375
376```c++
377vector<pair<Bar, int> > bar_ints = GetBarIntVector();
378
379EXPECT_TRUE(IsCorrectBarIntVector(bar_ints))
380    << "bar_ints = " << testing::PrintToString(bar_ints);
381```
382
383## Death Tests
384
385In many applications, there are assertions that can cause application failure if
386a condition is not met. These sanity checks, which ensure that the program is in
387a known good state, are there to fail at the earliest possible time after some
388program state is corrupted. If the assertion checks the wrong condition, then
389the program may proceed in an erroneous state, which could lead to memory
390corruption, security holes, or worse. Hence it is vitally important to test that
391such assertion statements work as expected.
392
393Since these precondition checks cause the processes to die, we call such tests
394_death tests_. More generally, any test that checks that a program terminates
395(except by throwing an exception) in an expected fashion is also a death test.
396
397Note that if a piece of code throws an exception, we don't consider it "death"
398for the purpose of death tests, as the caller of the code could catch the
399exception and avoid the crash. If you want to verify exceptions thrown by your
400code, see [Exception Assertions](#ExceptionAssertions).
401
402If you want to test `EXPECT_*()/ASSERT_*()` failures in your test code, see
403["Catching" Failures](#catching-failures).
404
405### How to Write a Death Test
406
407GoogleTest provides assertion macros to support death tests. See
408[Death Assertions](reference/assertions.md#death) in the Assertions Reference
409for details.
410
411To write a death test, simply use one of the macros inside your test function.
412For example,
413
414```c++
415TEST(MyDeathTest, Foo) {
416  // This death test uses a compound statement.
417  ASSERT_DEATH({
418    int n = 5;
419    Foo(&n);
420  }, "Error on line .* of Foo()");
421}
422
423TEST(MyDeathTest, NormalExit) {
424  EXPECT_EXIT(NormalExit(), testing::ExitedWithCode(0), "Success");
425}
426
427TEST(MyDeathTest, KillProcess) {
428  EXPECT_EXIT(KillProcess(), testing::KilledBySignal(SIGKILL),
429              "Sending myself unblockable signal");
430}
431```
432
433verifies that:
434
435*   calling `Foo(5)` causes the process to die with the given error message,
436*   calling `NormalExit()` causes the process to print `"Success"` to stderr and
437    exit with exit code 0, and
438*   calling `KillProcess()` kills the process with signal `SIGKILL`.
439
440The test function body may contain other assertions and statements as well, if
441necessary.
442
443Note that a death test only cares about three things:
444
4451.  does `statement` abort or exit the process?
4462.  (in the case of `ASSERT_EXIT` and `EXPECT_EXIT`) does the exit status
447    satisfy `predicate`? Or (in the case of `ASSERT_DEATH` and `EXPECT_DEATH`)
448    is the exit status non-zero? And
4493.  does the stderr output match `matcher`?
450
451In particular, if `statement` generates an `ASSERT_*` or `EXPECT_*` failure, it
452will **not** cause the death test to fail, as googletest assertions don't abort
453the process.
454
455### Death Test Naming
456
457{: .callout .important}
458IMPORTANT: We strongly recommend you to follow the convention of naming your
459**test suite** (not test) `*DeathTest` when it contains a death test, as
460demonstrated in the above example. The
461[Death Tests And Threads](#death-tests-and-threads) section below explains why.
462
463If a test fixture class is shared by normal tests and death tests, you can use
464`using` or `typedef` to introduce an alias for the fixture class and avoid
465duplicating its code:
466
467```c++
468class FooTest : public testing::Test { ... };
469
470using FooDeathTest = FooTest;
471
472TEST_F(FooTest, DoesThis) {
473  // normal test
474}
475
476TEST_F(FooDeathTest, DoesThat) {
477  // death test
478}
479```
480
481### Regular Expression Syntax
482
483On POSIX systems (e.g. Linux, Cygwin, and Mac), googletest uses the
484[POSIX extended regular expression](http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html#tag_09_04)
485syntax. To learn about this syntax, you may want to read this
486[Wikipedia entry](http://en.wikipedia.org/wiki/Regular_expression#POSIX_Extended_Regular_Expressions).
487
488On Windows, googletest uses its own simple regular expression implementation. It
489lacks many features. For example, we don't support union (`"x|y"`), grouping
490(`"(xy)"`), brackets (`"[xy]"`), and repetition count (`"x{5,7}"`), among
491others. Below is what we do support (`A` denotes a literal character, period
492(`.`), or a single `\\ ` escape sequence; `x` and `y` denote regular
493expressions.):
494
495Expression | Meaning
496---------- | --------------------------------------------------------------
497`c`        | matches any literal character `c`
498`\\d`      | matches any decimal digit
499`\\D`      | matches any character that's not a decimal digit
500`\\f`      | matches `\f`
501`\\n`      | matches `\n`
502`\\r`      | matches `\r`
503`\\s`      | matches any ASCII whitespace, including `\n`
504`\\S`      | matches any character that's not a whitespace
505`\\t`      | matches `\t`
506`\\v`      | matches `\v`
507`\\w`      | matches any letter, `_`, or decimal digit
508`\\W`      | matches any character that `\\w` doesn't match
509`\\c`      | matches any literal character `c`, which must be a punctuation
510`.`        | matches any single character except `\n`
511`A?`       | matches 0 or 1 occurrences of `A`
512`A*`       | matches 0 or many occurrences of `A`
513`A+`       | matches 1 or many occurrences of `A`
514`^`        | matches the beginning of a string (not that of each line)
515`$`        | matches the end of a string (not that of each line)
516`xy`       | matches `x` followed by `y`
517
518To help you determine which capability is available on your system, googletest
519defines macros to govern which regular expression it is using. The macros are:
520`GTEST_USES_SIMPLE_RE=1` or `GTEST_USES_POSIX_RE=1`. If you want your death
521tests to work in all cases, you can either `#if` on these macros or use the more
522limited syntax only.
523
524### How It Works
525
526See [Death Assertions](reference/assertions.md#death) in the Assertions
527Reference.
528
529### Death Tests And Threads
530
531The reason for the two death test styles has to do with thread safety. Due to
532well-known problems with forking in the presence of threads, death tests should
533be run in a single-threaded context. Sometimes, however, it isn't feasible to
534arrange that kind of environment. For example, statically-initialized modules
535may start threads before main is ever reached. Once threads have been created,
536it may be difficult or impossible to clean them up.
537
538googletest has three features intended to raise awareness of threading issues.
539
5401.  A warning is emitted if multiple threads are running when a death test is
541    encountered.
5422.  Test suites with a name ending in "DeathTest" are run before all other
543    tests.
5443.  It uses `clone()` instead of `fork()` to spawn the child process on Linux
545    (`clone()` is not available on Cygwin and Mac), as `fork()` is more likely
546    to cause the child to hang when the parent process has multiple threads.
547
548It's perfectly fine to create threads inside a death test statement; they are
549executed in a separate process and cannot affect the parent.
550
551### Death Test Styles
552
553The "threadsafe" death test style was introduced in order to help mitigate the
554risks of testing in a possibly multithreaded environment. It trades increased
555test execution time (potentially dramatically so) for improved thread safety.
556
557The automated testing framework does not set the style flag. You can choose a
558particular style of death tests by setting the flag programmatically:
559
560```c++
561testing::FLAGS_gtest_death_test_style="threadsafe"
562```
563
564You can do this in `main()` to set the style for all death tests in the binary,
565or in individual tests. Recall that flags are saved before running each test and
566restored afterwards, so you need not do that yourself. For example:
567
568```c++
569int main(int argc, char** argv) {
570  testing::InitGoogleTest(&argc, argv);
571  testing::FLAGS_gtest_death_test_style = "fast";
572  return RUN_ALL_TESTS();
573}
574
575TEST(MyDeathTest, TestOne) {
576  testing::FLAGS_gtest_death_test_style = "threadsafe";
577  // This test is run in the "threadsafe" style:
578  ASSERT_DEATH(ThisShouldDie(), "");
579}
580
581TEST(MyDeathTest, TestTwo) {
582  // This test is run in the "fast" style:
583  ASSERT_DEATH(ThisShouldDie(), "");
584}
585```
586
587### Caveats
588
589The `statement` argument of `ASSERT_EXIT()` can be any valid C++ statement. If
590it leaves the current function via a `return` statement or by throwing an
591exception, the death test is considered to have failed. Some googletest macros
592may return from the current function (e.g. `ASSERT_TRUE()`), so be sure to avoid
593them in `statement`.
594
595Since `statement` runs in the child process, any in-memory side effect (e.g.
596modifying a variable, releasing memory, etc) it causes will *not* be observable
597in the parent process. In particular, if you release memory in a death test,
598your program will fail the heap check as the parent process will never see the
599memory reclaimed. To solve this problem, you can
600
6011.  try not to free memory in a death test;
6022.  free the memory again in the parent process; or
6033.  do not use the heap checker in your program.
604
605Due to an implementation detail, you cannot place multiple death test assertions
606on the same line; otherwise, compilation will fail with an unobvious error
607message.
608
609Despite the improved thread safety afforded by the "threadsafe" style of death
610test, thread problems such as deadlock are still possible in the presence of
611handlers registered with `pthread_atfork(3)`.
612
613
614## Using Assertions in Sub-routines
615
616{: .callout .note}
617Note: If you want to put a series of test assertions in a subroutine to check
618for a complex condition, consider using
619[a custom GMock matcher](gmock_cook_book.md#NewMatchers)
620instead. This lets you provide a more readable error message in case of failure
621and avoid all of the issues described below.
622
623### Adding Traces to Assertions
624
625If a test sub-routine is called from several places, when an assertion inside it
626fails, it can be hard to tell which invocation of the sub-routine the failure is
627from. You can alleviate this problem using extra logging or custom failure
628messages, but that usually clutters up your tests. A better solution is to use
629the `SCOPED_TRACE` macro or the `ScopedTrace` utility:
630
631```c++
632SCOPED_TRACE(message);
633```
634```c++
635ScopedTrace trace("file_path", line_number, message);
636```
637
638where `message` can be anything streamable to `std::ostream`. `SCOPED_TRACE`
639macro will cause the current file name, line number, and the given message to be
640added in every failure message. `ScopedTrace` accepts explicit file name and
641line number in arguments, which is useful for writing test helpers. The effect
642will be undone when the control leaves the current lexical scope.
643
644For example,
645
646```c++
64710: void Sub1(int n) {
64811:   EXPECT_EQ(Bar(n), 1);
64912:   EXPECT_EQ(Bar(n + 1), 2);
65013: }
65114:
65215: TEST(FooTest, Bar) {
65316:   {
65417:     SCOPED_TRACE("A");  // This trace point will be included in
65518:                         // every failure in this scope.
65619:     Sub1(1);
65720:   }
65821:   // Now it won't.
65922:   Sub1(9);
66023: }
661```
662
663could result in messages like these:
664
665```none
666path/to/foo_test.cc:11: Failure
667Value of: Bar(n)
668Expected: 1
669  Actual: 2
670Google Test trace:
671path/to/foo_test.cc:17: A
672
673path/to/foo_test.cc:12: Failure
674Value of: Bar(n + 1)
675Expected: 2
676  Actual: 3
677```
678
679Without the trace, it would've been difficult to know which invocation of
680`Sub1()` the two failures come from respectively. (You could add an extra
681message to each assertion in `Sub1()` to indicate the value of `n`, but that's
682tedious.)
683
684Some tips on using `SCOPED_TRACE`:
685
6861.  With a suitable message, it's often enough to use `SCOPED_TRACE` at the
687    beginning of a sub-routine, instead of at each call site.
6882.  When calling sub-routines inside a loop, make the loop iterator part of the
689    message in `SCOPED_TRACE` such that you can know which iteration the failure
690    is from.
6913.  Sometimes the line number of the trace point is enough for identifying the
692    particular invocation of a sub-routine. In this case, you don't have to
693    choose a unique message for `SCOPED_TRACE`. You can simply use `""`.
6944.  You can use `SCOPED_TRACE` in an inner scope when there is one in the outer
695    scope. In this case, all active trace points will be included in the failure
696    messages, in reverse order they are encountered.
6975.  The trace dump is clickable in Emacs - hit `return` on a line number and
698    you'll be taken to that line in the source file!
699
700### Propagating Fatal Failures
701
702A common pitfall when using `ASSERT_*` and `FAIL*` is not understanding that
703when they fail they only abort the _current function_, not the entire test. For
704example, the following test will segfault:
705
706```c++
707void Subroutine() {
708  // Generates a fatal failure and aborts the current function.
709  ASSERT_EQ(1, 2);
710
711  // The following won't be executed.
712  ...
713}
714
715TEST(FooTest, Bar) {
716  Subroutine();  // The intended behavior is for the fatal failure
717                 // in Subroutine() to abort the entire test.
718
719  // The actual behavior: the function goes on after Subroutine() returns.
720  int* p = nullptr;
721  *p = 3;  // Segfault!
722}
723```
724
725To alleviate this, googletest provides three different solutions. You could use
726either exceptions, the `(ASSERT|EXPECT)_NO_FATAL_FAILURE` assertions or the
727`HasFatalFailure()` function. They are described in the following two
728subsections.
729
730#### Asserting on Subroutines with an exception
731
732The following code can turn ASSERT-failure into an exception:
733
734```c++
735class ThrowListener : public testing::EmptyTestEventListener {
736  void OnTestPartResult(const testing::TestPartResult& result) override {
737    if (result.type() == testing::TestPartResult::kFatalFailure) {
738      throw testing::AssertionException(result);
739    }
740  }
741};
742int main(int argc, char** argv) {
743  ...
744  testing::UnitTest::GetInstance()->listeners().Append(new ThrowListener);
745  return RUN_ALL_TESTS();
746}
747```
748
749This listener should be added after other listeners if you have any, otherwise
750they won't see failed `OnTestPartResult`.
751
752#### Asserting on Subroutines
753
754As shown above, if your test calls a subroutine that has an `ASSERT_*` failure
755in it, the test will continue after the subroutine returns. This may not be what
756you want.
757
758Often people want fatal failures to propagate like exceptions. For that
759googletest offers the following macros:
760
761Fatal assertion                       | Nonfatal assertion                    | Verifies
762------------------------------------- | ------------------------------------- | --------
763`ASSERT_NO_FATAL_FAILURE(statement);` | `EXPECT_NO_FATAL_FAILURE(statement);` | `statement` doesn't generate any new fatal failures in the current thread.
764
765Only failures in the thread that executes the assertion are checked to determine
766the result of this type of assertions. If `statement` creates new threads,
767failures in these threads are ignored.
768
769Examples:
770
771```c++
772ASSERT_NO_FATAL_FAILURE(Foo());
773
774int i;
775EXPECT_NO_FATAL_FAILURE({
776  i = Bar();
777});
778```
779
780Assertions from multiple threads are currently not supported on Windows.
781
782#### Checking for Failures in the Current Test
783
784`HasFatalFailure()` in the `::testing::Test` class returns `true` if an
785assertion in the current test has suffered a fatal failure. This allows
786functions to catch fatal failures in a sub-routine and return early.
787
788```c++
789class Test {
790 public:
791  ...
792  static bool HasFatalFailure();
793};
794```
795
796The typical usage, which basically simulates the behavior of a thrown exception,
797is:
798
799```c++
800TEST(FooTest, Bar) {
801  Subroutine();
802  // Aborts if Subroutine() had a fatal failure.
803  if (HasFatalFailure()) return;
804
805  // The following won't be executed.
806  ...
807}
808```
809
810If `HasFatalFailure()` is used outside of `TEST()` , `TEST_F()` , or a test
811fixture, you must add the `::testing::Test::` prefix, as in:
812
813```c++
814if (testing::Test::HasFatalFailure()) return;
815```
816
817Similarly, `HasNonfatalFailure()` returns `true` if the current test has at
818least one non-fatal failure, and `HasFailure()` returns `true` if the current
819test has at least one failure of either kind.
820
821## Logging Additional Information
822
823In your test code, you can call `RecordProperty("key", value)` to log additional
824information, where `value` can be either a string or an `int`. The *last* value
825recorded for a key will be emitted to the
826[XML output](#generating-an-xml-report) if you specify one. For example, the
827test
828
829```c++
830TEST_F(WidgetUsageTest, MinAndMaxWidgets) {
831  RecordProperty("MaximumWidgets", ComputeMaxUsage());
832  RecordProperty("MinimumWidgets", ComputeMinUsage());
833}
834```
835
836will output XML like this:
837
838```xml
839  ...
840    <testcase name="MinAndMaxWidgets" status="run" time="0.006" classname="WidgetUsageTest" MaximumWidgets="12" MinimumWidgets="9" />
841  ...
842```
843
844{: .callout .note}
845> NOTE:
846>
847> *   `RecordProperty()` is a static member of the `Test` class. Therefore it
848>     needs to be prefixed with `::testing::Test::` if used outside of the
849>     `TEST` body and the test fixture class.
850> *   *`key`* must be a valid XML attribute name, and cannot conflict with the
851>     ones already used by googletest (`name`, `status`, `time`, `classname`,
852>     `type_param`, and `value_param`).
853> *   Calling `RecordProperty()` outside of the lifespan of a test is allowed.
854>     If it's called outside of a test but between a test suite's
855>     `SetUpTestSuite()` and `TearDownTestSuite()` methods, it will be
856>     attributed to the XML element for the test suite. If it's called outside
857>     of all test suites (e.g. in a test environment), it will be attributed to
858>     the top-level XML element.
859
860## Sharing Resources Between Tests in the Same Test Suite
861
862googletest creates a new test fixture object for each test in order to make
863tests independent and easier to debug. However, sometimes tests use resources
864that are expensive to set up, making the one-copy-per-test model prohibitively
865expensive.
866
867If the tests don't change the resource, there's no harm in their sharing a
868single resource copy. So, in addition to per-test set-up/tear-down, googletest
869also supports per-test-suite set-up/tear-down. To use it:
870
8711.  In your test fixture class (say `FooTest` ), declare as `static` some member
872    variables to hold the shared resources.
8732.  Outside your test fixture class (typically just below it), define those
874    member variables, optionally giving them initial values.
8753.  In the same test fixture class, define a `static void SetUpTestSuite()`
876    function (remember not to spell it as **`SetupTestSuite`** with a small
877    `u`!) to set up the shared resources and a `static void TearDownTestSuite()`
878    function to tear them down.
879
880That's it! googletest automatically calls `SetUpTestSuite()` before running the
881*first test* in the `FooTest` test suite (i.e. before creating the first
882`FooTest` object), and calls `TearDownTestSuite()` after running the *last test*
883in it (i.e. after deleting the last `FooTest` object). In between, the tests can
884use the shared resources.
885
886Remember that the test order is undefined, so your code can't depend on a test
887preceding or following another. Also, the tests must either not modify the state
888of any shared resource, or, if they do modify the state, they must restore the
889state to its original value before passing control to the next test.
890
891Here's an example of per-test-suite set-up and tear-down:
892
893```c++
894class FooTest : public testing::Test {
895 protected:
896  // Per-test-suite set-up.
897  // Called before the first test in this test suite.
898  // Can be omitted if not needed.
899  static void SetUpTestSuite() {
900    shared_resource_ = new ...;
901  }
902
903  // Per-test-suite tear-down.
904  // Called after the last test in this test suite.
905  // Can be omitted if not needed.
906  static void TearDownTestSuite() {
907    delete shared_resource_;
908    shared_resource_ = nullptr;
909  }
910
911  // You can define per-test set-up logic as usual.
912  void SetUp() override { ... }
913
914  // You can define per-test tear-down logic as usual.
915  void TearDown() override { ... }
916
917  // Some expensive resource shared by all tests.
918  static T* shared_resource_;
919};
920
921T* FooTest::shared_resource_ = nullptr;
922
923TEST_F(FooTest, Test1) {
924  ... you can refer to shared_resource_ here ...
925}
926
927TEST_F(FooTest, Test2) {
928  ... you can refer to shared_resource_ here ...
929}
930```
931
932{: .callout .note}
933NOTE: Though the above code declares `SetUpTestSuite()` protected, it may
934sometimes be necessary to declare it public, such as when using it with
935`TEST_P`.
936
937## Global Set-Up and Tear-Down
938
939Just as you can do set-up and tear-down at the test level and the test suite
940level, you can also do it at the test program level. Here's how.
941
942First, you subclass the `::testing::Environment` class to define a test
943environment, which knows how to set-up and tear-down:
944
945```c++
946class Environment : public ::testing::Environment {
947 public:
948  ~Environment() override {}
949
950  // Override this to define how to set up the environment.
951  void SetUp() override {}
952
953  // Override this to define how to tear down the environment.
954  void TearDown() override {}
955};
956```
957
958Then, you register an instance of your environment class with googletest by
959calling the `::testing::AddGlobalTestEnvironment()` function:
960
961```c++
962Environment* AddGlobalTestEnvironment(Environment* env);
963```
964
965Now, when `RUN_ALL_TESTS()` is called, it first calls the `SetUp()` method of
966each environment object, then runs the tests if none of the environments
967reported fatal failures and `GTEST_SKIP()` was not called. `RUN_ALL_TESTS()`
968always calls `TearDown()` with each environment object, regardless of whether or
969not the tests were run.
970
971It's OK to register multiple environment objects. In this suite, their `SetUp()`
972will be called in the order they are registered, and their `TearDown()` will be
973called in the reverse order.
974
975Note that googletest takes ownership of the registered environment objects.
976Therefore **do not delete them** by yourself.
977
978You should call `AddGlobalTestEnvironment()` before `RUN_ALL_TESTS()` is called,
979probably in `main()`. If you use `gtest_main`, you need to call this before
980`main()` starts for it to take effect. One way to do this is to define a global
981variable like this:
982
983```c++
984testing::Environment* const foo_env =
985    testing::AddGlobalTestEnvironment(new FooEnvironment);
986```
987
988However, we strongly recommend you to write your own `main()` and call
989`AddGlobalTestEnvironment()` there, as relying on initialization of global
990variables makes the code harder to read and may cause problems when you register
991multiple environments from different translation units and the environments have
992dependencies among them (remember that the compiler doesn't guarantee the order
993in which global variables from different translation units are initialized).
994
995## Value-Parameterized Tests
996
997*Value-parameterized tests* allow you to test your code with different
998parameters without writing multiple copies of the same test. This is useful in a
999number of situations, for example:
1000
1001*   You have a piece of code whose behavior is affected by one or more
1002    command-line flags. You want to make sure your code performs correctly for
1003    various values of those flags.
1004*   You want to test different implementations of an OO interface.
1005*   You want to test your code over various inputs (a.k.a. data-driven testing).
1006    This feature is easy to abuse, so please exercise your good sense when doing
1007    it!
1008
1009### How to Write Value-Parameterized Tests
1010
1011To write value-parameterized tests, first you should define a fixture class. It
1012must be derived from both `testing::Test` and `testing::WithParamInterface<T>`
1013(the latter is a pure interface), where `T` is the type of your parameter
1014values. For convenience, you can just derive the fixture class from
1015`testing::TestWithParam<T>`, which itself is derived from both `testing::Test`
1016and `testing::WithParamInterface<T>`. `T` can be any copyable type. If it's a
1017raw pointer, you are responsible for managing the lifespan of the pointed
1018values.
1019
1020{: .callout .note}
1021NOTE: If your test fixture defines `SetUpTestSuite()` or `TearDownTestSuite()`
1022they must be declared **public** rather than **protected** in order to use
1023`TEST_P`.
1024
1025```c++
1026class FooTest :
1027    public testing::TestWithParam<const char*> {
1028  // You can implement all the usual fixture class members here.
1029  // To access the test parameter, call GetParam() from class
1030  // TestWithParam<T>.
1031};
1032
1033// Or, when you want to add parameters to a pre-existing fixture class:
1034class BaseTest : public testing::Test {
1035  ...
1036};
1037class BarTest : public BaseTest,
1038                public testing::WithParamInterface<const char*> {
1039  ...
1040};
1041```
1042
1043Then, use the `TEST_P` macro to define as many test patterns using this fixture
1044as you want. The `_P` suffix is for "parameterized" or "pattern", whichever you
1045prefer to think.
1046
1047```c++
1048TEST_P(FooTest, DoesBlah) {
1049  // Inside a test, access the test parameter with the GetParam() method
1050  // of the TestWithParam<T> class:
1051  EXPECT_TRUE(foo.Blah(GetParam()));
1052  ...
1053}
1054
1055TEST_P(FooTest, HasBlahBlah) {
1056  ...
1057}
1058```
1059
1060Finally, you can use the `INSTANTIATE_TEST_SUITE_P` macro to instantiate the
1061test suite with any set of parameters you want. GoogleTest defines a number of
1062functions for generating test parameters—see details at
1063[`INSTANTIATE_TEST_SUITE_P`](reference/testing.md#INSTANTIATE_TEST_SUITE_P) in
1064the Testing Reference.
1065
1066For example, the following statement will instantiate tests from the `FooTest`
1067test suite each with parameter values `"meeny"`, `"miny"`, and `"moe"` using the
1068[`Values`](reference/testing.md#param-generators) parameter generator:
1069
1070```c++
1071INSTANTIATE_TEST_SUITE_P(MeenyMinyMoe,
1072                         FooTest,
1073                         testing::Values("meeny", "miny", "moe"));
1074```
1075
1076{: .callout .note}
1077NOTE: The code above must be placed at global or namespace scope, not at
1078function scope.
1079
1080The first argument to `INSTANTIATE_TEST_SUITE_P` is a unique name for the
1081instantiation of the test suite. The next argument is the name of the test
1082pattern, and the last is the
1083[parameter generator](reference/testing.md#param-generators).
1084
1085You can instantiate a test pattern more than once, so to distinguish different
1086instances of the pattern, the instantiation name is added as a prefix to the
1087actual test suite name. Remember to pick unique prefixes for different
1088instantiations. The tests from the instantiation above will have these names:
1089
1090*   `MeenyMinyMoe/FooTest.DoesBlah/0` for `"meeny"`
1091*   `MeenyMinyMoe/FooTest.DoesBlah/1` for `"miny"`
1092*   `MeenyMinyMoe/FooTest.DoesBlah/2` for `"moe"`
1093*   `MeenyMinyMoe/FooTest.HasBlahBlah/0` for `"meeny"`
1094*   `MeenyMinyMoe/FooTest.HasBlahBlah/1` for `"miny"`
1095*   `MeenyMinyMoe/FooTest.HasBlahBlah/2` for `"moe"`
1096
1097You can use these names in [`--gtest_filter`](#running-a-subset-of-the-tests).
1098
1099The following statement will instantiate all tests from `FooTest` again, each
1100with parameter values `"cat"` and `"dog"` using the
1101[`ValuesIn`](reference/testing.md#param-generators) parameter generator:
1102
1103```c++
1104const char* pets[] = {"cat", "dog"};
1105INSTANTIATE_TEST_SUITE_P(Pets, FooTest, testing::ValuesIn(pets));
1106```
1107
1108The tests from the instantiation above will have these names:
1109
1110*   `Pets/FooTest.DoesBlah/0` for `"cat"`
1111*   `Pets/FooTest.DoesBlah/1` for `"dog"`
1112*   `Pets/FooTest.HasBlahBlah/0` for `"cat"`
1113*   `Pets/FooTest.HasBlahBlah/1` for `"dog"`
1114
1115Please note that `INSTANTIATE_TEST_SUITE_P` will instantiate *all* tests in the
1116given test suite, whether their definitions come before or *after* the
1117`INSTANTIATE_TEST_SUITE_P` statement.
1118
1119Additionally, by default, every `TEST_P` without a corresponding
1120`INSTANTIATE_TEST_SUITE_P` causes a failing test in test suite
1121`GoogleTestVerification`. If you have a test suite where that omission is not an
1122error, for example it is in a library that may be linked in for other reasons or
1123where the list of test cases is dynamic and may be empty, then this check can be
1124suppressed by tagging the test suite:
1125
1126```c++
1127GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(FooTest);
1128```
1129
1130You can see [sample7_unittest.cc] and [sample8_unittest.cc] for more examples.
1131
1132[sample7_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample7_unittest.cc "Parameterized Test example"
1133[sample8_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample8_unittest.cc "Parameterized Test example with multiple parameters"
1134
1135### Creating Value-Parameterized Abstract Tests
1136
1137In the above, we define and instantiate `FooTest` in the *same* source file.
1138Sometimes you may want to define value-parameterized tests in a library and let
1139other people instantiate them later. This pattern is known as *abstract tests*.
1140As an example of its application, when you are designing an interface you can
1141write a standard suite of abstract tests (perhaps using a factory function as
1142the test parameter) that all implementations of the interface are expected to
1143pass. When someone implements the interface, they can instantiate your suite to
1144get all the interface-conformance tests for free.
1145
1146To define abstract tests, you should organize your code like this:
1147
11481.  Put the definition of the parameterized test fixture class (e.g. `FooTest`)
1149    in a header file, say `foo_param_test.h`. Think of this as *declaring* your
1150    abstract tests.
11512.  Put the `TEST_P` definitions in `foo_param_test.cc`, which includes
1152    `foo_param_test.h`. Think of this as *implementing* your abstract tests.
1153
1154Once they are defined, you can instantiate them by including `foo_param_test.h`,
1155invoking `INSTANTIATE_TEST_SUITE_P()`, and depending on the library target that
1156contains `foo_param_test.cc`. You can instantiate the same abstract test suite
1157multiple times, possibly in different source files.
1158
1159### Specifying Names for Value-Parameterized Test Parameters
1160
1161The optional last argument to `INSTANTIATE_TEST_SUITE_P()` allows the user to
1162specify a function or functor that generates custom test name suffixes based on
1163the test parameters. The function should accept one argument of type
1164`testing::TestParamInfo<class ParamType>`, and return `std::string`.
1165
1166`testing::PrintToStringParamName` is a builtin test suffix generator that
1167returns the value of `testing::PrintToString(GetParam())`. It does not work for
1168`std::string` or C strings.
1169
1170{: .callout .note}
1171NOTE: test names must be non-empty, unique, and may only contain ASCII
1172alphanumeric characters. In particular, they
1173[should not contain underscores](faq.md#why-should-test-suite-names-and-test-names-not-contain-underscore)
1174
1175```c++
1176class MyTestSuite : public testing::TestWithParam<int> {};
1177
1178TEST_P(MyTestSuite, MyTest)
1179{
1180  std::cout << "Example Test Param: " << GetParam() << std::endl;
1181}
1182
1183INSTANTIATE_TEST_SUITE_P(MyGroup, MyTestSuite, testing::Range(0, 10),
1184                         testing::PrintToStringParamName());
1185```
1186
1187Providing a custom functor allows for more control over test parameter name
1188generation, especially for types where the automatic conversion does not
1189generate helpful parameter names (e.g. strings as demonstrated above). The
1190following example illustrates this for multiple parameters, an enumeration type
1191and a string, and also demonstrates how to combine generators. It uses a lambda
1192for conciseness:
1193
1194```c++
1195enum class MyType { MY_FOO = 0, MY_BAR = 1 };
1196
1197class MyTestSuite : public testing::TestWithParam<std::tuple<MyType, std::string>> {
1198};
1199
1200INSTANTIATE_TEST_SUITE_P(
1201    MyGroup, MyTestSuite,
1202    testing::Combine(
1203        testing::Values(MyType::MY_FOO, MyType::MY_BAR),
1204        testing::Values("A", "B")),
1205    [](const testing::TestParamInfo<MyTestSuite::ParamType>& info) {
1206      std::string name = absl::StrCat(
1207          std::get<0>(info.param) == MyType::MY_FOO ? "Foo" : "Bar",
1208          std::get<1>(info.param));
1209      absl::c_replace_if(name, [](char c) { return !std::isalnum(c); }, '_');
1210      return name;
1211    });
1212```
1213
1214## Typed Tests
1215
1216Suppose you have multiple implementations of the same interface and want to make
1217sure that all of them satisfy some common requirements. Or, you may have defined
1218several types that are supposed to conform to the same "concept" and you want to
1219verify it. In both cases, you want the same test logic repeated for different
1220types.
1221
1222While you can write one `TEST` or `TEST_F` for each type you want to test (and
1223you may even factor the test logic into a function template that you invoke from
1224the `TEST`), it's tedious and doesn't scale: if you want `m` tests over `n`
1225types, you'll end up writing `m*n` `TEST`s.
1226
1227*Typed tests* allow you to repeat the same test logic over a list of types. You
1228only need to write the test logic once, although you must know the type list
1229when writing typed tests. Here's how you do it:
1230
1231First, define a fixture class template. It should be parameterized by a type.
1232Remember to derive it from `::testing::Test`:
1233
1234```c++
1235template <typename T>
1236class FooTest : public testing::Test {
1237 public:
1238  ...
1239  using List = std::list<T>;
1240  static T shared_;
1241  T value_;
1242};
1243```
1244
1245Next, associate a list of types with the test suite, which will be repeated for
1246each type in the list:
1247
1248```c++
1249using MyTypes = ::testing::Types<char, int, unsigned int>;
1250TYPED_TEST_SUITE(FooTest, MyTypes);
1251```
1252
1253The type alias (`using` or `typedef`) is necessary for the `TYPED_TEST_SUITE`
1254macro to parse correctly. Otherwise the compiler will think that each comma in
1255the type list introduces a new macro argument.
1256
1257Then, use `TYPED_TEST()` instead of `TEST_F()` to define a typed test for this
1258test suite. You can repeat this as many times as you want:
1259
1260```c++
1261TYPED_TEST(FooTest, DoesBlah) {
1262  // Inside a test, refer to the special name TypeParam to get the type
1263  // parameter.  Since we are inside a derived class template, C++ requires
1264  // us to visit the members of FooTest via 'this'.
1265  TypeParam n = this->value_;
1266
1267  // To visit static members of the fixture, add the 'TestFixture::'
1268  // prefix.
1269  n += TestFixture::shared_;
1270
1271  // To refer to typedefs in the fixture, add the 'typename TestFixture::'
1272  // prefix.  The 'typename' is required to satisfy the compiler.
1273  typename TestFixture::List values;
1274
1275  values.push_back(n);
1276  ...
1277}
1278
1279TYPED_TEST(FooTest, HasPropertyA) { ... }
1280```
1281
1282You can see [sample6_unittest.cc] for a complete example.
1283
1284[sample6_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample6_unittest.cc "Typed Test example"
1285
1286## Type-Parameterized Tests
1287
1288*Type-parameterized tests* are like typed tests, except that they don't require
1289you to know the list of types ahead of time. Instead, you can define the test
1290logic first and instantiate it with different type lists later. You can even
1291instantiate it more than once in the same program.
1292
1293If you are designing an interface or concept, you can define a suite of
1294type-parameterized tests to verify properties that any valid implementation of
1295the interface/concept should have. Then, the author of each implementation can
1296just instantiate the test suite with their type to verify that it conforms to
1297the requirements, without having to write similar tests repeatedly. Here's an
1298example:
1299
1300First, define a fixture class template, as we did with typed tests:
1301
1302```c++
1303template <typename T>
1304class FooTest : public testing::Test {
1305  ...
1306};
1307```
1308
1309Next, declare that you will define a type-parameterized test suite:
1310
1311```c++
1312TYPED_TEST_SUITE_P(FooTest);
1313```
1314
1315Then, use `TYPED_TEST_P()` to define a type-parameterized test. You can repeat
1316this as many times as you want:
1317
1318```c++
1319TYPED_TEST_P(FooTest, DoesBlah) {
1320  // Inside a test, refer to TypeParam to get the type parameter.
1321  TypeParam n = 0;
1322  ...
1323}
1324
1325TYPED_TEST_P(FooTest, HasPropertyA) { ... }
1326```
1327
1328Now the tricky part: you need to register all test patterns using the
1329`REGISTER_TYPED_TEST_SUITE_P` macro before you can instantiate them. The first
1330argument of the macro is the test suite name; the rest are the names of the
1331tests in this test suite:
1332
1333```c++
1334REGISTER_TYPED_TEST_SUITE_P(FooTest,
1335                            DoesBlah, HasPropertyA);
1336```
1337
1338Finally, you are free to instantiate the pattern with the types you want. If you
1339put the above code in a header file, you can `#include` it in multiple C++
1340source files and instantiate it multiple times.
1341
1342```c++
1343using MyTypes = ::testing::Types<char, int, unsigned int>;
1344INSTANTIATE_TYPED_TEST_SUITE_P(My, FooTest, MyTypes);
1345```
1346
1347To distinguish different instances of the pattern, the first argument to the
1348`INSTANTIATE_TYPED_TEST_SUITE_P` macro is a prefix that will be added to the
1349actual test suite name. Remember to pick unique prefixes for different
1350instances.
1351
1352In the special case where the type list contains only one type, you can write
1353that type directly without `::testing::Types<...>`, like this:
1354
1355```c++
1356INSTANTIATE_TYPED_TEST_SUITE_P(My, FooTest, int);
1357```
1358
1359You can see [sample6_unittest.cc] for a complete example.
1360
1361## Testing Private Code
1362
1363If you change your software's internal implementation, your tests should not
1364break as long as the change is not observable by users. Therefore, **per the
1365black-box testing principle, most of the time you should test your code through
1366its public interfaces.**
1367
1368**If you still find yourself needing to test internal implementation code,
1369consider if there's a better design.** The desire to test internal
1370implementation is often a sign that the class is doing too much. Consider
1371extracting an implementation class, and testing it. Then use that implementation
1372class in the original class.
1373
1374If you absolutely have to test non-public interface code though, you can. There
1375are two cases to consider:
1376
1377*   Static functions ( *not* the same as static member functions!) or unnamed
1378    namespaces, and
1379*   Private or protected class members
1380
1381To test them, we use the following special techniques:
1382
1383*   Both static functions and definitions/declarations in an unnamed namespace
1384    are only visible within the same translation unit. To test them, you can
1385    `#include` the entire `.cc` file being tested in your `*_test.cc` file.
1386    (#including `.cc` files is not a good way to reuse code - you should not do
1387    this in production code!)
1388
1389    However, a better approach is to move the private code into the
1390    `foo::internal` namespace, where `foo` is the namespace your project
1391    normally uses, and put the private declarations in a `*-internal.h` file.
1392    Your production `.cc` files and your tests are allowed to include this
1393    internal header, but your clients are not. This way, you can fully test your
1394    internal implementation without leaking it to your clients.
1395
1396*   Private class members are only accessible from within the class or by
1397    friends. To access a class' private members, you can declare your test
1398    fixture as a friend to the class and define accessors in your fixture. Tests
1399    using the fixture can then access the private members of your production
1400    class via the accessors in the fixture. Note that even though your fixture
1401    is a friend to your production class, your tests are not automatically
1402    friends to it, as they are technically defined in sub-classes of the
1403    fixture.
1404
1405    Another way to test private members is to refactor them into an
1406    implementation class, which is then declared in a `*-internal.h` file. Your
1407    clients aren't allowed to include this header but your tests can. Such is
1408    called the
1409    [Pimpl](https://www.gamedev.net/articles/programming/general-and-gameplay-programming/the-c-pimpl-r1794/)
1410    (Private Implementation) idiom.
1411
1412    Or, you can declare an individual test as a friend of your class by adding
1413    this line in the class body:
1414
1415    ```c++
1416        FRIEND_TEST(TestSuiteName, TestName);
1417    ```
1418
1419    For example,
1420
1421    ```c++
1422    // foo.h
1423    class Foo {
1424      ...
1425     private:
1426      FRIEND_TEST(FooTest, BarReturnsZeroOnNull);
1427
1428      int Bar(void* x);
1429    };
1430
1431    // foo_test.cc
1432    ...
1433    TEST(FooTest, BarReturnsZeroOnNull) {
1434      Foo foo;
1435      EXPECT_EQ(foo.Bar(NULL), 0);  // Uses Foo's private member Bar().
1436    }
1437    ```
1438
1439    Pay special attention when your class is defined in a namespace. If you want
1440    your test fixtures and tests to be friends of your class, then they must be
1441    defined in the exact same namespace (no anonymous or inline namespaces).
1442
1443    For example, if the code to be tested looks like:
1444
1445    ```c++
1446    namespace my_namespace {
1447
1448    class Foo {
1449      friend class FooTest;
1450      FRIEND_TEST(FooTest, Bar);
1451      FRIEND_TEST(FooTest, Baz);
1452      ... definition of the class Foo ...
1453    };
1454
1455    }  // namespace my_namespace
1456    ```
1457
1458    Your test code should be something like:
1459
1460    ```c++
1461    namespace my_namespace {
1462
1463    class FooTest : public testing::Test {
1464     protected:
1465      ...
1466    };
1467
1468    TEST_F(FooTest, Bar) { ... }
1469    TEST_F(FooTest, Baz) { ... }
1470
1471    }  // namespace my_namespace
1472    ```
1473
1474## "Catching" Failures
1475
1476If you are building a testing utility on top of googletest, you'll want to test
1477your utility. What framework would you use to test it? googletest, of course.
1478
1479The challenge is to verify that your testing utility reports failures correctly.
1480In frameworks that report a failure by throwing an exception, you could catch
1481the exception and assert on it. But googletest doesn't use exceptions, so how do
1482we test that a piece of code generates an expected failure?
1483
1484`"gtest/gtest-spi.h"` contains some constructs to do this. After #including this header,
1485you can use
1486
1487```c++
1488  EXPECT_FATAL_FAILURE(statement, substring);
1489```
1490
1491to assert that `statement` generates a fatal (e.g. `ASSERT_*`) failure in the
1492current thread whose message contains the given `substring`, or use
1493
1494```c++
1495  EXPECT_NONFATAL_FAILURE(statement, substring);
1496```
1497
1498if you are expecting a non-fatal (e.g. `EXPECT_*`) failure.
1499
1500Only failures in the current thread are checked to determine the result of this
1501type of expectations. If `statement` creates new threads, failures in these
1502threads are also ignored. If you want to catch failures in other threads as
1503well, use one of the following macros instead:
1504
1505```c++
1506  EXPECT_FATAL_FAILURE_ON_ALL_THREADS(statement, substring);
1507  EXPECT_NONFATAL_FAILURE_ON_ALL_THREADS(statement, substring);
1508```
1509
1510{: .callout .note}
1511NOTE: Assertions from multiple threads are currently not supported on Windows.
1512
1513For technical reasons, there are some caveats:
1514
15151.  You cannot stream a failure message to either macro.
1516
15172.  `statement` in `EXPECT_FATAL_FAILURE{_ON_ALL_THREADS}()` cannot reference
1518    local non-static variables or non-static members of `this` object.
1519
15203.  `statement` in `EXPECT_FATAL_FAILURE{_ON_ALL_THREADS}()` cannot return a
1521    value.
1522
1523## Registering tests programmatically
1524
1525The `TEST` macros handle the vast majority of all use cases, but there are few
1526where runtime registration logic is required. For those cases, the framework
1527provides the `::testing::RegisterTest` that allows callers to register arbitrary
1528tests dynamically.
1529
1530This is an advanced API only to be used when the `TEST` macros are insufficient.
1531The macros should be preferred when possible, as they avoid most of the
1532complexity of calling this function.
1533
1534It provides the following signature:
1535
1536```c++
1537template <typename Factory>
1538TestInfo* RegisterTest(const char* test_suite_name, const char* test_name,
1539                       const char* type_param, const char* value_param,
1540                       const char* file, int line, Factory factory);
1541```
1542
1543The `factory` argument is a factory callable (move-constructible) object or
1544function pointer that creates a new instance of the Test object. It handles
1545ownership to the caller. The signature of the callable is `Fixture*()`, where
1546`Fixture` is the test fixture class for the test. All tests registered with the
1547same `test_suite_name` must return the same fixture type. This is checked at
1548runtime.
1549
1550The framework will infer the fixture class from the factory and will call the
1551`SetUpTestSuite` and `TearDownTestSuite` for it.
1552
1553Must be called before `RUN_ALL_TESTS()` is invoked, otherwise behavior is
1554undefined.
1555
1556Use case example:
1557
1558```c++
1559class MyFixture : public testing::Test {
1560 public:
1561  // All of these optional, just like in regular macro usage.
1562  static void SetUpTestSuite() { ... }
1563  static void TearDownTestSuite() { ... }
1564  void SetUp() override { ... }
1565  void TearDown() override { ... }
1566};
1567
1568class MyTest : public MyFixture {
1569 public:
1570  explicit MyTest(int data) : data_(data) {}
1571  void TestBody() override { ... }
1572
1573 private:
1574  int data_;
1575};
1576
1577void RegisterMyTests(const std::vector<int>& values) {
1578  for (int v : values) {
1579    testing::RegisterTest(
1580        "MyFixture", ("Test" + std::to_string(v)).c_str(), nullptr,
1581        std::to_string(v).c_str(),
1582        __FILE__, __LINE__,
1583        // Important to use the fixture type as the return type here.
1584        [=]() -> MyFixture* { return new MyTest(v); });
1585  }
1586}
1587...
1588int main(int argc, char** argv) {
1589  std::vector<int> values_to_test = LoadValuesFromConfig();
1590  RegisterMyTests(values_to_test);
1591  ...
1592  return RUN_ALL_TESTS();
1593}
1594```
1595## Getting the Current Test's Name
1596
1597Sometimes a function may need to know the name of the currently running test.
1598For example, you may be using the `SetUp()` method of your test fixture to set
1599the golden file name based on which test is running. The
1600[`TestInfo`](reference/testing.md#TestInfo) class has this information.
1601
1602To obtain a `TestInfo` object for the currently running test, call
1603`current_test_info()` on the [`UnitTest`](reference/testing.md#UnitTest)
1604singleton object:
1605
1606```c++
1607  // Gets information about the currently running test.
1608  // Do NOT delete the returned object - it's managed by the UnitTest class.
1609  const testing::TestInfo* const test_info =
1610      testing::UnitTest::GetInstance()->current_test_info();
1611
1612  printf("We are in test %s of test suite %s.\n",
1613         test_info->name(),
1614         test_info->test_suite_name());
1615```
1616
1617`current_test_info()` returns a null pointer if no test is running. In
1618particular, you cannot find the test suite name in `SetUpTestSuite()`,
1619`TearDownTestSuite()` (where you know the test suite name implicitly), or
1620functions called from them.
1621
1622## Extending googletest by Handling Test Events
1623
1624googletest provides an **event listener API** to let you receive notifications
1625about the progress of a test program and test failures. The events you can
1626listen to include the start and end of the test program, a test suite, or a test
1627method, among others. You may use this API to augment or replace the standard
1628console output, replace the XML output, or provide a completely different form
1629of output, such as a GUI or a database. You can also use test events as
1630checkpoints to implement a resource leak checker, for example.
1631
1632### Defining Event Listeners
1633
1634To define a event listener, you subclass either
1635[`testing::TestEventListener`](reference/testing.md#TestEventListener) or
1636[`testing::EmptyTestEventListener`](reference/testing.md#EmptyTestEventListener)
1637The former is an (abstract) interface, where *each pure virtual method can be
1638overridden to handle a test event* (For example, when a test starts, the
1639`OnTestStart()` method will be called.). The latter provides an empty
1640implementation of all methods in the interface, such that a subclass only needs
1641to override the methods it cares about.
1642
1643When an event is fired, its context is passed to the handler function as an
1644argument. The following argument types are used:
1645
1646*   UnitTest reflects the state of the entire test program,
1647*   TestSuite has information about a test suite, which can contain one or more
1648    tests,
1649*   TestInfo contains the state of a test, and
1650*   TestPartResult represents the result of a test assertion.
1651
1652An event handler function can examine the argument it receives to find out
1653interesting information about the event and the test program's state.
1654
1655Here's an example:
1656
1657```c++
1658  class MinimalistPrinter : public testing::EmptyTestEventListener {
1659    // Called before a test starts.
1660    void OnTestStart(const testing::TestInfo& test_info) override {
1661      printf("*** Test %s.%s starting.\n",
1662             test_info.test_suite_name(), test_info.name());
1663    }
1664
1665    // Called after a failed assertion or a SUCCESS().
1666    void OnTestPartResult(const testing::TestPartResult& test_part_result) override {
1667      printf("%s in %s:%d\n%s\n",
1668             test_part_result.failed() ? "*** Failure" : "Success",
1669             test_part_result.file_name(),
1670             test_part_result.line_number(),
1671             test_part_result.summary());
1672    }
1673
1674    // Called after a test ends.
1675    void OnTestEnd(const testing::TestInfo& test_info) override {
1676      printf("*** Test %s.%s ending.\n",
1677             test_info.test_suite_name(), test_info.name());
1678    }
1679  };
1680```
1681
1682### Using Event Listeners
1683
1684To use the event listener you have defined, add an instance of it to the
1685googletest event listener list (represented by class
1686[`TestEventListeners`](reference/testing.md#TestEventListeners) - note the "s"
1687at the end of the name) in your `main()` function, before calling
1688`RUN_ALL_TESTS()`:
1689
1690```c++
1691int main(int argc, char** argv) {
1692  testing::InitGoogleTest(&argc, argv);
1693  // Gets hold of the event listener list.
1694  testing::TestEventListeners& listeners =
1695      testing::UnitTest::GetInstance()->listeners();
1696  // Adds a listener to the end.  googletest takes the ownership.
1697  listeners.Append(new MinimalistPrinter);
1698  return RUN_ALL_TESTS();
1699}
1700```
1701
1702There's only one problem: the default test result printer is still in effect, so
1703its output will mingle with the output from your minimalist printer. To suppress
1704the default printer, just release it from the event listener list and delete it.
1705You can do so by adding one line:
1706
1707```c++
1708  ...
1709  delete listeners.Release(listeners.default_result_printer());
1710  listeners.Append(new MinimalistPrinter);
1711  return RUN_ALL_TESTS();
1712```
1713
1714Now, sit back and enjoy a completely different output from your tests. For more
1715details, see [sample9_unittest.cc].
1716
1717[sample9_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample9_unittest.cc "Event listener example"
1718
1719You may append more than one listener to the list. When an `On*Start()` or
1720`OnTestPartResult()` event is fired, the listeners will receive it in the order
1721they appear in the list (since new listeners are added to the end of the list,
1722the default text printer and the default XML generator will receive the event
1723first). An `On*End()` event will be received by the listeners in the *reverse*
1724order. This allows output by listeners added later to be framed by output from
1725listeners added earlier.
1726
1727### Generating Failures in Listeners
1728
1729You may use failure-raising macros (`EXPECT_*()`, `ASSERT_*()`, `FAIL()`, etc)
1730when processing an event. There are some restrictions:
1731
17321.  You cannot generate any failure in `OnTestPartResult()` (otherwise it will
1733    cause `OnTestPartResult()` to be called recursively).
17342.  A listener that handles `OnTestPartResult()` is not allowed to generate any
1735    failure.
1736
1737When you add listeners to the listener list, you should put listeners that
1738handle `OnTestPartResult()` *before* listeners that can generate failures. This
1739ensures that failures generated by the latter are attributed to the right test
1740by the former.
1741
1742See [sample10_unittest.cc] for an example of a failure-raising listener.
1743
1744[sample10_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample10_unittest.cc "Failure-raising listener example"
1745
1746## Running Test Programs: Advanced Options
1747
1748googletest test programs are ordinary executables. Once built, you can run them
1749directly and affect their behavior via the following environment variables
1750and/or command line flags. For the flags to work, your programs must call
1751`::testing::InitGoogleTest()` before calling `RUN_ALL_TESTS()`.
1752
1753To see a list of supported flags and their usage, please run your test program
1754with the `--help` flag. You can also use `-h`, `-?`, or `/?` for short.
1755
1756If an option is specified both by an environment variable and by a flag, the
1757latter takes precedence.
1758
1759### Selecting Tests
1760
1761#### Listing Test Names
1762
1763Sometimes it is necessary to list the available tests in a program before
1764running them so that a filter may be applied if needed. Including the flag
1765`--gtest_list_tests` overrides all other flags and lists tests in the following
1766format:
1767
1768```none
1769TestSuite1.
1770  TestName1
1771  TestName2
1772TestSuite2.
1773  TestName
1774```
1775
1776None of the tests listed are actually run if the flag is provided. There is no
1777corresponding environment variable for this flag.
1778
1779#### Running a Subset of the Tests
1780
1781By default, a googletest program runs all tests the user has defined. Sometimes,
1782you want to run only a subset of the tests (e.g. for debugging or quickly
1783verifying a change). If you set the `GTEST_FILTER` environment variable or the
1784`--gtest_filter` flag to a filter string, googletest will only run the tests
1785whose full names (in the form of `TestSuiteName.TestName`) match the filter.
1786
1787The format of a filter is a '`:`'-separated list of wildcard patterns (called
1788the *positive patterns*) optionally followed by a '`-`' and another
1789'`:`'-separated pattern list (called the *negative patterns*). A test matches
1790the filter if and only if it matches any of the positive patterns but does not
1791match any of the negative patterns.
1792
1793A pattern may contain `'*'` (matches any string) or `'?'` (matches any single
1794character). For convenience, the filter `'*-NegativePatterns'` can be also
1795written as `'-NegativePatterns'`.
1796
1797For example:
1798
1799*   `./foo_test` Has no flag, and thus runs all its tests.
1800*   `./foo_test --gtest_filter=*` Also runs everything, due to the single
1801    match-everything `*` value.
1802*   `./foo_test --gtest_filter=FooTest.*` Runs everything in test suite
1803    `FooTest` .
1804*   `./foo_test --gtest_filter=*Null*:*Constructor*` Runs any test whose full
1805    name contains either `"Null"` or `"Constructor"` .
1806*   `./foo_test --gtest_filter=-*DeathTest.*` Runs all non-death tests.
1807*   `./foo_test --gtest_filter=FooTest.*-FooTest.Bar` Runs everything in test
1808    suite `FooTest` except `FooTest.Bar`.
1809*   `./foo_test --gtest_filter=FooTest.*:BarTest.*-FooTest.Bar:BarTest.Foo` Runs
1810    everything in test suite `FooTest` except `FooTest.Bar` and everything in
1811    test suite `BarTest` except `BarTest.Foo`.
1812
1813#### Stop test execution upon first failure
1814
1815By default, a googletest program runs all tests the user has defined. In some
1816cases (e.g. iterative test development & execution) it may be desirable stop
1817test execution upon first failure (trading improved latency for completeness).
1818If `GTEST_FAIL_FAST` environment variable or `--gtest_fail_fast` flag is set,
1819the test runner will stop execution as soon as the first test failure is
1820found.
1821
1822#### Temporarily Disabling Tests
1823
1824If you have a broken test that you cannot fix right away, you can add the
1825`DISABLED_` prefix to its name. This will exclude it from execution. This is
1826better than commenting out the code or using `#if 0`, as disabled tests are
1827still compiled (and thus won't rot).
1828
1829If you need to disable all tests in a test suite, you can either add `DISABLED_`
1830to the front of the name of each test, or alternatively add it to the front of
1831the test suite name.
1832
1833For example, the following tests won't be run by googletest, even though they
1834will still be compiled:
1835
1836```c++
1837// Tests that Foo does Abc.
1838TEST(FooTest, DISABLED_DoesAbc) { ... }
1839
1840class DISABLED_BarTest : public testing::Test { ... };
1841
1842// Tests that Bar does Xyz.
1843TEST_F(DISABLED_BarTest, DoesXyz) { ... }
1844```
1845
1846{: .callout .note}
1847NOTE: This feature should only be used for temporary pain-relief. You still have
1848to fix the disabled tests at a later date. As a reminder, googletest will print
1849a banner warning you if a test program contains any disabled tests.
1850
1851{: .callout .tip}
1852TIP: You can easily count the number of disabled tests you have using
1853`grep`. This number can be used as a metric for
1854improving your test quality.
1855
1856#### Temporarily Enabling Disabled Tests
1857
1858To include disabled tests in test execution, just invoke the test program with
1859the `--gtest_also_run_disabled_tests` flag or set the
1860`GTEST_ALSO_RUN_DISABLED_TESTS` environment variable to a value other than `0`.
1861You can combine this with the `--gtest_filter` flag to further select which
1862disabled tests to run.
1863
1864### Repeating the Tests
1865
1866Once in a while you'll run into a test whose result is hit-or-miss. Perhaps it
1867will fail only 1% of the time, making it rather hard to reproduce the bug under
1868a debugger. This can be a major source of frustration.
1869
1870The `--gtest_repeat` flag allows you to repeat all (or selected) test methods in
1871a program many times. Hopefully, a flaky test will eventually fail and give you
1872a chance to debug. Here's how to use it:
1873
1874```none
1875$ foo_test --gtest_repeat=1000
1876Repeat foo_test 1000 times and don't stop at failures.
1877
1878$ foo_test --gtest_repeat=-1
1879A negative count means repeating forever.
1880
1881$ foo_test --gtest_repeat=1000 --gtest_break_on_failure
1882Repeat foo_test 1000 times, stopping at the first failure.  This
1883is especially useful when running under a debugger: when the test
1884fails, it will drop into the debugger and you can then inspect
1885variables and stacks.
1886
1887$ foo_test --gtest_repeat=1000 --gtest_filter=FooBar.*
1888Repeat the tests whose name matches the filter 1000 times.
1889```
1890
1891If your test program contains
1892[global set-up/tear-down](#global-set-up-and-tear-down) code, it will be
1893repeated in each iteration as well, as the flakiness may be in it. You can also
1894specify the repeat count by setting the `GTEST_REPEAT` environment variable.
1895
1896### Shuffling the Tests
1897
1898You can specify the `--gtest_shuffle` flag (or set the `GTEST_SHUFFLE`
1899environment variable to `1`) to run the tests in a program in a random order.
1900This helps to reveal bad dependencies between tests.
1901
1902By default, googletest uses a random seed calculated from the current time.
1903Therefore you'll get a different order every time. The console output includes
1904the random seed value, such that you can reproduce an order-related test failure
1905later. To specify the random seed explicitly, use the `--gtest_random_seed=SEED`
1906flag (or set the `GTEST_RANDOM_SEED` environment variable), where `SEED` is an
1907integer in the range [0, 99999]. The seed value 0 is special: it tells
1908googletest to do the default behavior of calculating the seed from the current
1909time.
1910
1911If you combine this with `--gtest_repeat=N`, googletest will pick a different
1912random seed and re-shuffle the tests in each iteration.
1913
1914### Controlling Test Output
1915
1916#### Colored Terminal Output
1917
1918googletest can use colors in its terminal output to make it easier to spot the
1919important information:
1920
1921<pre>...
1922<font color="green">[----------]</font> 1 test from FooTest
1923<font color="green">[ RUN      ]</font> FooTest.DoesAbc
1924<font color="green">[       OK ]</font> FooTest.DoesAbc
1925<font color="green">[----------]</font> 2 tests from BarTest
1926<font color="green">[ RUN      ]</font> BarTest.HasXyzProperty
1927<font color="green">[       OK ]</font> BarTest.HasXyzProperty
1928<font color="green">[ RUN      ]</font> BarTest.ReturnsTrueOnSuccess
1929... some error messages ...
1930<font color="red">[   FAILED ]</font> BarTest.ReturnsTrueOnSuccess
1931...
1932<font color="green">[==========]</font> 30 tests from 14 test suites ran.
1933<font color="green">[   PASSED ]</font> 28 tests.
1934<font color="red">[   FAILED ]</font> 2 tests, listed below:
1935<font color="red">[   FAILED ]</font> BarTest.ReturnsTrueOnSuccess
1936<font color="red">[   FAILED ]</font> AnotherTest.DoesXyz
1937
1938 2 FAILED TESTS
1939</pre>
1940
1941You can set the `GTEST_COLOR` environment variable or the `--gtest_color`
1942command line flag to `yes`, `no`, or `auto` (the default) to enable colors,
1943disable colors, or let googletest decide. When the value is `auto`, googletest
1944will use colors if and only if the output goes to a terminal and (on non-Windows
1945platforms) the `TERM` environment variable is set to `xterm` or `xterm-color`.
1946
1947#### Suppressing test passes
1948
1949By default, googletest prints 1 line of output for each test, indicating if it
1950passed or failed. To show only test failures, run the test program with
1951`--gtest_brief=1`, or set the GTEST_BRIEF environment variable to `1`.
1952
1953#### Suppressing the Elapsed Time
1954
1955By default, googletest prints the time it takes to run each test. To disable
1956that, run the test program with the `--gtest_print_time=0` command line flag, or
1957set the GTEST_PRINT_TIME environment variable to `0`.
1958
1959#### Suppressing UTF-8 Text Output
1960
1961In case of assertion failures, googletest prints expected and actual values of
1962type `string` both as hex-encoded strings as well as in readable UTF-8 text if
1963they contain valid non-ASCII UTF-8 characters. If you want to suppress the UTF-8
1964text because, for example, you don't have an UTF-8 compatible output medium, run
1965the test program with `--gtest_print_utf8=0` or set the `GTEST_PRINT_UTF8`
1966environment variable to `0`.
1967
1968
1969
1970#### Generating an XML Report
1971
1972googletest can emit a detailed XML report to a file in addition to its normal
1973textual output. The report contains the duration of each test, and thus can help
1974you identify slow tests.
1975
1976To generate the XML report, set the `GTEST_OUTPUT` environment variable or the
1977`--gtest_output` flag to the string `"xml:path_to_output_file"`, which will
1978create the file at the given location. You can also just use the string `"xml"`,
1979in which case the output can be found in the `test_detail.xml` file in the
1980current directory.
1981
1982If you specify a directory (for example, `"xml:output/directory/"` on Linux or
1983`"xml:output\directory\"` on Windows), googletest will create the XML file in
1984that directory, named after the test executable (e.g. `foo_test.xml` for test
1985program `foo_test` or `foo_test.exe`). If the file already exists (perhaps left
1986over from a previous run), googletest will pick a different name (e.g.
1987`foo_test_1.xml`) to avoid overwriting it.
1988
1989The report is based on the `junitreport` Ant task. Since that format was
1990originally intended for Java, a little interpretation is required to make it
1991apply to googletest tests, as shown here:
1992
1993```xml
1994<testsuites name="AllTests" ...>
1995  <testsuite name="test_case_name" ...>
1996    <testcase    name="test_name" ...>
1997      <failure message="..."/>
1998      <failure message="..."/>
1999      <failure message="..."/>
2000    </testcase>
2001  </testsuite>
2002</testsuites>
2003```
2004
2005*   The root `<testsuites>` element corresponds to the entire test program.
2006*   `<testsuite>` elements correspond to googletest test suites.
2007*   `<testcase>` elements correspond to googletest test functions.
2008
2009For instance, the following program
2010
2011```c++
2012TEST(MathTest, Addition) { ... }
2013TEST(MathTest, Subtraction) { ... }
2014TEST(LogicTest, NonContradiction) { ... }
2015```
2016
2017could generate this report:
2018
2019```xml
2020<?xml version="1.0" encoding="UTF-8"?>
2021<testsuites tests="3" failures="1" errors="0" time="0.035" timestamp="2011-10-31T18:52:42" name="AllTests">
2022  <testsuite name="MathTest" tests="2" failures="1" errors="0" time="0.015">
2023    <testcase name="Addition" status="run" time="0.007" classname="">
2024      <failure message="Value of: add(1, 1)&#x0A;  Actual: 3&#x0A;Expected: 2" type="">...</failure>
2025      <failure message="Value of: add(1, -1)&#x0A;  Actual: 1&#x0A;Expected: 0" type="">...</failure>
2026    </testcase>
2027    <testcase name="Subtraction" status="run" time="0.005" classname="">
2028    </testcase>
2029  </testsuite>
2030  <testsuite name="LogicTest" tests="1" failures="0" errors="0" time="0.005">
2031    <testcase name="NonContradiction" status="run" time="0.005" classname="">
2032    </testcase>
2033  </testsuite>
2034</testsuites>
2035```
2036
2037Things to note:
2038
2039*   The `tests` attribute of a `<testsuites>` or `<testsuite>` element tells how
2040    many test functions the googletest program or test suite contains, while the
2041    `failures` attribute tells how many of them failed.
2042
2043*   The `time` attribute expresses the duration of the test, test suite, or
2044    entire test program in seconds.
2045
2046*   The `timestamp` attribute records the local date and time of the test
2047    execution.
2048
2049*   Each `<failure>` element corresponds to a single failed googletest
2050    assertion.
2051
2052#### Generating a JSON Report
2053
2054googletest can also emit a JSON report as an alternative format to XML. To
2055generate the JSON report, set the `GTEST_OUTPUT` environment variable or the
2056`--gtest_output` flag to the string `"json:path_to_output_file"`, which will
2057create the file at the given location. You can also just use the string
2058`"json"`, in which case the output can be found in the `test_detail.json` file
2059in the current directory.
2060
2061The report format conforms to the following JSON Schema:
2062
2063```json
2064{
2065  "$schema": "http://json-schema.org/schema#",
2066  "type": "object",
2067  "definitions": {
2068    "TestCase": {
2069      "type": "object",
2070      "properties": {
2071        "name": { "type": "string" },
2072        "tests": { "type": "integer" },
2073        "failures": { "type": "integer" },
2074        "disabled": { "type": "integer" },
2075        "time": { "type": "string" },
2076        "testsuite": {
2077          "type": "array",
2078          "items": {
2079            "$ref": "#/definitions/TestInfo"
2080          }
2081        }
2082      }
2083    },
2084    "TestInfo": {
2085      "type": "object",
2086      "properties": {
2087        "name": { "type": "string" },
2088        "status": {
2089          "type": "string",
2090          "enum": ["RUN", "NOTRUN"]
2091        },
2092        "time": { "type": "string" },
2093        "classname": { "type": "string" },
2094        "failures": {
2095          "type": "array",
2096          "items": {
2097            "$ref": "#/definitions/Failure"
2098          }
2099        }
2100      }
2101    },
2102    "Failure": {
2103      "type": "object",
2104      "properties": {
2105        "failures": { "type": "string" },
2106        "type": { "type": "string" }
2107      }
2108    }
2109  },
2110  "properties": {
2111    "tests": { "type": "integer" },
2112    "failures": { "type": "integer" },
2113    "disabled": { "type": "integer" },
2114    "errors": { "type": "integer" },
2115    "timestamp": {
2116      "type": "string",
2117      "format": "date-time"
2118    },
2119    "time": { "type": "string" },
2120    "name": { "type": "string" },
2121    "testsuites": {
2122      "type": "array",
2123      "items": {
2124        "$ref": "#/definitions/TestCase"
2125      }
2126    }
2127  }
2128}
2129```
2130
2131The report uses the format that conforms to the following Proto3 using the
2132[JSON encoding](https://developers.google.com/protocol-buffers/docs/proto3#json):
2133
2134```proto
2135syntax = "proto3";
2136
2137package googletest;
2138
2139import "google/protobuf/timestamp.proto";
2140import "google/protobuf/duration.proto";
2141
2142message UnitTest {
2143  int32 tests = 1;
2144  int32 failures = 2;
2145  int32 disabled = 3;
2146  int32 errors = 4;
2147  google.protobuf.Timestamp timestamp = 5;
2148  google.protobuf.Duration time = 6;
2149  string name = 7;
2150  repeated TestCase testsuites = 8;
2151}
2152
2153message TestCase {
2154  string name = 1;
2155  int32 tests = 2;
2156  int32 failures = 3;
2157  int32 disabled = 4;
2158  int32 errors = 5;
2159  google.protobuf.Duration time = 6;
2160  repeated TestInfo testsuite = 7;
2161}
2162
2163message TestInfo {
2164  string name = 1;
2165  enum Status {
2166    RUN = 0;
2167    NOTRUN = 1;
2168  }
2169  Status status = 2;
2170  google.protobuf.Duration time = 3;
2171  string classname = 4;
2172  message Failure {
2173    string failures = 1;
2174    string type = 2;
2175  }
2176  repeated Failure failures = 5;
2177}
2178```
2179
2180For instance, the following program
2181
2182```c++
2183TEST(MathTest, Addition) { ... }
2184TEST(MathTest, Subtraction) { ... }
2185TEST(LogicTest, NonContradiction) { ... }
2186```
2187
2188could generate this report:
2189
2190```json
2191{
2192  "tests": 3,
2193  "failures": 1,
2194  "errors": 0,
2195  "time": "0.035s",
2196  "timestamp": "2011-10-31T18:52:42Z",
2197  "name": "AllTests",
2198  "testsuites": [
2199    {
2200      "name": "MathTest",
2201      "tests": 2,
2202      "failures": 1,
2203      "errors": 0,
2204      "time": "0.015s",
2205      "testsuite": [
2206        {
2207          "name": "Addition",
2208          "status": "RUN",
2209          "time": "0.007s",
2210          "classname": "",
2211          "failures": [
2212            {
2213              "message": "Value of: add(1, 1)\n  Actual: 3\nExpected: 2",
2214              "type": ""
2215            },
2216            {
2217              "message": "Value of: add(1, -1)\n  Actual: 1\nExpected: 0",
2218              "type": ""
2219            }
2220          ]
2221        },
2222        {
2223          "name": "Subtraction",
2224          "status": "RUN",
2225          "time": "0.005s",
2226          "classname": ""
2227        }
2228      ]
2229    },
2230    {
2231      "name": "LogicTest",
2232      "tests": 1,
2233      "failures": 0,
2234      "errors": 0,
2235      "time": "0.005s",
2236      "testsuite": [
2237        {
2238          "name": "NonContradiction",
2239          "status": "RUN",
2240          "time": "0.005s",
2241          "classname": ""
2242        }
2243      ]
2244    }
2245  ]
2246}
2247```
2248
2249{: .callout .important}
2250IMPORTANT: The exact format of the JSON document is subject to change.
2251
2252### Controlling How Failures Are Reported
2253
2254#### Detecting Test Premature Exit
2255
2256Google Test implements the _premature-exit-file_ protocol for test runners
2257to catch any kind of unexpected exits of test programs. Upon start,
2258Google Test creates the file which will be automatically deleted after
2259all work has been finished. Then, the test runner can check if this file
2260exists. In case the file remains undeleted, the inspected test has exited
2261prematurely.
2262
2263This feature is enabled only if the `TEST_PREMATURE_EXIT_FILE` environment
2264variable has been set.
2265
2266#### Turning Assertion Failures into Break-Points
2267
2268When running test programs under a debugger, it's very convenient if the
2269debugger can catch an assertion failure and automatically drop into interactive
2270mode. googletest's *break-on-failure* mode supports this behavior.
2271
2272To enable it, set the `GTEST_BREAK_ON_FAILURE` environment variable to a value
2273other than `0`. Alternatively, you can use the `--gtest_break_on_failure`
2274command line flag.
2275
2276#### Disabling Catching Test-Thrown Exceptions
2277
2278googletest can be used either with or without exceptions enabled. If a test
2279throws a C++ exception or (on Windows) a structured exception (SEH), by default
2280googletest catches it, reports it as a test failure, and continues with the next
2281test method. This maximizes the coverage of a test run. Also, on Windows an
2282uncaught exception will cause a pop-up window, so catching the exceptions allows
2283you to run the tests automatically.
2284
2285When debugging the test failures, however, you may instead want the exceptions
2286to be handled by the debugger, such that you can examine the call stack when an
2287exception is thrown. To achieve that, set the `GTEST_CATCH_EXCEPTIONS`
2288environment variable to `0`, or use the `--gtest_catch_exceptions=0` flag when
2289running the tests.
2290
2291### Sanitizer Integration
2292
2293The
2294[Undefined Behavior Sanitizer](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html),
2295[Address Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer),
2296and
2297[Thread Sanitizer](https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual)
2298all provide weak functions that you can override to trigger explicit failures
2299when they detect sanitizer errors, such as creating a reference from `nullptr`.
2300To override these functions, place definitions for them in a source file that
2301you compile as part of your main binary:
2302
2303```
2304extern "C" {
2305void __ubsan_on_report() {
2306  FAIL() << "Encountered an undefined behavior sanitizer error";
2307}
2308void __asan_on_error() {
2309  FAIL() << "Encountered an address sanitizer error";
2310}
2311void __tsan_on_report() {
2312  FAIL() << "Encountered a thread sanitizer error";
2313}
2314}  // extern "C"
2315```
2316
2317After compiling your project with one of the sanitizers enabled, if a particular
2318test triggers a sanitizer error, googletest will report that it failed.
2319