Lines Matching +full:external +full:- +full:contents
2 LLVM test-suite Makefile Guide
5 .. contents::
11 This document describes the features of the Makefile-based LLVM
12 test-suite. This way of interacting with the test-suite is deprecated in
13 favor of running the test-suite using LNT, but may continue to prove
14 useful for some users. See the Testing Guide's :ref:`test-suite Quickstart
15 <test-suite-quickstart>` section for more information.
20 The ``test-suite`` module contains a number of programs that can be
36 In addition for testing correctness, the ``test-suite`` directory also
42 ``test-suite`` tests are divided into three types of tests: MultiSource,
43 SingleSource, and External.
45 - ``test-suite/SingleSource``
52 - ``test-suite/MultiSource``
58 - ``test-suite/External``
60 The External directory contains Makefiles for building code that is
61 external to (i.e., not distributed with) LLVM. The most prominent
63 suites. The ``External`` directory does not contain these actual
66 external programs is configured by the test-suite ``configure``
96 #. Check out the ``test-suite`` module with:
98 .. code-block:: bash
100 % svn co http://llvm.org/svn/llvm-project/test-suite/trunk test-suite
102 This will get the test suite into ``llvm/projects/test-suite``.
106 #. Configure and build ``llvm-gcc``.
108 #. Install ``llvm-gcc`` somewhere.
110 #. *Re-configure* ``llvm`` from the top level of each build tree (LLVM
114 During the *re-configuration*, you must either: (1) have ``llvm-gcc``
116 just-built ``llvm-gcc`` is installed using
117 ``--with-llvmgccdir=$LLVM_GCC_DIR``.
122 .. code-block:: bash
124 % cd $LLVM_OBJ_ROOT ; $LLVM_SRC_ROOT/configure [--with-llvmgccdir=$LLVM_GCC_DIR]
127 *installed* llvm-gcc, not its src or obj directory.]
131 .. code-block:: bash
133 % cd $LLVM_OBJ_ROOT/projects/test-suite
140 Configuring External Tests
141 --------------------------
143 In order to run the External tests in the ``test-suite`` module, you
144 must specify *--with-externals*. This must be done during the
145 *re-configuration* step (see above), and the ``llvm`` re-configuration
146 must recognize the previously-built ``llvm-gcc``. If any of these is
147 missing or neglected, the External tests won't work.
149 * *--with-externals*
151 * *--with-externals=<directory>*
153 This tells LLVM where to find any external tests. They are expected to
171 -----------------------
173 In addition to the regular "whole program" tests, the ``test-suite``
191 ----------------------
201 specified test and usually adds per-program summaries to the output
203 explicitly outputs TEST-PASS or TEST-FAIL for every test after each
209 exact contents of the report are dependent on which ``TEST`` you are
217 ---------------------------------------
233 "``test-suite/TEST.XXX.Makefile``" fragment (where XXX is the name of
234 your test) and a "``test-suite/TEST.XXX.report``" file that indicates
242 .. code-block:: bash
244 % cd llvm/projects/test-suite/MultiSource/Benchmarks # or some other level
260 Prolangs-C/TimberWolfMC/timberwolfmc | 302 | * |
261 Prolangs-C/agrep/agrep | 33 | 12 |
262 Prolangs-C/allroots/allroots | * | * |
263 Prolangs-C/assembler/assembler | 47 | * |
264 Prolangs-C/bison/mybison | 74 | * |
267 This basically is grepping the -stats output and displaying it in a
271 The source for this is in ``test-suite/TEST.libcalls.*``. The format is
273 "``opt -simplify-libcalls -stats``"), and the report contains one line