Lines Matching +full:test +full:- +full:suite

11   recursive search to find the test suite for each item, but then we only do a
13   that matches a file on disk we explicitly construct a test instance (bypassing
18   * The test format doesn't have control over the test instances that result
24   * The test format doesn't have full control over the discovery of tests in
28   are resolved to test suites, and then the resolution of the input specifier is
29   delegated to each test suite. This could take a couple forms:
31   * We could resolve to test suites, then fully load each test suite, then have
32     a fixed process to map input specifiers to tests in the test suite
33     (presumably based on path-in-suite derivations). This has the benefit of
34     being consistent across all test formats, but the downside of requiring
35     loading the entire test suite.
37   * We could delegate all of the resolution of specifiers to the test
38     suite. This would allow formats that anticipate large test suites to manage
41     for directories, but allow the test format control over what happens for
44 2. Consider move to identifying all tests by path-to-test-suite and then path to
45    subtest, and don't use test suite names.
47   Currently the test suite name is presented as part of test names, but it has
49   cut-and-paste a name to subsequently use to rerun a test. If we just
50   represented each test suite by the path to its suite, then it would allow more
51   easy cut-and-paste of the test output lines. This has the downside that the
54 3. Allow 'lit' driver to cooperate with test formats and suites to add options
57   We have started to use the --params method more and more extensively, and it is
60   ShTest test format.
62   It would be really nice if we could allow test formats and test suites to add
64   don't know what test formats or test suites are in use until we have parsed the
65   input specifiers. For test formats we could ostensibly require all the possible
66   formats to be registered in order to have options, but for test suites we would
67   certainly have to load the suite before we can query it for what options it
73     having done option parsing first (the exception is ``--config-prefix``) but
76   * We could make a distinction in the command line syntax for test format and
77     test suite options. For example, we could require something like::
79       lit -j 1 -sv input-specifier -- --some-format-option
87     infrastructure and format or suite specific options.
91   Currently, the ShTest format uses tests written with shell-script like syntax,
103   developer writing the test.
117     using --use-processes.
126   Many of the existing test formats are cleanly divided into two phases, once
127   parses the test format and extracts XFAIL and REQUIRES information, etc., and
128   the other code actually executes the test.
141   * This affords a clean implementation of --no-execute.
143   * One possible downside could be for test formats that cannot determine their
144     subtests without having executed the test. Supporting such formats would
145     either force the test to actually be executed in the setup stage (which
157 * Move temp directory name into local test config.
163 * Create an explicit test suite object (instead of using the top-level
169   for killing a process's children which is needed for a per test timeout. On