Lines Matching full:suite
21 from autotest_lib.server.cros.dynamic_suite.suite import ProvisionSuite
22 from autotest_lib.server.cros.dynamic_suite.suite import Suite
26 """CrOS dynamic test suite generation and execution module.
35 dynamic suite across all reimaged devices.
37 The public API for defining a suite includes one method: reimage_and_run().
38 A suite control file can be written by importing this module and making
55 "SUITE=" clause, schedule jobs to reimage |num| or less devices in the
61 atest suite create -b <board> -i <build/name> <suite>
63 atest suite create -b x86-mario -i x86-mario/R20-2203.0.0 bvt
68 A Suite instance represents a single test suite, defined by some predicate
69 run over all known control files. The simplest example is creating a Suite
72 create_suite_job() takes the parameters needed to define a suite run (board,
73 build to test, machine pool, and which suite to run), ensures important
74 preconditions are met, finds the appropraite suite control file, and then
86 RPC pulls the control file for the suite to be run from the dev server and
87 uses it to create the suite job with the autotest frontend.
100 +-------------+ suite control file +--------------------------+
103 Suite Job (hostless)
121 on it. In the case of a dynamic_suite, many tests in the suite may have
123 they specify, along with any suite dependencies that were specified, and the
127 A Suite instance uses the labels specified in the suite dependencies to
130 run a suite in the background _after_ it has completed all the things
131 necessary for reimaging. Before running a suite, reimage_and_run() calls out
136 0) At instantiation time, find all appropriate control files for this suite
143 | |---------------------->| [Suite Job] |
146 1) Now that the Suite instance exists, it schedules jobs for every control
149 build they were testing and which suite they were for.
154 | [Suite Job] |----------->| Host 2 |
161 'suite': suite_name}
185 | |<--------------------------| [Suite Job] |
189 2) As jobs finish, we record their success or failure in the status of the suite
190 job. We also record a 'job keyval' in the suite job for each test, noting
192 3) Once all jobs are complete, status is recorded for the suite job, and the
193 job_repo_url host attribute is removed from all hosts used by the suite.
201 """This class contains the info that defines a suite run."""
253 @param name: a value of the SUITE control file variable to search for.
255 currently running suite job.
263 @param file_bugs: File bugs when tests in this suite fail.
265 this suite will run.
267 that this suite runs.
268 @param suite_dependencies: A list of strings of suite level
275 filing options for failures in this suite.
279 included in suite. If argument is absent, suite
280 behavior will default to creating a suite of based
281 on the SUITE field of control files.
282 @param wait_for_results: Set to False to run the suite job without
285 @param max_retries: Maximum retry limit at suite level if not None.
288 happening in the suite can't exceed max_retries.
295 @param run_prod_code: If true, the suite will run the test code that
345 warnings.warn('Ignored key %r was passed to suite with value %r'
364 self.predicate = Suite.name_in_tag_predicate(self.name)
370 """Initialize suite dependencies attribute."""
391 self.test_source_build = Suite.get_test_source_build(
422 Run a provision suite.
428 currently running suite job.
449 suite = ProvisionSuite(
474 _run_suite_with_spec(suite, spec)
484 provided builds, and then run the indicated test suite on them.
522 suite = Suite.create_from_predicates(
546 _run_suite_with_spec(suite, spec)
549 def _run_suite_with_spec(suite, spec): argument
553 @param suite: _BaseSuite instance to run.
557 suite=suite,
567 suite,
575 Run a suite.
577 @param suite: _BaseSuite instance.
579 currently running suite job.
586 filing options for failures in this suite.
607 suite.schedule(job.record_entry)
609 if suite.wait_for_results:
610 logging.debug('Waiting on suite.')
611 reporter = suite.get_result_reporter(bug_template)
612 suite.wait(job.record_entry, reporter=reporter)
613 logging.debug('Finished waiting on suite. '
616 logging.info('wait_for_results is set to False, suite job will exit '
621 """Stage artifacts for a suite job.