Lines Matching +full:build +full:- +full:and +full:- +full:test
4 Android and iOS devices.
6 This package contains the code for test framework and the tests. More generally
10 Build results are available at https://flutter-dashboard.appspot.com.
14 ## The build page
16 The build page is accessible at https://flutter-dashboard.appspot.com/build.html.
17 This page reports the health of build servers, called _agents_, and the statuses
18 of build tasks.
22 A green agent is considered healthy and ready to receive new tasks to build. A
23 red agent is broken and does not receive new tasks.
25 In the example below, the dashboard shows that the `linux2` agent is broken and
28 
32 The table below the agent statuses displays the statuses of build tasks. Task
33 statuses are color-coded. The following statuses are available:
35 **New task** (light blue): the task is waiting for an agent to pick it up and
36 start the build.
50 **Task was skipped** (transparent): the task is not scheduled for a build. This
55 In addition to color-coding, a task may display a question mark. This means
57 when considering whether the build is broken or not. For example, if a flaky
59 status of a non-flaky task is red, all pending PRs will contain a warning about
60 the broken build and recommend caution when submitting.
66 The example below shows that commit `e122d5d` caused a wide-spread breakage,
67 which was fixed by `bdc6f10`. It also shows that Cirrus and Chrome
68 Infra (left-most tasks) decided to skip building these commits. Hovering over
73 
77 The dashboard aggregates build results from multiple build environments,
78 including Cirrus, Chrome Infra, and devicelab. While devicelab
80 may skip some commits. For example, Cirrus will only test the
89 reports the success to the server and the dashboard shows that task in green.
92 and puts the task back in the pool of available tasks. If a task does not
94 the task is marked as failed and is displayed using red color on the dashboard.
101 reproduce a CI test failure locally.
106 tests on Android. If you have a local build of the Flutter engine, then you have
115 Notably, it will start and stop gradle, for instance.
119 To run all tests defined in `manifest.yaml`, use option `-a` (`--all`):
122 ../../bin/cache/dart-sdk/bin/dart bin/run.dart -a
127 To run a test, use option `-t` (`--task`):
131 ../../bin/cache/dart-sdk/bin/dart bin/run.dart -t {NAME_OR_PATH_OF_TEST}
136 - the _name_ of a task, which you can find in the `manifest.yaml` file in this
138 - the path to a Dart _file_ corresponding to a task, which resides in `bin/tasks`.
139 Tip: most shells support path auto-completion using the Tab key. Example:
142 To run multiple tests, repeat option `-t` (`--task`) multiple times:
145 ../../bin/cache/dart-sdk/bin/dart bin/run.dart -t test1 -t test2 -t test3
148 To run tests from a specific stage, use option `-s` (`--stage`).
150 `devicelab_ios` and `devicelab_win`.
154 ../../bin/cache/dart-sdk/bin/dart bin/run.dart -s {NAME_OF_STAGE}
157 ## Running tests against a local engine build
159 To run device lab tests against a local engine build, pass the appropriate
163 ../../bin/cache/dart-sdk/bin/dart bin/run.dart --task=[some_task] \
164 --local-engine-src-path=[path_to_local]/engine/src \
165 --local-engine=[local_engine_architecture]
173 revision. Note the name of the test that failed. In the example above the
174 failing test is `flutter_gallery__transition_perf`. This name can be passed to
178 ../../bin/cache/dart-sdk/bin/dart bin/run.dart -t flutter_gallery__transition_perf
183 A test is a simple Dart program that lives under `bin/tests` and uses
184 `package:flutter_devicelab/framework/framework.dart` to define and run a _task_.
210 of tests internally. A task has a name. It succeeds and fails independently of
211 other tasks, and is reported to the dashboard independently of other tasks.
213 A task runs in its own standalone Dart VM and reports results via Dart VM
214 service protocol. This ensures that tasks do not interfere with each other and
215 lets the CI system time out and clean up tasks that get stuck.
220 your test edit `manifest.yaml` and add the following in the "tasks" dictionary:
231 - `{NAME_OF_TEST}` is the name of your test that also matches the name of the
233 - `{DESCRIPTION}` is the plain English description of your test that helps
234 others understand what this test is testing.
235 - `{STAGE}` is `devicelab` if you want to run on Android, or `devicelab_ios` if
237 - `{CAPABILITIES}` is an array that lists the capabilities required of
238 the test agent (the computer that runs the test) to run your test. Available
239 capabilities are: `has-android-device`, `has-ios-device`.