Lines Matching +full:test +full:- +full:infra
3 This CI is used on-top of (not in replacement of) AOSP's TreeHugger.
12 
32 It uses a non-privileged gmail account and has no meaningful voting power.
36 - It periodically (every 15s) polls Gerrit for CLs updated in the last 24h.
37 - It checks the list of CLs against the list of already known CLs in the DB.
38 - For each new CL it enqueues `N` new jobs in the database, one for each
39 configuration defined in [config.py](/infra/ci/config.py) (e.g. `linux-debug`,
40 `android-release`, ...).
41 - It monitors the state of jobs. When all jobs for a CL have been completed,
42 it posts a comment and adds the vote if the CL is marked as `Presubmit-Ready`.
43 - It does some other less-relevant bookkeeping.
44 - AppEngine is highly reliable and self-healing. If a task fails (e.g. because
45 of a Gerrit 500) it will be automatically re-tried with exponential backoff.
53 - It's an almost fully static website based on HTML and Javascript.
54 - The only backend-side code ([frontend.py](/infra/ci/frontend/main.py))
57 - Such XHR requests are GET-only and anonymous.
58 - The frontend python code also serves as a memcache layer for Gerrit requests
65 The GCE instance is running a CrOS-based
66 [Container-Optimized](https://cloud.google.com/container-optimized-os/docs/) OS.
68 The whole system image is read-only. The VM itself is stateless. No state is
78 associated. Workers are always alive (they work in polling-mode), while
79 sandboxes are started and stopped by the worker on-demand.
90 on the checkout under test. The reason why workers are Docker containers is NOT
95 - Poll for an available job from the `/jobs_queued` sub-tree of the DB.
96 - Move such job into `/jobs_running`.
97 - Start the sandbox container, passing down the job config and the git revision
99 - Stream the sandbox stdout to the `/logs` sub-tree of the DB.
100 - Terminate the sandbox container prematurely in case of timeouts or job
102 - Upload UI artifacts to GCS.
103 - Update the DB to reflect completion of jobs, removing the entry from
109 (for git pull / install-build-deps) but they cannot impersonate the GCE service
116 - Checkout the code at the revision specified in the job config.
117 - Run one of the [test/ci/](/test/ci/) scripts which will build and run tests.
118 - Return either a success (0) or fail (!= 0) exit code.
121 the semi-ephemeral `/ci/cache` mount-point. This mount-point is tmpfs-based
129 It is a high-scale JSON object accessible via a simple REST API.
130 Clients can GET/PUT/PATCH/DELETE individual sub-nodes without having a local
131 full-copy of the DB.
135 # For post-submit jobs.
137 /main-20190626000853
138 # ┃ ┗━ Committer-date of the HEAD of the branch.
143 subject: "Cloud-based CI"
144 time_committed: "2019-07-06T02:35:14Z"
147 20190708153242--branches-main-20190626000853--android-...: 0
148 20190708153242--branches-main-20190626000853--linux-...: 0
152 /main-20190701235742 {...}
154 # For pre-submit jobs.
156 /1000515-65
159 time_queued: "2019-07-08T15:32:42Z"
160 time_ended: "2019-07-08T15:33:25Z"
165 20190708153242--cls-1000515-65--android-clang: 0
167 20190708153242--cls-1000515-65--ui-clang: 0
170 /1000515-66 {...}
172 /1011130-3 {...}
178 /1000515-65: 0
181 /20190708153242--cls-1000515-65--android-clang-arm-debug:
186 src: "cls/1000515-66"
194 time_ended: "2019-07-07T12:47:22Z"
195 time_queued: "2019-07-07T12:34:22Z"
196 time_started: "2019-07-07T12:34:25Z"
197 type: "android-clang-arm-debug"
198 worker: "zqz2-worker-2"
200 /20190707123422--cls-1000515-66--android-clang-arm-rel {..}
205 /20190708153242--cls-1000515-65--android-clang-arm-debug: 0
210 /20190707123422--cls-1000515-66--android-clang-arm-rel
213 /20190707123422--cls-1000515-66--android-clang-arm-rel
214 /00a053-0000: "+ chmod 777 /ci/cache /ci/artifacts"
218 # ┗━ Hex-encoded timestamp, relative since start of test.
219 /00a053-0001: "+ chown perfetto.perfetto /ci/ramdisk"
226 This is what happens, in order, on a worker instance from boot to the test run.
229 make -C /infra/ci worker-start
232 [GCE] # From /infra/ci/worker/gce-startup-script.sh
233 docker run worker-1 ...
235 docker run worker-N ...
237 [worker-X] # From /infra/ci/worker/Dockerfile
238 ┗━ /infra/ci/worker/worker.py
239 ┗━ docker run sandbox-X ...
241 [sandbox-X] # From /infra/ci/sandbox/Dockerfile
242 ┗━ /infra/ci/sandbox/init.sh
243 ┗━ /infra/ci/sandbox/testrunner.sh
246 ┇ # This env var is passed by the test definition
247 ┇ # specified in /infra/ci/config.py .
250 ┣━ /test/ci/android_tests.sh
251 ┣━ /test/ci/fuzzer_tests.sh
252 ┣━ /test/ci/linux_tests.sh
253 ┗━ /test/ci/ui_tests.sh
255 ┗━ out/dist/{unit,integration,...}test
258 ### [gce-startup-script.sh](/infra/ci/worker/gce-startup-script.sh)
260 - Is ran once per GVE vm, at (re)boot.
261 - It prepares the tmpfs mountpoint for the shared ccache.
262 - It wipes the SSD scratch disk for the build artifacts
263 - It pulls the latest {worker, sandbox} container images from
265 - Sets up Docker and `iptables` (for the sandboxed network).
266 - Starts `N` worker containers in Docker.
268 ### [worker.py](/infra/ci/worker/worker.py)
270 - It polls the DB to retrieve a job.
271 - When a job is retrieved starts a sandbox container.
272 - It streams the container stdout/stderr to the DB.
273 - It upload the build artifacts to GCS.
275 ### [testrunner.sh](/infra/ci/sandbox/testrunner.sh)
277 - It is pinned in the container image. Does NOT depend on the particular
279 - Checks out the repo at the revision specified (by the Controller) in the
281 - Sets up ccache
282 - Deals with caching of buildtools/.
283 - Runs the test script specified in the job config from the checkout.
285 ### [{android,fuzzer,linux,ui}_tests.sh](/test/ci/linux_tests.sh)
287 - Are NOT pinned in the container and are ran from the checked out revision.
288 - Finally build and run the test.
294 Test-locally: `make -C infra/ci/frontend test`
296 Deploy with `make -C infra/ci/frontend deploy`
300 Deploy with `make -C infra/ci/controller deploy`
302 It is possible to try locally via the `make -C infra/ci/controller test`
305 - Manually stopping the production AppEngine instance via the Cloud Console
312 `make -C infra/ci build push`
316 `make -C infra/ci restart-workers`
321 - Stop the workers: `make -C infra/ci stop-workers`
322 - Open https://console.firebase.google.com/u/0/project/perfetto-ci/database/perfetto-ci/data/~2Fci
323 - Delete the `jobs_running`, `jobs_queued`, `workers` subtrees
324 - Restart the workers: `make -C infra/ci start-workers`
328 - Both the Firebase DB and the gs://perfetto-artifacts GCS bucket are
329 world-readable and writable by the GAE and GCE service accounts.
331 - The GAE service account also has the ability to log into Gerrit using a
334 - Overall, no account in this project has any interesting privilege:
335 - The Gerrit account used for commenting on CLs is just a random gmail account
337 - The service accounts of GAE and GCE don't have any special capabilities
340 - This CI deals only with functional and performance testing and doesn't deal
343 - Presubmit jobs are only triggered if at least one of the following is true:
344 - The owner of the CL is a @google.com account.
345 - The user that applied the Presubmit-Ready label is a @google.com account.
347 - Sandboxes are not too hard to escape (Docker is the only boundary) and can
350 - As such neither pre-submit nor post-submit build artifacts are considered
354 - Binaries built by the CI are not ran on any other machines outside of the
357 - The only build artifacts that are retained (for up to 30 days) and uploaded to
361 - UI artifacts are served from a different origin (the GCS per-bucket API) than