Lines Matching +full:localhost +full:- +full:test +full:- +full:linux
1 # How to run FAFT (Fully Automated Firmware Test) {#faft-how-to-run}
3 _Self-link: [go/faft-running](https://goto.google.com/faft-running)_
7 ## FAFT Overview {#faft-overview}
11 The features tested by FAFT are implemented through low-level software
13 (Semi-Automated Firmware Tests) and you can locate tests in the [FAFT suite]
18 - Fully automated, no human intervention required
19 - Real test of physical hardware, like USB plug-in, Ctrl-D key press
20 - High test coverage of complicated verified boot flows
21 - Easy to integrate with existing test infrastructure (e.g. test lab, continuous testing, etc).
23 To access some of these low-level capabilities, the tests require a
25 [servo] board ([servo v2], [servo v4] with [servo micro] or [servo v4 Type-C])
27 The servo board is connected directly to the DUT (Device Under Test) to enable
28 access to low-level hardware interfaces, as well as staging areas for backup
31 The [FAFT framework] runs the tests with a tool called [test that] and it is
32 based on a client-server architecture, where the client runs on the DUT and
47 ## Hardware Setup {#hardware-setup}
66 The OS on the USB drive and on the disk must be a test image. If not, it
68 that `rsync` can't be found you're not using a test image and while
71 non-test ChromeOS lacks a suitable python interpreter.
73 ### ServoV4 Type-A with Micro {#servov4-typea-micro}
75 The hardware configuration for running FAFT on a servo v4 Type-A
78 - A test controller (your host workstation with a working chroot environment)
79 - The test device (a device / DUT that can boot ChromeOS)
80 - A servo board
81 - Related cables and components
82 - servo-micro cable
83 - USB type-A to USB micro cable for DUT connection (~ 2' in length)
84 - USB type-A to USB micro cable for test controller connection (~ 4' - 6' in length)
85 - Ethernet cable
86 - USB drive (flashed with the appropriate OS image)
89 servoV4 Type-A and servo micro, to the test controller, DUT, and network.
95 **Figure 1.Diagram of hardware configuration for a ServoV4 Type-A with servo micro.**
97 Details of servoV4 Type-A with micro connections:
106 ### ServoV4 Type-C {#servov4-typec}
108 The hardware configuration for running FAFT with a servo v4 type-C includes:
110 - A test controller (your host workstation with a working chroot environment)
111 - The test device (a device / DUT that can boot ChromeOS)
112 - A servo board
113 - Related cables and components
114 - USB type-A to USB micro cable for test controller connection (~ 4' - 6' in length)
115 - Ethernet cable
116 - USB drive (flashed with the appropriate OS image)
118 Figure 2 shows a diagram of how to connect a servoV4 Type-C, to the test
124 **Figure 2.Diagram of hardware configuration for a ServoV4 Type-C.**
126 Details of servoV4 Type-C connections in Figure 2:
128 1. Connect the USB Type-C cable of the servoV4 to the DUT.
133 ### ServoV4 Type-C with servo micro {#servov4-typec-micro}
144 **Figure 3.Diagram of hardware configuration for a ServoV4 Type-C with servo micro.**
146 Details about FAFT PD's ServoV4 Type-C + servo micro setup (Figure 3):
148 - The suite should only be run on devices released in 2019 and forward.
149 - The charger connected to the servo must have support for 5V, 12V, and 20V.
150 - The servo v4 and servo micro cable must be updated to their latest FW:
151 - Servo_v4: servo_v4_v2.3.30-b35860984
152 - servo micro: servo_micro_v2.3.30-b35960984
156 - Have the servo v4 connected to your workstation/labstation along with the servo micro connected t…
157 - Run the following commands on chroot one after the other:
158 - sudo servo_updater -b servo_v4
159 - sudo servo_updater -b servo_micro
161 ### (Deprecated) ServoV2 {#servov2-deprecated}
164 deprecated servo v2 board to the test controller, test device, and network.
178 ### Installing Test Image onto USB Stick {#image-onto-usb}
181 prepare and install a test Chromium OS image:
183 1. Build the binary (chromiumos_test_image.bin) with build_image test, or fetch the file from a bui…
184 2. Load the test image onto a USB drive (use cros flash).
186 4. Install the test image onto the internal disk by booting from the USB drive and running chromeos…
188 ## Running Tests {#faft-running-tests}
192 … tests are run using the `tast run` command, which is documented at [go/tast-running](http://chrom…
194 ### Setup Confirmation {#setup-confirmation}
199 the SDK, using either multiple SDK instances (`cros_sdk --enter --no-ns-pid`)
204 1. Run `repo sync -j8`
206 …1) Run `$ sudo servod --board=$BOARD` where `$BOARD` is the code name of the board you are testing…
208 4. (chroot 2) Run the `firmware_FAFTSetup` test to verify basic functionality and ensure that your…
209 …`/usr/bin`, the syntax is `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/fi…
210 6. Run the `firmware.Pre.normal` test to verify tast tests are working also. `tast run --var=servo…
212 You can omit the --autotest_dir if you have built packages for the board and want to use the build …
214 (chroot) `$ ./build_packages --board=$BOARD` where `$BOARD` is the code name of the board under test
215 (chroot) `$ /usr/bin/test_that --board=$BOARD $DUT_IP firmware_FAFTSetup`
217 ### Sample Commands {#sample-commands}
221 Running FAFT test with test case name
223 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
227 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
231 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
235 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
239 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
243 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
247 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
251 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
255 - `$ sudo servod --board=$BOARD --port $port_number --serial $servo_serial_number`
256 - `$ /usr/bin/test_that --autotest_dir ~/trunk/src/third_party/autotest/files/ --board=$BOARD $DUT_…
261 [SSH watcher](https://chromium.googlesource.com/chromiumos/platform/dev-util/+/HEAD/contrib/sshwatc…
266 - `$ test_that localhost:2222 --args="servo_host=localhost servo_host_ssh_port=2223 servo_port=9901…
267 - `$ tast run -build=false -var=servo=127.0.0.1:9901:ssh:2223 127.0.0.1:2222 $TESTS`
274 servo=localhost:${LOCAL_SERVO_PORT}:nossh
276 ## Running FAFT on a new kernel {#faft-kernel-next}
278 The lab hosts shown in go/cros-testing-kernelnext provide a static environment
280 sustainable approach of using these DUTs for kernel-next FAFT execution.
282 Local execution via go/faft-running may be required to debug layers of
283 accumulated problems in boards where end-to-end integration tests lack an
284 effective continuous execution. Install a kernelnext image onto the test USB
295 - A1: Run `firmware_FAFTSetup` as a single test. Once it fails, check the log and determine which s…
296 - A2: Check that the servo has all the wired connections and a USB drive with the valid OS plugged …
300 - A1: Re-run the failed tests and try to isolate between flaky infrastructure, an actual firmware b…
301 - A2: See if you were running FAFT without the AC charger connected. The DUT's battery may have co…
305 - A: Try joining the [FAFT-users chromium.org mailing list](https://groups.google.com/a/chromium.or…
307 Q: I got an error while running FAFT: `AutoservRunError: command execution error: sudo -n which fl…
309 - A: Run `sudo emerge chromeos-ec` inside your chroot.
314 - A: This happens when the stateful partition that holds Python is wiped by a
319 Linux distribution would.
323 - A1: Most cases of corruption are triggered by a test performing an EC reset,
327 - A2: If the outer stateful partition (`/mnt/stateful_partition`) becomes full,
328 the inner loop-mounted DM device (`/mnt/stateful_partition/encrypted`)
335 Q: Can I compare the results obtained with a Type-C servo to those obtained with a Type-A servo + m…
337 - A: When running tests with a Type-C servo, it is recommended to to rerun a failure using the Type…
341 - A: The lab is a good source of devices for FAFT per go/cros-testing-kernelnext. If DUTs are not a…
344 …googlesource.com/chromiumos/third_party/hdctools/+/refs/heads/main/README.md#Power-Measurement <br>
348 \[servo v4 Type-C\]: https://chromium.googlesource.com/chromiumos/third_party/hdctools/+/refs/heads…
350 …ium.googlesource.com/chromiumos/third_party/autotest/+/refs/heads/main/docs/faft-design-doc.md <br>
351 …/chromium.googlesource.com/chromiumos/third_party/autotest/+/refs/heads/main/docs/faft-code.md <br>
353 \[test that\]: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/refs/heads/main/…