1# WALT Setup 2 3WALT (**W**hat an **A**wesome **L**atency **T**ester) is a tool for testing latency on Android devices. 4This document will show you how to set up and use your WALT. 5 6About WALT: 7 8 * Source code: https://github.com/google/walt 9 * [Youtube playlist](https://www.youtube.com/playlist?list=PLd6Fi7WgXfcCEJg1FDqNCoQfpWo7W3J5a) 10 * [Android Developers blog post](http://android-developers.blogspot.com/2016/04/a-new-method-to-measure-touch-and-audio.html) 11 12## Prerequisites 13 14WALT kit: 15 * Assembled WALT PCB and base, with DIP socket for Teensy LC 16 * [Teensy LC](http://www.pjrc.com/teensy/teensyLC.html) inserted into DIP socket 17 * 3.5mm male-to-male TRRS cable 18 ([like this](http://www.infinitecables.com/audio-and-pro-audio-cables/3.5mm-cables/3.5mm-male-to-3.5mm-male-4c/3.5mm-male-to-3.5mm-male-4c-black/2ft-3.5mm-4c-male-to-male-28awg-ft4-black/)) 19 * Pen modified to replace the top button by a male TS 20 * Optional: laser pointer for measuring touch drag latency 21 22If anything is missing from the list above, please contact the person from whom you received your WALT. 23 24A test device and cables: 25 * Android device with USB micro-B female or type C female and a 3.5mm headset jack 26 * USB OTG (On-The-Go) adapter for your Android device (Examples: [A female to C male](https://store.google.com/product/usb_type_c_to_usb_standard_a_adapter), [A female to micro-B male](https://www.amazon.com/Cable-Matters-Micro-USB-Adapter-6-Inch/dp/B00GM0OZ4O)) 27 * USB [Micro-B male to A male](http://www.staples.com/Staples-6-USB-to-Micro-B-Cable-18809-/product_837384) (the usual one we used for everything before type-C) 28 * Laser pointer, if not supplied in the kit 29 30 31 32 33## Setup 34 35 * Install and run the [WALT Android App](https://play.google.com/store/apps/details?id=org.kamrik.latency.walt&hl=en) 36 * Connect WALT to the Android headset using the 3.5mm male to male TRRS cable 37 * Connect WALT to the Android USB port using the cables above. (see photo at top of doc) 38 * You may be prompted to grant the WALT app permission to use the USB, tap "OK" 39 * All 3 LEDs will light up when WALT receives power (see photo below), they will be turned off once the app synchronizes clocks with WALT. 40 41 42## Programming the WALT firmware 43 441. Launch the app, the main screen is pictured below. 451. Verify you have the [latest version](https://github.com/google/walt/blob/master/android/WALT/app/build.gradle#L13) installed by tapping on "View log" 461. Tap "Clock sync" on main menu. 471. Press the Teensy reset button (see photo below) 481. Tap "Reprogram WALT" 491. At this stage you will see several USB permission dialogs. Confirm them all by pressing OK. This is due to WALT presenting itself as different USB devices during reprogramming and normal operation 501. All 3 LEDs will light briefly (see photo below) 511. Tap "Reconnect" 521. Verify that the WALT device has been programmed correctly by running one of the tests below 53 54 55 56 57 58Below is an example of WALT app log when it starts and successfully connects 59to a WALT device. Note the first line and make sure that you use the latest 60version. The last line means that the WALT device clock is behind the Android 61clock but by no more than 161 microseconds. Values about about 1000 us are a 62sign of clock synchronization problems. 63 64 65``` 66WALT v0.1.5 (versionCode=6) 67WALT protocol version 4 68DEVICE INFO: 69 google/volantis/flounder:6.0.1/MMB29K/2419427:user/release-keys 70 Build.SDK_INT=23 71 os.version=3.10.40-ga54a4f4 72Requesting permission for USB device. 73Interface claimed successfully 74 75Synced clocks, maxE=161us 76``` 77 78# Running the latency tests 79 80This section describes how to run each test and interpret the results. 81 82 83## Audio input latency (microphone) 84 85The Teensy will generate a pulse of audio data which will be received at the 3.5mm headset jack. The time delta between the audio pulse being generated and being received by the WALT application is measured. 86 87 * Tap "Audio latency" 88 * Tap the microphone icon in the top left 89 * Wait for results 90 91 92Below is the output of a test on Nexus 9. The last two lines show the 6 individual measurements and their median. The time measured here is the time that passed from the recording of the _last_ frame in the buffer and until the software callback fired. The length of the optimal buffer in time units can be calculated using the first two lines: 128 frames divided by 48kHz frame rate is about 2.67 ms. 93 94 95``` 96Optimal frame rate is: 48000 97Optimal frames per buffer is: 128 98Audio engine created 99Audio recorder created; starting test 100Synced clocks, maxE=154us 101Beeped, reply: 1039241 102Processed: L_cb = 1.167 ms, L_eq = 171.558 ms, noisy frame = 16799 103... 104Beeped, reply: 13621259 105Processed: L_cb = 0.907 ms, L_eq = 203.991 ms, noisy frame = 14465 106Remote clock delayed between -30 and 892 us 107deltas: [1.1671666666666278, 1.1871666666666278, 0.984, 0.9981666666666279, 1.7278333333333722, 0.9071666666666278] 108Median audio recording latency 1.1 ms 109``` 110 111## Audio output latency 112 113The WALT app will generate a pulse of audio data which will be transmitted over the 3.5mm headset jack and received by the Teensy. The time delta between audio being transmitted by the app and received by the Teensy is measured 114 115 * Set volume to maximum in Settings -> Sound 116 * Tap "Audio latency" 117 * Tap the play button ▶ in the top right 118 * Results are displayed in the log window, example below. 119 120``` 121Beeping... 122Beeped, dtJ2N = 0.059 ms 123beep detected, total latency = 10.44, normal latency = 10.09, mInitiatedBeeps = 1, mDetectedBeeps = 1 124... 125Remote clock delayed between -53 and 349 us 126deltas: [10.439, 12.272, 11.708, 12.194, 12.919, 11.458, 12.985, 10.914, 10.935, 10.631] 127Median Java to native latency 0.006 ms 128Median total audio latency 11.6 ms 129Median callback to output time 9.9 ms 130``` 131 132The callback to output time on the last line, 9.9 ms is the time from the execution of the 133[OpenSLES BufferQueue callback that enqueued the buffer](https://github.com/google/walt/blob/master/android/WALT/app/src/main/jni/player.c#L107) 134until the first frames of that buffer were detected by WALT device via the headphone jack. 135 136 137## Screen response latency 138 139 * Tap screen response 140 * Set screen brightness to maximum. On phones using PWM for screen backlight brightness control, yo man need to turn adaptive brightness off (settings -> Display -> Adaptive Brightness) 141 * Position WALT with the screen sensor looking at the screen (the light sensor recessed into the large plastic surface). 142 * Click start ▶ 143 * The screen will blink some 20 times. The orange LED on the Teensy will also blink every time the bottom light sensor detects a change. 144 * Results are displayed in the log window 145 146``` 147Starting screen response measurement 148Synced clocks, maxE=162us 149Blink count was: 0 150Synced clocks, maxE=857us 151Starting Listener 152====== 153doBlink.run(), mInitiatedBlinks = 0 mDetectedBlinks = 0 154blink counts 1 1 155====== 156... 157====== 158doBlink.run(), mInitiatedBlinks = 20 mDetectedBlinks = 20 159Stopping Listener 160Listener stopped 161Remote clock delayed between -16 and 628 us 162deltas: [69.908, 53.902, 54.715, 50.867, 70.73, 50.188, 71.344, 85.259, 40.691, 68.554, 72.016, 51.666, 71.826, 51.234, 71.896, 52.131, 68.943, 51.768, 71.919, 48.812] 163Median latency 61.6 ms 164``` 165 166At the time of this writing (WALT v0.1.5) the number reported on the last line 167of the log above is the time from a call to View.setBackgroundColor() until 168the measured screen brightness crosses a threshold 169[hard coded in WALT firmware](https://github.com/google/walt/blob/master/arduino/walt/walt.ino#L70). 170This has significant drawbacks and will change in future versions. 171 172 173 174## Tap latency 175 176[Video demonstration](https://www.youtube.com/watch?v=1xAReF75Cts&list=PLd6Fi7WgXfcCEJg1FDqNCoQfpWo7W3J5a&index=2) 177 178 * Disconnect the audio cable 179 * Insert the supplied stylus (pen with audio jack) into the WALT's audio connector 180 * Click "Tap Latency" 181 * Tap the black part of the screen with WALT's copper tape tip while holding by the pen (photo below). Repeat some 20-30 times. Try to keep WALT pressed against the screen for a bit longer than you would usually keep the finger on the glass during a momentary tap. 182 * Click the checkmark button (top left of the screen), results will be calculated and displayed 183 184WALT has an accelerometer that is used to detect when the copper tip collides with the glass. The copper tip is grounded and triggers the touch sensor. The stylus springy action helps generate a collision shock detectable by accelerometer on the way back, away from the glass. 185 186 187 188In the log below the following abbreviations are used: 189 - p2k - physical to kernel - in most cases this is the interesting time. 190 - k2c - kernel to (java) callback 191 192 193``` 194## Restarting tap latency measurement. Re-sync clocks ... 195Synced clocks, maxE=158us 196 197... 198 199ACTION_DOWN event received: Event: t=10269422 x=871.7 y=1026.1 slot=-1 num=0 ACTION_DOWN 200Received S reply: 10247405 201 202ACTION_DOWN: dt_p2k = 22.0 ms 203Ignoring ACTION_MOVE 11 204Ignoring ACTION_MOVE 12 205Ignoring ACTION_MOVE 13 206Ignoring ACTION_MOVE 14 207Ignoring ACTION_MOVE 15 208 209ACTION_UP event received: Event: t=10365071 x=871.7 y=1026.1 slot=-1 num=0 ACTION_UP 210Received S reply: 10348526 211 212ACTION_UP: dt_p2k = 16.5 ms 213 214... 215 216## Processing tap latency data 217Remote clock delayed between -60 and 93 us 218Counts: ACTION_DOWN 20 (bad 1), ACTION_UP 21 (bad 0), ACTION_MOVE 162 219 220ACTION_DOWN: 221[26.515, 22.017, 26.952, 29.363, 29.63, 26.624, 31.581, 26.565, 31.335, 26.83, 31.786, 14.702, 19.796, 20.626, 16.688, 18.001, 19.044, 20.261, 17.66, 26.044] 222[1.774, 0.872, 0.888, 0.948, 1.749, 1.781, 1.698, 1.375, 1.709, 1.985, 2.522, 0.994, 1.008, 0.813, 0.746, 1.289, 1.158, 1.093, 0.891, 3.755] 223Medians, p2k & k2c [ms]: 26.3 1.2 224 225ACTION_UP: 226[16.572, 19.515, 16.545, 18.724, 15.188, 14.682, 18.156, 11.69, 20.367, 15.135, 20.601, 44.948, 17.517, 129.513, 24.681, 21.21, 21.979, 22.805, 133.306, 23.876, 17.579] 227[2.161, 2.02, 1.761, 2.222, 1.228, 1.24, 1.48, 1.42, 1.315, 4.034, 1.279, 1.957, 1.041, 2.058, 2.903, 1.725, 0.99, 2.124, 1.398, 3.018, 0.945] 228Medians, p2k & k2c [ms]: 19.5 1.7 229 230``` 231 232## Drag/scroll latency 233 234Drag latency is measured by dragging a finger up and down along the screen. The finger interrupts a laser beam positioned across the screen. By comparing the timings of those beam interruptions with Android touch events the WALT app calculates the delay. 235 236 237 * Position a laser pointer so that the beam crosses the screen roughly in the middle. A convenient method for positioning the laser pointer is by using modeling clay (plasticine) 238 * Place WALT on the other side so that the beam hits the WALT optical sensor looking sideways. A green LED (marked D3) will light up whenever the beam hits the sensor 239 * Click the start button ▶, a green animated line will start going up and down the screen 240 * Drag your finger on the screen, following the animated line 241 * The green counter in the top right corner counts how many times the laser sensor changed state. Each crossing of the beam add 2 to that counter. Continue until the counter shows at least 60 242 * Click the finish **✓**(check mark) button, results will be displayed in the black window 243 244 245 246# Collecting Systrace logs 247 248WALT is able to log physical events as well as system events into systrace. This allows us to look at the system-level calls that occur between the application triggering a change and the WALT device receiving a physical response. 249 250 * In the WALT app go to Settings->General and check `Enable systrace logging`. 251 * On your workstation, follow the [instructions for systrace](https://developer.android.com/studio/profile/systrace-commandline.html) on the command line 252 * From the `android-sdk/platform-tools/systrace` directory, run `python systrace.py --walt`. If `--walt` is not recognized as an option, clone the latest version of [systrace repo](https://github.com/catapult-project/catapult) and from the `catapult/systrace/systrace` directory run `python run_systrace.py --walt`. 253 * Conduct a test through the WALT app. When the test is complete, you should see that TraceLogger wrote some events to a file. 254 * Press enter on your workstation to end the trace and pull the logs from the phone, and the traceview will be generated as HTML. 255 256 257# Troubleshooting 258 259 * Make sure your micro-USB cable is not power-only (can carry data). If it has a button to switch between data and no-data modes (e.g. the KAA cable), click the button. 260 * Make sure volume control on Android DUT is at maximum (update in settings -> sound while TRRS cable is inserted). 261 * Some devices use pulse width modulation (PWM) to control screen brightness. Disable adaptive screen brightness and set brightness manually to maximum. 262
