1- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- 2- LLDB for Renderscript Test Suite 3- 4- 16/03/2016 5- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- 6 7Overview: 8 9 The LLDB for Renderscript test suite is written in python and relies on 10 LLDB's python bindings. The test suite will push several test app's onto 11 a target device, and makes coordinated use of LLDB and ADB to run automated 12 debug sessions. 13 14Set up the test suite: 15 16 Check out the AOSP and build it for your target. Navigate to 17 /frameworks/rs/test/lldb and type mm. This should successfully build the 18 binaries that the testsuite uses. They will be placed in 19 <path to out folder>/target/product/<product code name>/data/app, system/lib 20 and system/bin. 21 22Prerequisite: 23 24 An lldb-server executable must be present on your device/emulator. 25 LLDB must be compiled on your host machine along with its python interface. 26 lldb-server and lldb should be built from the same source revisions. 27 28Running the test suite: 29 30 The test suite can be run via the following command: 31 32 > python run_tests.py [-h] 33 [--config path] 34 [--device DEVICE] 35 [--test path] 36 [--install-only] 37 [--no-install] 38 [--no-uninstall] 39 [--print-to-stdout] 40 [--verbose] 41 [--wimpy] 42 [--run-emu] 43 [--adb-path ADB_PATH] 44 [--aosp-product-path AOSP_PRODUCT_PATH] 45 [--blacklist BLACKLIST [BLACKLIST ...]] 46 [--device-port DEVICE_PORT] 47 [--emu-cmd EMU_CMD] 48 [--host-port HOST_PORT] 49 [--lldb-path LLDB_PATH] 50 [--lldb-server-path-device LLDB_SERVER_PATH_DEVICE] 51 [--lldb-server-path-host LLDB_SERVER_PATH_HOST] 52 [--log-file-path LOG_FILE_PATH] 53 [--results-file-path RESULTS_FILE_PATH] 54 [--timeout TIMEOUT] 55 56 optional arguments: 57 -h, --help show this help message and exit 58 --config path, -c path 59 Path to a custom config file. 60 --device DEVICE, -d DEVICE 61 Specify the device id of the device to test on. 62 --test path, -t path Specify a specific test to run. 63 --install-only It only runs the pre-run stage of the test suite. It 64 installs the required APKs but does not execute the 65 tests. 66 --no-install, -n Stop the test suite installing apks to device. 67 --no-uninstall Stop the test suite uninstalling apks after 68 completion. 69 --print-to-stdout Print all logging information to standard out. 70 --verbose, -v Store extra info in the log. 71 --wimpy, -w Test only a core subset of features. 72 --run-emu Spawn an emulator and run the test suite on that. 73 Specify the emulator command line in the config file 74 or with -emu-cmd. 75 --adb-path ADB_PATH Path to android debug bridge on the host. 76 --aosp-product-path AOSP_PRODUCT_PATH 77 The path to the "out" folder of the AOSP repository. 78 --blacklist BLACKLIST [BLACKLIST ...] 79 Provide a test blacklist for skipping specific tests. 80 To specify the blacklist from the command line the 81 following can be used: --blacklist test1.py test2.py 82 ... 83 --device-port DEVICE_PORT 84 Specify the port number that lldb-server (on the 85 device) listens on. When lldb-server is spawned on the 86 device it will listen on this port. Each successive 87 test will increment onwards from this port. 88 --emu-cmd EMU_CMD The command line for the emulator (if using -run-emu). 89 --host-port HOST_PORT 90 Specify host port which lldb-server will be forwarded 91 to. Specify the starting host port number that lldb- 92 server (on the target) will be forwarded to on the 93 host. Each successive test will increment onwards from 94 this initial port. 95 --lldb-path LLDB_PATH 96 The path to lldb executable on the host. 97 --lldb-server-path-device LLDB_SERVER_PATH_DEVICE 98 Path to the lldb-server executable on the device. 99 --lldb-server-path-host LLDB_SERVER_PATH_HOST 100 Path to the lldb-server executable on host (if using 101 -run-emu). 102 --log-file-path LOG_FILE_PATH 103 The path to the file where the log will be written. 104 --results-file-path RESULTS_FILE_PATH 105 The path to the file where junit results.xml will be 106 written. 107 --timeout TIMEOUT Timeout period for a single command, expressed in 108 seconds 109 110 An optional config file can be passed to the test suite which will provide 111 details of your specific environment. The user file should define a custom 112 class inheriting from Config in config.py. The class Config presents the 113 default set of options, available to be overridden. 114 115 All options in the config file can also be specified on the command line. 116 117 If your config and command line do not specify a path to the host lldb, 118 the PYTHONPATH environment variable must be set. The appropriate value to 119 set this to can be obtained by running the following command: 120 121 > lldb -P 122 123 This will print out a path to the lldb python bindings on your local machine. 124 125Build Requirements: 126 127 128 The following revisions are from the llvm git mirror: 129 130 llvm : 5786b73 131 clang: b6d0b32 132 lldb : 98712eb 133 134 lldb has the following dependencies: 135 136 Python2.7.6 137 swig2.0 138 lldb-server 139 140Building LLDB python bindings: 141 142 Linux: 143 144 Build instructions for Linux lldb can be found on the official lldb web 145 page: 146 147 http://lldb.llvm.org/build.html 148 149 The following CMake variables should be enabled when generating: 150 151 LLDB_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION = True 152 153 As a post build step, swig will generate the python bindings for lldb. 154 155 Windows: 156 157 Prerequisites: 158 159 Visual Studio 2015 160 Custom x64 Python2.7.10 package: 161 http://p-nand-q.com/python/2015.08.07-Python2710-x64-vs2015.7z 162 Swig Version 3.0.5 (Added to the $PATH) 163 Cmake 3.4.0 (So we can generate Visual Studio 2015 solutions) 164 165 Build Python for Windows: 166 167 http://p-nand-q.com/python/building-python-27-with-vs2010.html 168 169 The important thing here is that the above python distribution 170 contains debug versions of the libraries and is built with the same 171 Visual Studio version we are using so the runtimes do not conflict. 172 173 Build LLDB on Windows: 174 175 Select the Cmake generator "Microsoft Visual Studio 2015 Win64". 176 The following CMake variables should be enabled when generating: 177 178 LLDB_DISABLE_PYTHON=False 179 LLDB_RELOCATABLE_PYTHON=False 180 PYTHON_HOME=<path to the above python release> 181 182 Using cmake-gui is a good idea and lets you make sure that swig has 183 been correctly detected. 184 185 In the CMake configure step, you should see something similar to the 186 following: 187 -- Found Python version 2.7.9 188 -- LLDB Found PythonExecutable: E:/Python27/python.exe and 189 E:/Python27/python_d.exe 190 -- LLDB Found PythonLibs: E:/Python27/libs/python27.lib and 191 E:/Python27/libs/python27_d.lib 192 -- LLDB Found PythonDLL: E:/Python27/python27.dll and 193 E:/Python27/python27_d.dll 194 -- LLDB Found PythonIncludeDirs: E:/Python27/Include 195 LLDB version: 3.8.0 196 Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 197 Found PythonInterp: 198 $<$<CONFIG:Debug>:E:/Python27/python_d.exe>$<$<NOT:$<CONFIG:Debug>>:E:/Python27/python.exe> 199 (found version "1.4") 200 201 Build LLDB as a RelWithDebInfo build, because debug builds of lldb 202 produce an lldb_d.pyd file, but the __init__.py still refers to 203 lldb.pyd. 204 205 The LLDB python bindings should have built in 206 "llvm_build\RelWithDebInfo\lib\site-packages\lldb". Point the test 207 suite towards "llvm_build\RelWithDebInfo\bin\lldb.exe". 208 When running the test suite itself, make sure to use the python 209 executable from the custom package. 210 211A typical test transcript: 212 213 Located ADB 214 Located device ZX1G427S2S 215 Pushing all tests... 216 Pushed all tests 217 Pre run complete 218 Found 86 tests 219 Running test_allocation_dump_2_cpp.py: PASS 220 Running test_breakpoint_fileline_multiple_rs_files_cpp.py: PASS 221 Running test_read_local_cpp.py: PASS 222 Running test_breakpoint_kernel_multiple_rs_files_cpp.py: PASS 223 Running test_multiple_rs_files_cpp.py: PASS 224 Running test_breakpoint_kernel_all_multiple_rs_files_cpp.py: PASS 225 Running test_dwarf_lang_cpp.py: PASS 226 Running test_write_global_element_cpp.py: PASS 227 Running test_allocation_list_cpp.py: PASS 228 Running test_breakpoint_coordinate_cpp.py: PASS 229 Running test_rs_consts_cpp.py: PASS 230 Running test_allocation_file_cpp.py: PASS 231 Running test_allocation_dump_1_cpp.py: PASS 232 Running test_source_step_cpp.py: PASS 233 Running test_breakpoint_kernel_2_cpp.py: PASS 234 Running test_backtrace_cpp.py: PASS 235 Running test_language_subcmds_no_debug_cpp.py: PASS 236 Running test_breakpoint_kernel_1_cpp.py: PASS 237 Running test_language_subcmds_cpp.py: PASS 238 Running test_write_global_cpp.py: PASS 239 Running test_invoke_fun_cpp.py: PASS 240 Running test_breakpoint_fileline_cpp.py: PASS 241 Running test_write_local_cpp.py: PASS 242 Running test_breakpoint_kernel_all_cpp.py: PASS 243 Running test_write_local_element_cpp.py: PASS 244 Running test_call_api_funs_cpp.py: PASS 245 Running test_coordinates_cpp.py: PASS 246 Running test_read_global_cpp.py: PASS 247 Running test_language_subcmds.py: PASS 248 Running test_coordinates.py: PASS 249 Running test_language_subcmds_no_debug.py: PASS 250 Running test_read_local.py: PASS 251 Running test_call_api_funs.py: PASS 252 Running test_breakpoint_kernel_1.py: PASS 253 Running test_breakpoint_fileline.py: PASS 254 Running test_breakpoint_fileline_multiple_rs_files.py: PASS 255 Running test_rs_consts.py: PASS 256 Running test_invoke_fun.py: PASS 257 Running test_write_local_element.py: PASS 258 Running test_source_step.py: PASS 259 Running test_allocation_file.py: PASS 260 Running test_allocation_list.py: PASS 261 Running test_breakpoint_kernel_multiple_rs_files.py: PASS 262 Running test_allocation_dump_1.py: PASS 263 Running test_breakpoint_kernel_all.py: PASS 264 Running test_allocation_dump_2.py: PASS 265 Running test_allocation_dump_struct.py: PASS 266 Running test_read_global.py: PASS 267 Running test_language.py: PASS 268 Running test_dwarf_lang.py: PASS 269 Running test_breakpoint_coordinate.py: PASS 270 Running test_write_global.py: PASS 271 Running test_multiple_rs_files.py: PASS 272 Running test_write_global_element.py: PASS 273 Running test_breakpoint_kernel_all_multiple_rs_files.py: PASS 274 Running test_breakpoint_kernel_2.py: PASS 275 Running test_write_local.py: PASS 276 Running test_backtrace.py: PASS 277 Running test_call_api_funs_jni.py: PASS 278 Running test_invoke_fun_jni.py: PASS 279 Running test_allocation_dump_1_jni.py: PASS 280 Running test_breakpoint_fileline_multiple_rs_files_jni.py: PASS 281 Running test_allocation_file_jni.py: PASS 282 Running test_breakpoint_fileline_jni.py: PASS 283 Running test_source_step_jni.py: PASS 284 Running test_coordinates_jni.py: PASS 285 Running test_rs_consts_jni.py: PASS 286 Running test_breakpoint_kernel_all_multiple_rs_files_jni.py: PASS 287 Running test_multiple_rs_files_jni.py: PASS 288 Running test_allocation_dump_2_jni.py: PASS 289 Running test_allocation_list_jni.py: PASS 290 Running test_write_local_element_jni.py: PASS 291 Running test_breakpoint_kernel_all_jni.py: PASS 292 Running test_breakpoint_coordinate_jni.py: PASS 293 Running test_language_subcmds_no_debug_jni.py: PASS 294 Running test_read_local_jni.py: PASS 295 Running test_dwarf_lang_jni.py: PASS 296 Running test_breakpoint_kernel_2_jni.py: PASS 297 Running test_breakpoint_kernel_multiple_rs_files_jni.py: PASS 298 Running test_write_global_element_jni.py: PASS 299 Running test_breakpoint_kernel_1_jni.py: PASS 300 Running test_read_global_jni.py: PASS 301 Running test_language_subcmds_jni.py: PASS 302 Running test_write_global_jni.py: PASS 303 Running test_backtrace_jni.py: PASS 304 Running test_write_local_jni.py: PASS 305 Uninstalled/Deleted all tests 306 86 of 86 passed 307 100% rate 308