1HOWTO - using the library with perf {#howto_perf} 2=================================== 3 4@brief Using command line perf and OpenCSD to collect and decode trace. 5 6This HOWTO explains how to use the perf cmd line tools and the openCSD 7library to collect and extract program flow traces generated by the 8CoreSight IP blocks on a Linux system. The examples have been generated using 9an aarch64 Juno-r0 platform. 10 11Note on building OpenCSD and Kernel/Perf Versions 12------------------------------------------------- 13 14OpenCSD is developed independently of the linux kernel and perf, though the ABI between 15the library and the perf tool generally remains stable. This ABI is formed of a stream of 16decoded packets from the library to the tool. 17 18The exception to this is when new ARM architecture results in significant changes to trace 19elements and therefore to the decoder. This can then change the interface between the OpenCSD 20library and client program i.e. `perf`, introducing new packet types. When this occurs, patches 21for perf are submitted to the kernel at the same time as the decoder is updated. 22 23The general advice is to always use the latest versions of OpenCSD and kernel/perf. 24 25However the following points should be noted if this is not possible: 26 27- perf build enforces a minimum version of OpenCSD that it will compile against. 28 29- building perf from an older kernel against an OpenCSD version later than the minimum 30 version will generally work, unless the version of OpenCSD has packet types that are 31 not supported in the cs_etm_decoder handling code in perf. This may result in a build 32 error such as enum types not supported. 33 34- The minimum version of kernel recommended is 5.16. However there is some limited support 35 for older kernels at present in the form of backports at:- 36 37 <https://gitlab.arm.com/linux-arm/linux-coresight-backports> 38 39 See the auto-fdo documentation (@ref older_kernels) for more information on older kernels. 40 41 42On Target Trace Acquisition - Perf Record 43----------------------------------------- 44 45Compile the perf tool from the same kernel source code version you are using with: 46 47 make -C tools/perf 48 49This will yield a `perf` executable that will support CoreSight trace collection. 50 51*Note:* If traces are to be decompressed **off** target, there is no need to download 52and compile the openCSD library (on the target). 53 54If you are instead planning to use perf to record and decode the trace on the target, 55compile the perf tool linking against the openCSD library, in the following way: 56 57 make -C tools/perf VF=1 CORESIGHT=1 58 59Further information on the needed build environments and options are detailed later 60in the section **Off Target Perf Tools Compilation**. 61 62Before launching a trace run a sink that will collect trace data needs to be 63identified. All CoreSight blocks identified by the framework are registed in 64sysFS: 65 66 67 linaro@linaro-nano:~$ ls /sys/bus/coresight/devices/ 68 etm0 etm2 etm4 etm6 funnel0 funnel2 funnel4 stm0 tmc_etr0 69 etm1 etm3 etm5 etm7 funnel1 funnel3 replicator0 tmc_etf0 70 71 72CoreSight blocks are listed in the device tree for a specific system and 73discovered at boot time. Since tracers can be linked to more than one sink, 74the sink that will recieve trace data needs to be identified and given as an 75option on the perf command line. Once a sink has been identify trace collection 76can start. An easy and yet interesting example is the `uname` command: 77 78 linaro@linaro-nano:~/kernel$ ./tools/perf/perf record -e cs_etm/@tmc_etr0/ --per-thread uname 79 80This will generate a `perf.data` file where execution has been traced for both 81user and kernel space. To narrow the field to either user or kernel space the 82`u` and `k` options can be specified. For example the following will limit 83traces to user space: 84 85 86 linaro@linaro-nano:~/kernel$ ./tools/perf/perf record -vvv -e cs_etm/@tmc_etr0/u --per-thread uname 87 Problems setting modules path maps, continuing anyway... 88 ----------------------------------------------------------- 89 perf_event_attr: 90 type 8 91 size 112 92 { sample_period, sample_freq } 1 93 sample_type IP|TID|IDENTIFIER 94 read_format ID 95 disabled 1 96 exclude_kernel 1 97 exclude_hv 1 98 enable_on_exec 1 99 sample_id_all 1 100 ------------------------------------------------------------ 101 sys_perf_event_open: pid 11375 cpu -1 group_fd -1 flags 0x8 102 ------------------------------------------------------------ 103 perf_event_attr: 104 type 1 105 size 112 106 config 0x9 107 { sample_period, sample_freq } 1 108 sample_type IP|TID|IDENTIFIER 109 read_format ID 110 disabled 1 111 exclude_kernel 1 112 exclude_hv 1 113 mmap 1 114 comm 1 115 enable_on_exec 1 116 task 1 117 sample_id_all 1 118 mmap2 1 119 comm_exec 1 120 ------------------------------------------------------------ 121 sys_perf_event_open: pid 11375 cpu -1 group_fd -1 flags 0x8 122 mmap size 266240B 123 AUX area mmap length 131072 124 perf event ring buffer mmapped per thread 125 Synthesizing auxtrace information 126 Linux 127 auxtrace idx 0 old 0 head 0x11ea0 diff 0x11ea0 128 [ perf record: Woken up 1 times to write data ] 129 overlapping maps: 130 7f99daf000-7f99db0000 0 [vdso] 131 7f99d84000-7f99db3000 0 /lib/aarch64-linux-gnu/ld-2.21.so 132 7f99d84000-7f99daf000 0 /lib/aarch64-linux-gnu/ld-2.21.so 133 7f99db0000-7f99db3000 0 /lib/aarch64-linux-gnu/ld-2.21.so 134 failed to write feature 8 135 failed to write feature 9 136 failed to write feature 14 137 [ perf record: Captured and wrote 0.072 MB perf.data ] 138 139 linaro@linaro-nano:~/kernel$ ls -l ~/.debug/ perf.data 140 _-rw------- 1 linaro linaro 77888 Mar 2 20:41 perf.data 141 142 /home/linaro/.debug/: 143 total 16 144 drwxr-xr-x 2 linaro linaro 4096 Mar 2 20:40 [kernel.kallsyms] 145 drwxr-xr-x 2 linaro linaro 4096 Mar 2 20:40 [vdso] 146 drwxr-xr-x 3 linaro linaro 4096 Mar 2 20:40 bin 147 drwxr-xr-x 3 linaro linaro 4096 Mar 2 20:40 lib 148 149Trace data filtering 150-------------------- 151The amount of traces generated by CoreSight tracers is staggering, event for 152the most simple trace scenario. Reducing trace generation to specific areas 153of interest is desirable to save trace buffer space and avoid getting lost in 154the trace data that isn't relevant. Supplementing the 'k' and 'u' options 155described above is the notion of address filters. 156 157On CoreSight two types of address filter have been implemented - address range 158and start/stop filter: 159 160**Address range filters:** 161With address range filters traces are generated if the instruction pointer 162falls within the specified range. Any work done by the CPU outside of that 163range will not be traced. Address range filters can be specified for both 164user and kernel space session: 165 166 perf record -e cs_etm/@tmc_etr0/k --filter 'filter 0xffffff8008562d0c/0x48' --per-thread uname 167 168 perf record -e cs_etm/@tmc_etr0/u --filter 'filter 0x72c/0x40@/opt/lib/libcstest.so.1.0' --per-thread ./main 169 170When dealing with kernel space trace addresses are typically taken in the 171'System.map' file. In user space addresses are relocatable and can be 172extracted from an objdump output: 173 174 $ aarch64-linux-gnu-objdump -d libcstest.so.1.0 175 ... 176 ... 177 000000000000072c <coresight_test1>: <------------ Beginning of traces 178 72c: d10083ff sub sp, sp, #0x20 179 730: b9000fe0 str w0, [sp,#12] 180 734: b9001fff str wzr, [sp,#28] 181 738: 14000007 b 754 <coresight_test1+0x28> 182 73c: b9400fe0 ldr w0, [sp,#12] 183 740: 11000800 add w0, w0, #0x2 184 744: b9000fe0 str w0, [sp,#12] 185 748: b9401fe0 ldr w0, [sp,#28] 186 74c: 11000400 add w0, w0, #0x1 187 750: b9001fe0 str w0, [sp,#28] 188 754: b9401fe0 ldr w0, [sp,#28] 189 758: 7100101f cmp w0, #0x4 190 75c: 54ffff0d b.le 73c <coresight_test1+0x10> 191 760: b9400fe0 ldr w0, [sp,#12] 192 764: 910083ff add sp, sp, #0x20 193 768: d65f03c0 ret 194 ... 195 ... 196 197Following the address the amount of byte is specified and if tracing in user 198space, the full path to the binary (or library) being traced. 199 200**Start/Stop filters:** 201With start/stop filters traces are generated when the instruction pointer is 202equal to the start address. Incidentally traces stop being generated when the 203insruction pointer is equal to the stop address. Anything that happens between 204there to events is traced: 205 206 perf record -e cs_etm/@tmc_etr0/k --filter 'start 0xffffff800856bc50,stop 0xffffff800856bcb0' --per-thread uname 207 208 perf record -vvv -e cs_etm/@tmc_etr0/u --filter 'start 0x72c@/opt/lib/libcstest.so.1.0, \ 209 stop 0x40082c@/home/linaro/main' \ 210 --per-thread ./main 211 212**Limitation on address filters:** 213The only limitation on address filters is the amount of address comparator 214found on an implementation and the mutual exclusion between range and 215start stop filters. As such the following example would _not_ work: 216 217 perf record -e cs_etm/@tmc_etr0/k --filter 'start 0xffffff800856bc50,stop 0xffffff800856bcb0, \ // start/stop 218 filter 0x72c/0x40@/opt/lib/libcstest.so.1.0' \ // address range 219 --per-thread uname 220 221Additional Trace Options 222------------------------ 223Additional options can be used during trace collection that add information to the captured trace. 224 225- Timestamps: These packets are added to the trace streams to allow correlation of different sources where tools support this. 226- Cycle Counts: These packets are added to get a count of cycles for blocks of executed instructions. Adding cycle counts will considerably increase the amount of generated trace. 227The relationship between cycle counts and executed instructions differs according to the trace protocol. 228For example, the ETMv4 protocol will emit counts for groups of instructions according to a minimum count threshold. 229Presently this threshold is fixed at 256 cycles for `perf record`. 230 231Command line options in `perf record` to use these features are part of the options for the `cs_etm` event: 232 233 perf record -e cs_etm/timestamp,cycacc,@tmc_etr0/ --per-thread uname 234 235At current version, `perf record` and `perf script` do not use this additional information. 236 237The cs_etm perf event 238--------------------- 239 240System information for this perf pmu event can be found at: 241 242 /sys/devices/cs_etm 243 244This contains internal format of the parameters described above: 245 246 root@linaro-developer:~# ls /sys/devices/cs_etm/format 247 contextid cycacc retstack sinkid timestamp 248 249and names of registered sinks: 250 251 root@linaro-developer:~# ls /sys/devices/cs_etm/sinks 252 tmc_etf0 tmc_etr0 tpiu0 253 254Note: The `sinkid` parameter is there to document the usage of a 32-bit internal parameter to 255pass the sink name used in the cs_etm/@sink/ command to the kernel drivers. It can be used 256directly as cs_etm/sinkid=<hash_value>/ but this is not recommended as the values used are 257considered opaque and subject to changes. 258 259On Target Trace Collection 260-------------------------- 261The entire program flow will have been recorded in the `perf.data` file. 262Information about libraries and executable is stored under `$HOME/.debug`: 263 264 linaro@linaro-nano:~/kernel$ tree ~/.debug 265 .debug 266 ├── [kernel.kallsyms] 267 │ └── 0542921808098d591a7acba5a1163e8991897669 268 │ └── kallsyms 269 ├── [vdso] 270 │ └── 551fbbe29579eb63be3178a04c16830b8d449769 271 │ └── vdso 272 ├── bin 273 │ └── uname 274 │ └── ed95e81f97c4471fb2ccc21e356b780eb0c92676 275 │ └── elf 276 └── lib 277 └── aarch64-linux-gnu 278 ├── ld-2.21.so 279 │ └── 94912dc5a1dc8c7ef2c4e4649d4b1639b6ebc8b7 280 │ └── elf 281 └── libc-2.21.so 282 └── 169a143e9c40cfd9d09695333e45fd67743cd2d6 283 └── elf 284 285 13 directories, 5 files 286 linaro@linaro-nano:~/kernel$ 287 288 289All this information needs to be collected in order to successfully decode 290traces off target: 291 292 linaro@linaro-nano:~/kernel$ tar czf results.trace.tgz perf.data ~/.debug 293 294 295Note that file `vmlinux` should also be added to the bundle if kernel traces 296have also been collected. 297 298This `results.trace.tgz` can then be transferred to the host for decoding. 299 300Off Target OpenCSD Compilation 301------------------------------ 302The openCSD library is not part of the perf tools. It is available on 303[github][1] and needs to be compiled before the perf tools. Checkout the 304required branch/tag version into a local directory. 305 306 linaro@t430:~/linaro/coresight$ git clone https://github.com/Linaro/OpenCSD.git my-opencsd 307 Cloning into 'OpenCSD'... 308 remote: Counting objects: 2063, done. 309 remote: Total 2063 (delta 0), reused 0 (delta 0), pack-reused 2063 310 Receiving objects: 100% (2063/2063), 2.51 MiB | 1.24 MiB/s, done. 311 Resolving deltas: 100% (1399/1399), done. 312 Checking connectivity... done. 313 linaro@t430:~/linaro/coresight$ ls my-opencsd 314 decoder LICENSE README.md HOWTO.md TODO 315 316Once the source code has been acquired compilation of the openCSD library can 317take place. By default this will build for the current host into the `./decoder/lib/builddir` 318directory. 319 320 linaro@t430:~/linaro/coresight$ cd my-opencsd 321 linaro@t430:~/linaro/coresight/my-opencsd$ ls ./decoder/build/linux 322 makefile makefile.dev rctdl_c_api_lib ref_trace_decode_lib 323 324 linaro@t430:~/linaro/coresight/my-opencsd$ make -C decoder/build/linux 325 ... 326 ... 327 328 linaro@t430:~/linaro/coresight/my-opencsd$ ls ./decoder/lib/builddir 329 libopencsd.a libopencsd_c_api.so libopencsd_c_api.so.1.5.5 libopencsd.so.1 libopencsd_c_api.a 330 libopencsd_c_api.so.1 libopencsd.so libopencsd.so.1.5.5 331 332From there the header file and libraries need to be installed on the system, 333something that requires root privileges. The default installation path is 334/usr/include/opencsd for the header files and /usr/lib/ for the libraries: 335 336 linaro@t430:~/linaro/coresight/my-opencsd$ sudo make -C decoder/build/linux install 337 linaro@t430:~/linaro/coresight/my-opencsd$ ls -l /usr/include/opencsd 338 total 60 339 drwxr-xr-x 2 root root 4096 Feb 25 13:51 c_api 340 drwxr-xr-x 2 root root 4096 Feb 25 13:51 ete 341 drwxr-xr-x 2 root root 4096 Feb 25 13:51 etmv3 342 drwxr-xr-x 2 root root 4096 Feb 25 13:51 etmv4 343 -rw-r--r-- 1 root root 34668 Feb 25 13:51 ocsd_if_types.h 344 -rw-r--r-- 1 root root 2561 Feb 25 13:51 ocsd_if_version.h 345 drwxr-xr-x 2 root root 4096 Feb 25 13:51 ptm 346 drwxr-xr-x 2 root root 4096 Feb 25 13:51 stm 347 -rw-r--r-- 1 root root 10452 Feb 25 13:51 trc_gen_elem_types.h 348 -rw-r--r-- 1 root root 3972 Feb 25 13:51 trc_pkt_types.h 349 350 linaro@t430:~/linaro/coresight/my-opencsd$ ls -al /usr/lib/libopencsd* 351 352 -rw-r--r-- 1 root root 1709380 Feb 25 13:51 /usr/lib/libopencsd.a 353 -rw-r--r-- 1 root root 152314 Feb 25 13:51 /usr/lib/libopencsd_c_api.a 354 lrwxrwxrwx 1 root root 21 Feb 25 13:51 /usr/lib/libopencsd_c_api.so -> libopencsd_c_api.so.1 355 lrwxrwxrwx 1 root root 25 Feb 25 13:51 /usr/lib/libopencsd_c_api.so.1 -> libopencsd_c_api.so.1.5.5 356 -rw-r--r-- 1 root root 98816 Feb 25 13:51 /usr/lib/libopencsd_c_api.so.1.5.5 357 lrwxrwxrwx 1 root root 15 Feb 25 13:51 /usr/lib/libopencsd.so -> libopencsd.so.1 358 lrwxrwxrwx 1 root root 19 Feb 25 13:51 /usr/lib/libopencsd.so.1 -> libopencsd.so.1.5.5 359 -rw-r--r-- 1 root root 797680 Feb 25 13:51 /usr/lib/libopencsd.so.1.5.5 360 361 362A "clean_install" target is also available so that openCSD installed files can 363be removed from a system. Going forward the goal is to have the openCSD library 364packaged as a Debian or RPM archive so that it can be installed from a 365distribution without having to be compiled. 366 367Complete information on building and installing the library can be found in build_libs.md (@ref build_lib) 368 369Off Target Perf Tools Compilation 370--------------------------------- 371 372As mentioned above the openCSD library is not part of the perf tools' code base 373and needs to be installed on a system prior to compilation. Information about 374the status of the openCSD library on a system is given at compile time by the 375perf tools build script: 376 377 linaro@t430:~/linaro/linux-kernel$ make CORESIGHT=1 VF=1 -C tools/perf 378 Auto-detecting system features: 379 ... dwarf: [ on ] 380 ... dwarf_getlocations: [ on ] 381 ... glibc: [ on ] 382 ... gtk2: [ on ] 383 ... libaudit: [ on ] 384 ... libbfd: [ OFF ] 385 ... libelf: [ on ] 386 ... libnuma: [ OFF ] 387 ... numa_num_possible_cpus: [ OFF ] 388 ... libperl: [ on ] 389 ... libpython: [ on ] 390 ... libslang: [ on ] 391 ... libcrypto: [ on ] 392 ... libunwind: [ OFF ] 393 ... libdw-dwarf-unwind: [ on ] 394 ... zlib: [ on ] 395 ... lzma: [ OFF ] 396 ... get_cpuid: [ on ] 397 ... bpf: [ on ] 398 ... libopencsd: [ on ] <------- 399 400 401If libopencsd is not installed, or the version is too old for correct compilation, 402then an error will be issued and compilation will fail. 403 404At the end of the compilation a new perf binary is available in `tools/perf/`, 405which we can verify is linked to `libopencsd`: 406 407 linaro@t430:~/linaro/linux-kernel$ ldd tools/perf/perf 408 linux-vdso.so.1 => (0x00007fff135db000) 409 libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f15f9176000) 410 librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f15f8f6e000) 411 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f15f8c64000) 412 libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f15f8a60000) 413 libopencsd_c_api.so.1 => /usr/lib/libopencsd_c_api.so.1 (0x00007f5645cfb000) <------- 414 libelf.so.1 => /usr/lib/x86_64-linux-gnu/libelf.so.1 (0x00007f15f8635000) 415 libdw.so.1 => /usr/lib/x86_64-linux-gnu/libdw.so.1 (0x00007f15f83ec000) 416 libaudit.so.1 => /lib/x86_64-linux-gnu/libaudit.so.1 (0x00007f15f81c5000) 417 libslang.so.2 => /lib/x86_64-linux-gnu/libslang.so.2 (0x00007f15f7e38000) 418 libperl.so.5.22 => /usr/lib/x86_64-linux-gnu/libperl.so.5.22 (0x00007f15f7a5d000) 419 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f15f7693000) 420 libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f15f7104000) 421 libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f15f6eea000) 422 /lib64/ld-linux-x86-64.so.2 (0x0000559b88038000) 423 424Additional debug output from the decoder can be compiled in by setting the 425`CSTRACE_RAW` environment variable at build time. Setting this to `packed` gets trace frame 426output as follows:- 427 428 Frame Data; Index 576; RAW_PACKED; d6 d6 d6 d6 d6 d6 d6 d6 fc fb d6 d6 d6 d6 e0 7f 429 Frame Data; Index 576; ID_DATA[0x14]; d7 d6 d7 d6 d7 d6 d7 d6 fd fb d7 d6 d7 d6 e0 430 431Set to any other value will remove the RAW_PACKED lines. 432 433Working with an alternate version of the openCSD library 434-------------------------------------------------------- 435When compiling the perf tools it is possible to reference another version of 436the openCSD library than the one installed on the system. This is useful when 437working with multiple development trees or having the desire to keep system 438libraries intact. Two environment variable are available to tell the perf tools 439build script where to get the header file and libraries, namely CSINCLUDES and 440CSLIBS: 441 442 linaro@t430:~/linaro/linux-kernel$ export CSINCLUDES=~/linaro/coresight/my-opencsd/decoder/include/ 443 linaro@t430:~/linaro/linux-kernel$ export CSLIBS=~/linaro/coresight/my-opencsd/decoder/lib/builddir/ 444 linaro@t430:~/linaro/linux-kernel$ make CORESIGHT=1 VF=1 -C tools/perf 445 446This will have the effect of compiling and linking against the provided library. 447Since the system's openCSD library is in the loader's search patch the 448LD_LIBRARY_PATH environment variable needs to be set. 449 450 linaro@t430:~/linaro/linux-kernel$ export LD_LIBRARY_PATH=$CSLIBS 451 452 453Trace Decoding with Perf Report 454------------------------------- 455Trace bundles can be extracted anywhere and have no dependencies on where the perf tools and openCSD library have been compiled. 456Take the results file from the target and extract into a working directory (e.g. the `results.trace.tgz` file from above) 457 458 linaro@t430:~/linaro/coresight$ mkdir trace-results 459 linaro@t430:~/linaro/coresight$ cd trace-results 460 linaro@t430:~/linaro/coresight/trace-results$ scp <path/to/target>/~/kernel/results.trace.tgz . 461 linaro@t430:~/linaro/coresight/trace-results$ tar xf results.trace.tgz 462 linaro@t430:~/linaro/coresight/trace-results$ ls -la 463 total 1312 464 drwxrwxr-x 3 linaro linaro 4096 Mar 3 10:26 . 465 drwxrwxr-x 5 linaro linaro 4096 Mar 3 10:13 .. 466 drwxr-xr-x 7 linaro linaro 4096 Feb 24 12:21 .debug 467 -rw------- 1 linaro linaro 78016 Feb 24 12:21 perf.data 468 -rw-rw-r-- 1 linaro linaro 1245881 Feb 24 12:25 results.trace.tgz 469 470Perf is expecting files related to the trace capture (`perf.data`) to be located in the `buildid` directory. 471By default this is under `~/.debug`. Alternatively the default `buildid` directory can be changed 472using the command: 473 474 perf config --system buildid.dir=/my/own/buildid/dir 475 476This example will remove the current `~/.debug` directory to be sure everything is clean. 477 478 linaro@t430:~/linaro/coresight/trace-results$ rm -rf ~/.debug 479 linaro@t430:~/linaro/coresight/trace-results$ cp -dpR .debug ~/ 480 linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf report --stdio 481 482 # To display the perf.data header info, please use --header/--header-only options. 483 # 484 # 485 # Total Lost Samples: 0 486 # 487 # Samples: 0 of event 'cs_etm//u' 488 # Event count (approx.): 0 489 # 490 # Children Self Command Shared Object Symbol 491 # ........ ........ ....... ............. ...... 492 # 493 494 495 # Samples: 0 of event 'dummy:u' 496 # Event count (approx.): 0 497 # 498 # Children Self Command Shared Object Symbol 499 # ........ ........ ....... ............. ...... 500 # 501 502 503 # Samples: 115K of event 'instructions:u' 504 # Event count (approx.): 522009 505 # 506 # Children Self Command Shared Object Symbol 507 # ........ ........ ....... ................ ...................... 508 # 509 4.13% 4.13% uname libc-2.21.so [.] 0x0000000000078758 510 3.81% 3.81% uname libc-2.21.so [.] 0x0000000000078e50 511 2.06% 2.06% uname libc-2.21.so [.] 0x00000000000fcaf4 512 1.65% 1.65% uname libc-2.21.so [.] 0x00000000000fcae4 513 1.59% 1.59% uname ld-2.21.so [.] 0x000000000000a7f4 514 1.50% 1.50% uname libc-2.21.so [.] 0x0000000000078e40 515 1.43% 1.43% uname libc-2.21.so [.] 0x00000000000fcac4 516 1.31% 1.31% uname libc-2.21.so [.] 0x000000000002f0c0 517 1.26% 1.26% uname ld-2.21.so [.] 0x0000000000016888 518 1.24% 1.24% uname libc-2.21.so [.] 0x0000000000078e7c 519 1.24% 1.24% uname libc-2.21.so [.] 0x00000000000fcab8 520 ... 521 522Additional data can be obtained, which contains a dump of the trace packets received using the command 523 524 mjl@ubuntu-vbox:./perf-opencsd-master/coresight/tools/perf/perf report --stdio --dump 525 526resulting a large amount of data, trace looking like:- 527 528 0x618 [0x30]: PERF_RECORD_AUXTRACE size: 0x11ef0 offset: 0 ref: 0x4d881c1f13216016 idx: 0 tid: 15244 cpu: -1 529 530 . ... CoreSight ETM Trace data: size 73456 bytes 531 532 0: I_ASYNC : Alignment Synchronisation. 533 12: I_TRACE_INFO : Trace Info. 534 17: I_TRACE_ON : Trace On. 535 18: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000007F89F24D80; Ctxt: AArch64,EL0, NS; 536 28: I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE 537 29: I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE 538 30: I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE 539 32: I_ATOM_F6 : Atom format 6.; EEEEN 540 33: I_ATOM_F1 : Atom format 1.; E 541 34: I_EXCEPT : Exception.; Data Fault; Ret Addr Follows; 542 36: I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000007F89F2832C; 543 45: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0xFFFFFFC000083400; Ctxt: AArch64,EL1, NS; 544 56: I_TRACE_ON : Trace On. 545 57: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000007F89F2832C; Ctxt: AArch64,EL0, NS; 546 68: I_ATOM_F3 : Atom format 3.; NEE 547 69: I_ATOM_F3 : Atom format 3.; NEN 548 70: I_ATOM_F3 : Atom format 3.; NNE 549 71: I_ATOM_F5 : Atom format 5.; ENENE 550 72: I_ATOM_F5 : Atom format 5.; NENEN 551 73: I_ATOM_F5 : Atom format 5.; ENENE 552 74: I_ATOM_F5 : Atom format 5.; NENEN 553 75: I_ATOM_F5 : Atom format 5.; ENENE 554 76: I_ATOM_F3 : Atom format 3.; NNE 555 77: I_ATOM_F3 : Atom format 3.; NNE 556 78: I_ATOM_F3 : Atom format 3.; NNE 557 80: I_ATOM_F3 : Atom format 3.; NNE 558 81: I_ATOM_F3 : Atom format 3.; ENN 559 82: I_EXCEPT : Exception.; Data Fault; Ret Addr Follows; 560 84: I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000007F89F283F0; 561 93: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0xFFFFFFC000083400; Ctxt: AArch64,EL1, NS; 562 104: I_TRACE_ON : Trace On. 563 105: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000007F89F283F0; Ctxt: AArch64,EL0, NS; 564 116: I_ATOM_F5 : Atom format 5.; NNNNN 565 117: I_ATOM_F5 : Atom format 5.; NNNNN 566 567 568Trace Decoding with Perf Script 569------------------------------- 570Working with perf scripts needs more command line options but yields 571interesting results. 572 573 linaro@t430:~/linaro/coresight/trace-results$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-master/tools/perf/ 574 linaro@t430:~/linaro/coresight/trace-results$ export SCRIPT_PATH=$EXEC_PATH/scripts/python/ 575 linaro@t430:~/linaro/coresight/trace-results$ export XTOOL_PATH=/your/aarch64/toolchain/path/bin/ 576 linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf --exec-path=${EXEC_PATH} script --script=python:${SCRIPT_PATH}/arm-cs-trace-disasm.py -- -d ${XTOOL_PATH}/aarch64-linux-gnu-objdump 577 578 7f89f24d80: 910003e0 mov x0, sp 579 7f89f24d84: 94000d53 bl 7f89f282d0 <free@plt+0x3790> 580 7f89f282d0: d11203ff sub sp, sp, #0x480 581 7f89f282d4: a9ba7bfd stp x29, x30, [sp,#-96]! 582 7f89f282d8: 910003fd mov x29, sp 583 7f89f282dc: a90363f7 stp x23, x24, [sp,#48] 584 7f89f282e0: 9101e3b7 add x23, x29, #0x78 585 7f89f282e4: a90573fb stp x27, x28, [sp,#80] 586 7f89f282e8: a90153f3 stp x19, x20, [sp,#16] 587 7f89f282ec: aa0003fb mov x27, x0 588 7f89f282f0: 910a82e1 add x1, x23, #0x2a0 589 7f89f282f4: a9025bf5 stp x21, x22, [sp,#32] 590 7f89f282f8: a9046bf9 stp x25, x26, [sp,#64] 591 7f89f282fc: 910102e0 add x0, x23, #0x40 592 7f89f28300: f800841f str xzr, [x0],#8 593 7f89f28304: eb01001f cmp x0, x1 594 7f89f28308: 54ffffc1 b.ne 7f89f28300 <free@plt+0x37c0> 595 7f89f28300: f800841f str xzr, [x0],#8 596 7f89f28304: eb01001f cmp x0, x1 597 7f89f28308: 54ffffc1 b.ne 7f89f28300 <free@plt+0x37c0> 598 7f89f28300: f800841f str xzr, [x0],#8 599 7f89f28304: eb01001f cmp x0, x1 600 7f89f28308: 54ffffc1 b.ne 7f89f28300 <free@plt+0x37c0> 601 602Kernel Trace Decoding 603--------------------- 604 605When dealing with kernel space traces the vmlinux file has to be communicated 606explicitely to perf using the "--vmlinux" command line option: 607 608 linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf report --stdio --vmlinux=./vmlinux 609 ... 610 ... 611 linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf script --vmlinux=./vmlinux 612 613When using scripts things get a little more convoluted. Using the same example 614an above but for traces but for kernel traces, the command line becomes: 615 616 linaro@t430:~/linaro/coresight/trace-results$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-master/tools/perf/ 617 linaro@t430:~/linaro/coresight/trace-results$ export SCRIPT_PATH=$EXEC_PATH/scripts/python/ 618 linaro@t430:~/linaro/coresight/trace-results$ export XTOOL_PATH=/your/aarch64/toolchain/path/bin/ 619 linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf --exec-path=${EXEC_PATH} script \ 620 --vmlinux=./vmlinux \ 621 --script=python:${SCRIPT_PATH}/arm-cs-trace-disasm.py -- \ 622 -d ${XTOOLS_PATH}/aarch64-linux-gnu-objdump \ 623 -k ./vmlinux 624 ... 625 ... 626 627The option "--vmlinux=./vmlinux" is interpreted by the "perf script" command 628the same way it is for "perf report". The option "-k ./vmlinux" is dependant 629on the script being executed and has no related to the "--vmlinux", though it 630is highly advised to keep them synchronized. 631 632 633Perf Test Environment Scripts 634----------------------------- 635 636The decoder library comes with a number of `bash` scripts that ease the setting up of the 637offline build and test environment for perf, and executing tests. 638 639These scripts can be found in 640 641 decoder/tests/perf-test-scripts 642 643There are three scripts provided: 644 645- `perf-setup-env.bash` : this sets up all the environment variables mentioned above. 646- `perf-test-report.bash` : this runs `perf report` - using the environment setup by `perf-setup-env.bash` 647- `perf-test-script.bash` : this runs `perf script` - using the environment setup by `perf-setup-env.bash` 648 649Use as follows:- 650 6511. Prior to building perf, edit `perf-setup-env.bash` to conform to your environment. There are four lines at the top of the file that will require editing. 652 6532. Execute the script using the command: 654 655 source perf-setup-env.bash 656 657 This will set up a perf execute environment for using the perf report and script commands. 658 659 Alternatively use the command: 660 661 source perf-setup-env.base buildenv 662 663 This will add in the build environment variables mentioned in the sections on building above alongside the 664 environment for using the used by the `perf-test...` scripts to run the tests. 665 6663. Build perf as described above. 6674. Follow the instructions for downloading the test capture, or create a capture from your target. 6685. Copy the `perf-test...` scripts into the capture data directory -> the one that contains `perf.data`. 669 6706. The scripts can now be run. No options are required for the default operation, but any command line options will be added to the perf report / perf script command line. 671 672e.g. 673 674 ./perf-test-report.bash --dump 675 676will add the --dump option to the end of the command line and run 677 678 ${PERF_EXEC_PATH}/perf report --stdio --dump 679 680 681Generating coverage files for Feedback Directed Optimization: AutoFDO 682--------------------------------------------------------------------- 683 684See autofdo.md (@ref AutoFDO) for details and scripts. 685 686 687The Linaro CoreSight Team 688------------------------- 689- Mike Leach 690- Mathieu Poirier 691 692 693One Last Thing 694-------------- 695We welcome help on this project. If you would like to add features or help 696improve the way things work, we want to hear from you. 697 698Best regards, 699*The Linaro CoreSight Team* 700 701-------------------------------------- 702[1]: https://github.com/Linaro/OpenCSD 703