1# Interoperability Benchmarks 2 3## Tests 4 5There are 4 type of tests: 6 7- Dynamic-To-Static (tagged `d2s`, platform `interop_d2s`) 8- Static-To-Dynamic (tagged `s2d`, platform `interop_s2d`) 9- Dynamic-To-Dynamic (tagged `d2d`, platform `interop_d2d`) 10- Static-To-Static (tagged `s2s`, platform `arkts_host` i.e. `ArkTS` + `ets` import ) 11 12First letter signifies context in which measurement cycle runs, 13calling methods from context signified by second letter. 14 15Each type could only be run in separate launch of `vmb`. 16 17Note: `lib*` prefix is reserved for 'imported' code, 18this should not be mixed with 'included' or 'benchmark' files. 19All other file and directory names are arbitrary. 20 21## How to run 22 23```sh 24# Build vmb if needed 25pushd $PANDA_ROOT/tests/vm-benchmarks 26make vmb 27popd 28 29# Set required env vars 30# assuming ArkjsVM and PANDA are built with interop support 31export PANDA_BUILD=$PANDA_ROOT/build 32export PANDA_STDLIB_SRC=$PANDA_ROOT/plugins/ets/stdlib 33 34# This is just a short cut and has no special meaning. 35# Free arguments to `vmb` could be any set of paths (files and/or directories) 36TESTS=$PANDA_ROOT/plugins/ets/tests/benchmarks/interop_js 37 38# Run Dynamic-To-Static benches: 39vmb all -p interop_d2s -T d2s -L js -l js --exclude-list=$TESTS/known-fails-d2s.txt --timeout=60 $TESTS 40 41# Run Static-To-Dynamic benches: 42vmb all -p interop_s2d -T s2d -A --exclude-list=$TESTS/known-fails-s2d.txt --timeout=60 $TESTS 43 44# Run Dynamic-To-Dynamic benches: 45vmb all -p interop_d2d -T d2d -L js -l js --exclude-list=$TESTS/known-fails-d2d.txt --timeout=60 $TESTS 46 47# Run Static-To-Static benches: 48vmb all -p arkts_host -T s2s -A --exclude-list=$TESTS/known-fails-s2s.txt --timeout=60 $TESTS 49``` 50 51## Other options 52 53See also [vmb manual](../../../../../tests/vm-benchmarks/readme.md) 54 55```sh 56# List all options 57vmb help 58 59# List command-specific options: vmb [gen,run,report,all,version,list] --help 60vmb all --help 61vmb report --help 62``` 63 64```sh 65# Example 66vmb all -p interop_s2d \ 67 --fast-iters=1 \ 68 --aot-skip-libs \ 69 -v debug \ 70 --tests=testToString_hex \ 71 --tags=s2d \ 72 --skip-tags=promise \ 73 --ark-custom-option=--gc-trigger-type=heap-trigger \ 74 --ark-custom-option=--compiler-enable-jit=true \ 75 --ark-custom-option=--run-gc-in-place=false \ 76 --ark-custom-option=--log-components=ets_interop_js \ 77 --ark-custom-option=--load-runtimes=ets \ 78 --exclude-list=$TESTS/known-fails-s2d.txt \ 79 $TESTS 80 81``` 82 83## How to compare results 84 85```sh 86# run 1: 87 vmb all -p interop_s2d -T s2d -A --report-json=1.json $TESTS/array_map_reduce/ 88 89# run 2: 90 vmb all -p interop_d2s -T d2s -L js -l js --report-json=2.json $TESTS/array_map_reduce/ 91 92# compare 93vmb report --compare --full 1.json 2.json 94 95## print json report 96vmb report --full 1.json 97``` 98