README.md
1# Checked Tests
2
3Checked tests is the tests that have additional functionality to check result of the test being run.
4For example, check some event was raised or some IR instruction is appeared after specific optimization.
5
6Each checker's command should start with special token (`'#!'` for panda assembly language, `//!` for other languages) at the
7beginning of the line.
8
9Allowed multiple checkers in a single file. Each checker starts with command `CHECKER` and ends with line without
10command token at the zero position.
11
12Each command is a valid code in the `ruby` language.
13
14## List of commands
15
16* **CHECKER** (description: string) begin new Checker with specified description
17* **RUN** run panda application, following named arguments are allowed:
18 - *options: string* - additional options for Panda VM
19 - *entry: string* - entry point, default - `_GLOBAL::main`
20 - *result: int* - expected value to be returned by the `panda` application
21 - *abort: int* - expected terminal signal
22 - *env: string* - environment variables setup string to be used as execution command prefix
23 - *force_jit: bool* - run jit compilation for every executed method
24 - *force_profiling: bool* - enables profdata collection for every interpreter-executed method
25 - *pgo_emit_profdata: bool* - enables on-disk generating for profile data to be used in PGO
26* **RUN_PAOC** run paoc application on the compiled panda file. Output panda file will be passed to the following panda
27 run. Thus, `RUN_PAOC` command must be placed before `RUN` command. Following options are allowed:
28 - *options: string* - additional options for paoc
29 - *bool: bool* - enables boot output
30 - *result: int* - expected value to be returned by the `paoc`
31 - *abort: int* - expected terminal signal
32 - *env: string* - environment variables setup string to be used as execution command prefix
33 - *inputs: string* - list of panda files to compile
34 - *output: string* - name of AOT file to be generated (incompatible to subsequent `RUN` call for now)
35 - *pgo_use_profdata: bool* - usese profile data for PGO. Should be used after `RUN` call with `pgo_emit_profdata` enabled
36* **RUN_PGO_PROF** - runs panda application with forces profiling and PGO profdata emitting. Output profdata will be passed to the following AOT run that use PGO. Syntax sugar for `RUN` command with `pgo_emit_profdata` enabled
37* **RUN_PGO_PAOC** - runs paoc to compile panda file using collected profdata. Should be run after `RUN` with `pgo_emit_profdata` enabled. Syntax sugar for `RUN_PAOC` call
38* **RUN_LLVM** - runs paoc application on the compiled panda file with LLVM AOT backend. Requires LLVM support at `paoc` and passing `--with-llvm` to `checker.rb`
39* **RUN_AOT** - runs paoc application on the compiled panda file with various AOT modes. Currently, command creates separate checkes for `RUN_PAOC` and `RUN_LLVM` backends
40* **RUN_BCO** - runs frontend with bytecode optimizer
41* **EVENT** (event: pattern) search event within all events
42* **EVENT_NEXT** (event: pattern) ordered search event, i.e. search from position of the last founded event
43* **EVENT_NOT** (event: pattern) ensure event is not occurred
44* **EVENT_NEXT_NOT** (event: pattern) ensure event is not occurred after current position
45* **METHOD** (name: string) start check of specified method, all following checks that require specific method will use method specified by this command
46* **PASS_AFTER** (pass_name: string) specify pass after which IR commands should operate
47* **PASS_BEFORE** (pass_name: string) select pass that is right before the specified one
48* **INST** (inst: pattern) search specified instruction in the ir dump file specified by commands `METHOD` and `PASS_AFTER`
49* **INST_NOT** (inst: pattern) equal to `NOT INST`, i.e. check that instruction is not exist
50* **INST_NEXT_NOT** (event: pattern) ensure instruction is not occurred after current position
51* **IR_COUNT** (inst: string) search specified phrase and counts the number in the ir dump file specified by commands `METHOD` and `PASS_AFTER`, returns the value
52* **BLOCK_COUNT** () equal to `IR_COUNT ("BB ")`, i.e. search specified basic blocks and counts the number
53* **TRUE** (condition) ensure the condition is correct
54* **SKIP_IF** (condition) if condition is `true`, skip all commands from that to end of this checker
55* **ASM_METHOD** (name: string) select a specified method in disasm file, next "ASM*" checks will be applied only for this method's code.
56* **ASM_INST** (inst: pattern) select a specified instruction in disasm file, next "ASM*" checks will be applied only for this instruction's code.
57* **ASM/ASM_NEXT/ASM_NOT/ASM_NEXT_NOT** (inst: pattern) same as other similar checks, but search only in a current disasm scope, defined by `ASM_METHOD` or `ASM_INST`.
58If none of these checks were specified, then search will be applied in the whole disasm file.
59* **IN_BLOCK** (block: pattern) limits the search for instructions to one block. The block is defined by lines "props: ..." and "succs: ...". The search pattern is found in the first line "props: rest\_of\_line\_for\_matching". You can define only one block for searching and you can't return to the search in the entire method after in the currently analized compiler pass.
60
61*pattern* can be a string(surrounded by quotes) or regex(surrounded by slashes): string - `"SearchPattern"`, regex - `/SearchPattern/`.
62