1# Run Spec Coverage Tool 2 3The tool reads ISA spec files and the given set of test files, then computes the coverage and outputs a bunch of coverage reports. 4 5``` 6spectrac -r path_to_summary_report_yaml_file \ 7 -d directory_where_tests_located \ 8 -g glob_to_select_tests_files_in_tests_directory \ 9 -s paths_to_isa_yaml_files \ 10 -n path_to_nontestable_yaml_file \ 11 -u path_to_output_notcovered_yaml_file \ 12 -U path_to_output_notcovered_markdown_file \ 13 -o path_to_output_yaml_file_with_irrelevant_tests \ 14 -O path_to_output_markdown_file_with_irrelevant_tests \ 15 -f path_to_output_yaml_file_with_coverage_data \ 16 -F path_to_output_markdown_file_with_coverage_data 17``` 18 19Options: 20``` 21--testdir (-d) - directory with tests, required 22--testglob (-g) - glob for selecting test files in tests directory ("**/*.pa" for example), required 23--spec (-s) - ISA spec files, comma-separated, merged into one if multiple specified, required 24--non-testable (-n) - yaml file with non-testable assertions 25--full (-f) - output the full spec in yaml format with additional fields showing the coverage data (mark assertions as covered/not-covered/non-testable, provide coverage metric for each group) 26--full_md (-F) - same as --full, but in markdown format 27--uncovered (-u) - output yaml document listing the spec areas not covered by tests 28--uncovered_md (-U) - same as --uncovered, but in markdown format 29--orphaned (-o) - output list of test files that found not relevant to the current spec (the assertions in test file aren't found in the spec) 30--orphaned_md (-O) - same as --orphaned, but in markdown format 31--report (-r) - output the test coverage summary report in yaml 32``` 33 34Example: 35``` 36cd ~/panda 37tests/cts-coverage-tool/bin/spectrac.rb \ 38 -g "**/*.pa" \ 39 -d tests/cts-generator/cts-generated/ \ 40 -s isa/isa.yaml,plugins/java/isa/isa.yaml \ 41 -f full.yaml \ 42 -u uncovered.yaml \ 43 -o orphaned.yaml \ 44 -n tests/cts-coverage-tool/config/non_testable.yaml 45``` 46