Functional and Stress Tests: Definition and Structure ====================================================== This document describes what functional and stress tests are, how they are created, and the structure that they follow in CVS. Functional Test Definition -------------------------- Functional tests test the behavior of functional areas of the POSIX* specification. They do not relate directly to lines in the POSIX specifications, but rather test that the overall behavior specified by the functional area is present. Stress Test Definition ---------------------- Stress tests are designed to monitor how the system behaves when it is taxed by excessively using the functional areas in the POSIX specification or monitoring how the functional areas in the POSIX specification behave when the system is taxed. Directory Structure ------------------- Functional and stress tests follow the same structure. The functional/ and stress directories off of posixtestsuite are where the functional and stress tests, respectively, are stored. Each of these are further subdivided into POSIX area (Timers, Semaphores, etc.). For example, functional/timers. Within the functional/ directory, you should create the following files. - Makefile - master Makefile which will make all tests within the area - run.sh - file which will run all functional tests. The run.sh can run conformance tests, functional tests from a different area (i.e., functional/threads/run.sh can call functional/timers/threadstests/test.c), as well as functional tests in the current area. If there are tests that apply to more than one area, they can go in either area and be called from the other if needed. Assume that the run.sh is called from the directory it is currently in when writing path names to files. - assertions.xml - This file maps test descriptions to test cases as in conformance tests. The grammar for functional/stress tests is: ]> Where the tag follows the same structure outlined in HOWTO_Assertions, and the files="..." field tags a comma delimited list of files which are used to test the assertion. For example, Three threads can be run in parallel. - coverage.txt - This document contain information on the completion of test cases for assertions as in the conformance tests. Content is lines with content: So long as the functional/stress tests follow these guidelines, they can be built and run by the framework. How the directory is subdivided is up to the area creator to define, but some suggestions have been: - create directories corresponding to POSIX chapters (see .1-1990 or .1-1996 specs) - create a number corresponding to the assertion ID for each directory * POSIX (R) is a registered trademark of the IEEE Contributors: julie.n.fleischer REMOVE-THIS AT intel DOT com ajosey REMOVE-THIS AT rdg DOT opengroup DOT org geoffrey.r.gustafson REMOVE-THIS AT intel DOT com