1Functional and Stress Tests: Definition and Structure 2====================================================== 3 4This document describes what functional and stress tests are, how they 5are created, and the structure that they follow in CVS. 6 7Functional Test Definition 8-------------------------- 9 10Functional tests test the behavior of functional areas of the POSIX* 11specification. They do not relate directly to lines in the POSIX 12specifications, but rather test that the overall behavior specified by 13the functional area is present. 14 15Stress Test Definition 16---------------------- 17 18Stress tests are designed to monitor how the system behaves when it is 19taxed by excessively using the functional areas in the POSIX specification 20or monitoring how the functional areas in the POSIX specification behave 21when the system is taxed. 22 23Directory Structure 24------------------- 25Functional and stress tests follow the same structure. 26 27The functional/ and stress directories off of posixtestsuite are where the 28functional and stress tests, respectively, are stored. 29 30Each of these are further subdivided into POSIX area (Timers, Semaphores, 31etc.). For example, functional/timers. 32 33Within the functional/<POSIX area> directory, you should create the 34following files. 35- Makefile - master Makefile which will make all tests within the area 36 37- run.sh - file which will run all functional tests. The run.sh can run 38conformance tests, functional tests from a different area (i.e., 39functional/threads/run.sh can call functional/timers/threadstests/test.c), 40as well as functional tests in the current area. If there are tests that 41apply to more than one area, they can go in either area and be called from 42the other if needed. 43Assume that the run.sh is called from the directory it is currently in 44when writing path names to files. 45 46- assertions.xml - This file maps test descriptions to test cases as in 47conformance tests. The grammar for functional/stress tests is: 48<?xml version "1.0"?> 49<!DOCTYPE assertions [ 50 <!ELEMENT assertion> 51 <!ATTLIST assertion 52 files CDATA #REQUIRED 53 tag CDATA #REQUIRED 54 > 55]> 56Where the tag follows the same structure outlined in HOWTO_Assertions, and 57the files="..." field tags a comma delimited list of files which are 58used to test the assertion. For example, 59 <assertion id="1" files="threadone,threadtwo,threadthree" tag="pt:THR"> 60 Three threads can be run in parallel. 61 </assertion> 62 63- coverage.txt - This document contain information on the completion of 64test cases for assertions as in the conformance tests. 65Content is lines with content: 66<assertion ID> <YES/NO - for completed/not completed> 67 68So long as the functional/stress tests follow these guidelines, they 69can be built and run by the framework. How the directory is subdivided 70is up to the area creator to define, but some suggestions have been: 71 - create directories corresponding to POSIX chapters (see .1-1990 or 72 .1-1996 specs) 73 - create a number corresponding to the assertion ID for each directory 74 75* POSIX (R) is a registered trademark of the IEEE 76 77Contributors: julie.n.fleischer REMOVE-THIS AT intel DOT com 78 ajosey REMOVE-THIS AT rdg DOT opengroup DOT org 79 geoffrey.r.gustafson REMOVE-THIS AT intel DOT com 80