libc-test is developed as part of the musl project
http://www.musl-libc.org/

configuring:
	cp config.mak.def config.mak
	edit config.mak
build and run tests:
	make
clean up:
	make clean

make builds all test binaries and runs them to create
a REPORT file that contains all build and runtime errors
(this means that make does not stop at build failures)

contributing tests:

design goals:

- tests should be easy to run and build even a single test in isolation
(so test should be self contained if possible)
- failure of one test should not interfere with others
(build failure, crash or unexpected results are all failures)
- test output should point to the cause of the failure
- test results should be robust
- the test system should have minimal dependency
(libc, posix sh, gnu make)
- the test system should run on all archs and libcs
- tests should leave the system in a clean state

conventions:

each test is in a separate file at a path like src/directory/file.c with
its own main

the test should return 0 on success and non-0 on failure, on failure it
should print error messages to standard out if possible, on success no
message should be printed

to help with the above test protocol use t_error function for printing
errors and return t_status from main, see src/common/test.h
(t_error allows standard printf formatting, outputs at most 512bytes
in a single write call to fd 1, so there is no buffering, long outputs
are truncated, it sets the global t_status to 1)

it is common to do many similar checks in a test, in such cases macros
may be used to simplify the code like
#define T1(a,b) (check(a,b) || (t_error("check(%s,%s) failed\n", a, b),0))
#define T2(f,w) (result=(f), result==(w) || (t_error("%s failed: got %s, want %s\n", #f, result, w),0))

binaries should be possible to run from arbitrary directory.
the build system runs the tests using the src/common/runtest tool which
kills the test process after a timeout and reports the exit status
in case of failure

directories:

src/api: interface tests, build time include header tests
src/common: common utilities compiled into libtest.a
src/functional: functional tests aiming for large coverage of libc
src/math: tests for each math function with input-output test vectors
src/regression: regression tests aiming for testing particular bugs

initial set of functional tests are derived from the libc-testsuit of
Rich Felker, regression tests should contain reference of the bug
(musl commit hash, glibc bug tracker url, etc)

build system:

the main non-file make targets are all, run, clean and cleanall.
(cleanall removes the reports unlike clean, run reruns the dynamically
linked executables)

make variable can be overridden from config.mak or the make command line,
the variable B sets the build directory which is src by default

for each directory under src there are targets like $(B)/directory/all,
$(B)/directory/run and $(B)/directory/clean to make only the contents
of that directory, each directory has its own Makefile set up so it
invokes the top level make with B=src src/directory/foo for the foo
target, so it is possible to work only under a specific test directory

the build and runtime errors of each target are accumulated into a
target.err file and in the end they are concatenated into a REPORT

each .c file in src/functional and src/regression are built into a
dynamic linked and a static linked executable test binary by default,
this behaviour can be changed by a similarly named .mk file changing
make variables and specifying additional rules:

$(B)/$(N) is the name of the binary target (the file name without the .c)
$(B)/$(N)-static is the name of the static binary target
$(B)/$(D) is the build directory
$(N).CFLAGS are added to the CFLAGS at compilation
$(N).LDFLAGS are added to the LDFLAGS at linking
$(N).LDLIBS are added to the LDLIBS at linking
$(N).BINS are the targets (if empty no binaries are built)
$(N).LIBS are the non-executable targets (shared objects may use it)

if a binary is linked together from several .o files then they
have to be specified as prerequisits for the binary targets and
added to the $(N).LDLIBS as well

if a binary depends on a file at runtime (eg. a .so opened by dlopen)
then the $(N).err target should depend on that file

## libc-test脚本使用说明

### 一、编译

编译命令，仅编译libc-test测试用例
```
./build.sh --product-name rk3568 --build-target libctest
```
用例可执行文件生成的目录位于  `out/rk3568/musl/libc-test`
以及相关的动态库位于 `out/rk3568/musl/libc-test-lib`

### 二、执行测试

1. 将HDC(Windows版)工具hdc.exe加入到windows环境变量。连接上开发板，保证HDC工具可以连接。

2. 进入musl/scripts，并根据实际环境，修改runtest_Windows.bat中测试用例生成目录完整路径（必要）
   ```bat
   @REM 本地OpenHarmony源码目录,需根据实际环境修改
   set LOCAL=\\wsl$\ubuntu-20.04\home\code\OpenHarmony
   @REM 本地编译out目录
   set OUTDIR=out\rk3568
   ```

3. 将bat脚本复制到windows环境下使用！windows端双击运行runtest_Windows.bat，执行完毕会将详细错误结果信息REPORT返回到当前bat所在目录，并对结果进行统计生成FileList.txt（本次测试执行用例列表）与Summary.txt（统计报告），内容如下：
   $\color{red}{提交PR需在提交里附上对应统计报告！}$
   ```
   SUMMARY
   All: 1177 | Run: 1147 | Skip: 30
   Pass: [1083/1147]
   Fail: [64/1147]
   Check REPORT for more details.

   [Failed List]:
   FAIL __fwritable [signal Segmentation fault]
   FAIL __fwriting [signal Segmentation fault]
   FAIL a_stack_chk_fail [status 255]
   FAIL asctime [status 1]
   ...

   [Skip List]:
   trace_stresstest
   fatal_message
   tgkill_ext
   exittest01
   ...
   ```

   注：此bat文件是从WSL映射目录传输文件到单板目录下，由于WSL无法直接访问windows端口，所以在WSL目录中使用可能无法传输用例。

4. 若要单独执行测试用例，将编译结果目录下的`runtest`与需要执行的用例传输到单板

   单板执行 `./runtest -w '' {filename}`，例如：

   测试成功，无报错信息
   ```
   # ./runtest -w '' memmem
   ```

   测试失败，提示FAIL+用例名+错误码
   ```
   # ./runtest -w '' pthread_atfork-errno-clobber
   ../../third_party/musl/libc-test/src/regression/pthread_atfork-errno-clobber.c:23: (pid = fork()) == -1 failed: fork succeeded despite rlimit
   FAIL pthread_atfork-errno-clobber [status 1]
   ```

### 三、增加用例

举例：在functional目录中新增用例new_test

1. 在libc-test/src/functional目录下新建`new_test.c`

2. 编辑libc-test/src/functional目录下的`test_src_functional.gni`，并在`functional_list`末尾添加`"new_test",`

3. （可选）利用模板添加编译选项，修改`libc-test/test_template.gni`中编译选项内容或针对指定的用例添加，例如：
   ```
   if (target_name == "tls_init") {
     cflags += [ "-Wno-literal-range" ]
     ldflags += [ "-Wl,-rpath=./" ]
     libs += [ "//${root_out_dir}/${test_lib_dir}/libtls_init_dso.so" ]
   }
   ```

3. 执行步骤一（编译）中的编译命令，即可生成对应可执行文件。