Lines Matching +full:gcc +full:- +full:8 +full:- +full:asan +full:- +full:ubsan +full:- +full:fuzz
6 bus][1]. In other words, it's meant to make the [bus factor][1] a non-issue.
25 and C. They both are arbitrary-precision command-line calculators with their own
30 In addition, it is also possible to build the arbitrary-precision math as a
58 This project became a passion project for me, and I continued. In mid-2019,
64 about things like fuzzing, [`scan-build`][19], [valgrind][20],
148 pseudo-random number generator (for the same seed).
181 * Pseudo-Random Number Generator (see [`include/rand.h`][37]).
206 8. Headers.
211 1. High-level requirements
212 2. Low-level requirements
213 3. High-level implementation
214 4. Low-level implementation
231 alias makej='make -j16'
233 alias mcmakej='make clean && make -j16'
234 alias bcdebug='CPPFLAGS="-DBC_DEBUG_CODE=1" CFLAGS="-Weverything -Wno-padded \
235 -Wno-switch-enum -Wno-format-nonliteral -Wno-cast-align \
236 -Wno-unreachable-code-return -Wno-missing-noreturn \
237 -Wno-disabled-macro-expansion -Wno-unreachable-code -Wall -Wextra \
238 -pedantic -std=c99" ./configure.sh'
239 alias bcconfig='CFLAGS="-Weverything -Wno-padded -Wno-switch-enum \
240 -Wno-format-nonliteral -Wno-cast-align -Wno-unreachable-code-return \
241 -Wno-missing-noreturn -Wno-disabled-macro-expansion -Wno-unreachable-code \
242 -Wall -Wextra -pedantic -std=c99" ./configure.sh'
243 alias bcnoassert='CPPFLAGS="-DNDEBUG" CFLAGS="-Weverything -Wno-padded \
244 -Wno-switch-enum -Wno-format-nonliteral -Wno-cast-align \
245 -Wno-unreachable-code-return -Wno-missing-noreturn \
246 -Wno-disabled-macro-expansion -Wno-unreachable-code -Wall -Wextra \
247 -pedantic -std=c99" ./configure.sh'
248 alias bcdebugnoassert='CPPFLAGS="-DNDEBUG -DBC_DEBUG_CODE=1" \
249 CFLAGS="-Weverything -Wno-padded -Wno-switch-enum -Wno-format-nonliteral \
250 -Wno-cast-align -Wno-unreachable-code-return -Wno-missing-noreturn \
251 -Wno-disabled-macro-expansion -Wno-unreachable-code -Wall -Wextra \
252 -pedantic -std=c99" ./configure.sh'
259 command-line.
277 export BC_ENV_ARGS="-l $HOME/.bcrc"
283 run `bc -s` on my `$HOME/.bcrc`.
299 t-=s
301 t-=m
303 t-=h
374 * If single-line bodies are on a separate line from their headers, and the
401 machine-useful style on `bc`. All three failed. Otherwise, the style in this
406 Functions are documented with Doxygen-style doc comments. Functions that appear
460 Turing-complete programming language. It's a terrible one, but it works.
474 1. It processes command-line arguments and figure out what the user wants to
477 3. One-by-one, it replaces placeholders (in [`Makefile.in`][70]) of the form
508 [POSIX `make`][74]-compatible `Makefile` (minus the placeholders). Here are a
627 A text file containing the text displayed for `bc -h` or `bc --help`.
629 This text just contains the command-line options and a short summary of the
640 A text file containing the text displayed for `dc -h` or `dc --help`.
642 This text just contains the command-line options and a short summary of the
671 First, all constants must be one digit. This is because otherwise, multi-digit
673 This does not happen with single-digit numbers because they are guaranteed to be
680 to set it with a single-digit number and beware the snare below...
683 from any function in the library. This is because without the `-g` option,
691 must not use *any* extensions. It has to work when users use the `-s` or `-w`
698 Like [`lib.bc`][8], and for the same reasons, this file should have no
703 Also, be sure to check [`lib.bc`][8] for the snares that can trip up unwary
760 This file is the API for processing command-line arguments.
764 This header is the API for `bc`-only items. This includes the `bc_main()`
765 function and the `bc`-specific lexing and parsing items.
781 that it has platform-specific fixes for Windows. (If the fixes were not in this
788 This header is the API for `dc`-only items. This includes the `dc_main()`
789 function and the `dc`-specific lexing and parsing items.
806 This header is for `bc`'s implementation of command-line editing/history, which
807 is based on a [UTF-8-aware fork][28] of [`linenoise`][29].
809 For more information, see the [Command-Line History][189] section.
846 This header is the API for parsing command-line arguments.
849 to process the command-line arguments into global data *after* they have already
851 parses the command-line arguments, and [`args.h`][31] turns that parsed data
857 adapted a [public-domain option parsing library][34] to do the job instead. And
885 This header defines the API for the [pseudo-random number generator
888 The PRNG only generates fixed-size integers. The magic of generating random
908 * Compiler-specific fixes.
909 * Platform-specific fixes.
1101 collection mechanism of the [`BcNum` caching][96] works. It has been little-used
1125 #### `exec-install.sh`
1152 and run the script when I want to fuzz.
1252 8. Calculates and outputs SHA512 and SHA256 sums for all of the files,
1256 meant for outside use. This means that some non-POSIX utilities can be used,
1269 A script to test `bc`'s command-line expression parsing code, which, while
1288 previously-installed version of this `bc` and `dc`.
1291 `bc` using `./scripts/fuzz_prep.sh -a`, and then run it under this script. Any
1303 report mode. One-by-one, it will go through the "checklist," the list of failed
1326 alone for 32-bit.
1330 5. Runs the [test suite][124] under [ASan, UBSan, and MSan][21] for every build
1337 #### `safe-install.sh`
1375 Code for processing command-line arguments.
1443 The code for `bc`'s implementation of command-line editing/history, which is
1444 based on a [UTF-8-aware fork][28] of [`linenoise`][29].
1446 For more information, see the [Command-Line History][189] section.
1490 The code for all of the arbitrary-precision [numbers][177] and [math][178] in
1497 The code for parsing command-line options.
1515 The code for the [pseudo-random number generator (PRNG)][179] and the special
1518 The PRNG only generates fixed-size integers. The magic of generating random
1587 This script also re-runs the test three times if it fails. This is because
1722 fuzz run. For more information, see the [Convenience][130] subsection of the
1728 fuzz run. For more information, see the [Convenience][130] subsection of the
1838 the Karatsuba multiplication algorithm switches to brute-force
1860 disabled, but both `BC_ENABLED` and `DC_ENABLED` must be non-zero.
1911 : If this macro expands to a non-zero integer, then `bc` is built with *a lot*
1948 make -j<cores> test
1955 I have even tried as much as possible, to put longer-running tests near the
1981 are generated by a GNU-compatible `bc` or `dc`. See the [Generated Tests][143]
1990 and its results are generated by a GNU-compatible `bc` or `dc`. See the
1993 To add a non-generated standard test, do the following:
2139 Single-letter numbers always be set to the same value, regardless of
2178 classify them, but it's really testing multi-line numbers.
2214 : Tests the pseudo-random number generator and its special stack handling.
2337 : Tests the pseudo-random number generator and its special stack handling.
2369 `-g` and `--global-stacks` flags. This means that all of the script tests for
2370 `bc` are written assuming the `-g` flag was given on the command-line
2497 machine-generated (by [AFL++][125]) ones.
2519 are read line-by-line and shoved through `stdin`, and each line is considered a
2527 on the command-line.
2531 Is it a simple one-liner, and you don't care if it's tested through a file?
2539 command-line, or the error requires multiple lines to reproduce, then put the
2547 The `stdin` tests specifically test the lexing and parsing of multi-line
2568 other types of tests. They usually include things like command-line parsing and
2610 parallel execution with `make -j<cores> test`.
2628 Yes, the error code is not a success all the time. This is because of the UTF-8
2629 tests; `bc` gives a fatal error on any non-ASCII data because ASCII is all `bc`
2659 Instead, the tests depend on the existence of a GNU-compatible `bc` in the
2662 If [`configure.sh`][69] was run with the `-G` argument, which disables generated
2665 GNU-compatible `bc` installed.
2670 For example, `bc`'s test suite will automatically use a GNU-compatible `bc` to
2694 GNU-compatible `bc` is used to generate the `<test>.txt` results file.
2713 In addition, there are more fine-grained targets available:
2839 In order to get test coverage information, you need `gcc`, `gcov`, and `gcovr`.
2844 CC=gcc ./configure -gO3 -c
2845 make -j<cores>
2849 Note that `make coverage` does not have a `-j<cores>` part; it cannot be run in
2850 parallel. If you try, you will get errors. And note that `CC=gcc` is used.
2870 make -j<cores>
2879 make -j<cores>
2880 make -j<cores> test
2892 CFLAGS="-fsanitize=<sanitizer> ./configure -gO3 -m
2893 make -j<cores>
2894 make -j<cores> test
2899 that has zero optimization, so for `UBSan`, use the following commands:
2902 CFLAGS="-fsanitize=undefined" ./configure -gO0 -m
2903 make -j<cores>
2904 make -j<cores> test
2912 ./configure -gO3 -v
2913 make -j<cores>
2914 make -j<cores> test
2935 1. No `bash`-isms.
2937 3. Only command-line options defined in the POSIX standard for POSIX utilities
2962 ### Maintainer-Only Scripts
3010 POSIX-compatible locales are terrible.
3014 First, `gencat` does not work for generating cross-compilation. In other words,
3015 it does not generate machine-portable files. There's nothing I can do about
3058 [`scan-build`][19]. I only use it in debug mode because I have to add some
3063 function to be initialized. [`scan-build`][19] misses that fact, so I
3064 pre-initialize such locals to prevent the warnings.
3066 To run `scan-build`, do the following:
3070 scan-build make
3073 `scan-build` will print its warnings to `stdout`.
3135 stability: the pseudo-random number generator *must* start with *exactly* the
3163 assumes the existence of `afl-clang-lto` in the `$PATH`, but if that exists, it
3164 automatically configures and builds `bc` with a fuzz-ideal build.
3166 A fuzz-ideal build has several things:
3168 * `afl-clang-lto` as the compiler. (See [AFL++ Quickstart][129].)
3170 * Full optimization (including [Link-Time Optimization][126]), for performance.
3175 There is one big thing that a fuzz-ideal build does *not* have: it does not use
3181 However, to add to [`scripts/fuzz_prep.sh`][119] making a fuzz-ideal build, in
3317 to use with varying command-line arguments, which was needed for testing `bc`'s
3318 command-line expression parsing code, and [AFL++][125] is best when testing
3325 not had to touch the command-line expression parsing code since.
3347 ./scripts/fuzz_prep.sh -a
3348 ./scripts/afl.py --asan bc1
3349 ./scripts/afl.py --asan bc2
3350 ./scripts/afl.py --asan bc3
3351 ./scripts/afl.py --asan dc
3357 These commands build an [ASan][21]-enabled build of `bc` and `dc` and then they
3363 in `bc` or `dc`, [`scripts/afl.py`][94] has a timeout of 8 seconds, which is far
3367 If [AFL++][125] fails to find crashes *and* [ASan][21] fails to find memory
3382 POSIX mode is `bc`-only.
3385 These modes are designed to help users write POSIX-compatible `bc` scripts.
3389 Standard Mode is activated with the `-s` or `--standard` flags.
3396 Warning Mode is activated with the `-w` or `--warn` flags.
3420 ### [Async-Signal-Safe][115] Signal Handling
3423 `bc` is, by and large, CPU-bound. This has several consequences, but the biggest
3433 Alternatively, I/O bound programs can use the [self-pipe trick][199].
3463 Other than that, the rest of the interrupt-based implementation is best
3468 First, signal handlers can only call [async-signal-safe][115] functions.
3481 Vectors and numbers needed special consideration with the interrupt-based signal
3488 be able to interrupt a non-[async-signal-safe][115] function like `malloc()` and
3526 At first glance, it may seem like putting asserts for pointers being non-`NULL`
3664 for general-purpose allocators to handle efficiently.
3681 slab is put into the vacated spot. This ensures that a non-special slab is
3684 ### Command-Line History
3689 It sucked, and the biggest reason why was because of the lack of command-line
3692 At first, I just dealt with it, not knowing how command-line history might be
3695 Eventually, I caved and attempted to adapt [`linenoise-mob`][28], which I had
3701 Understanding command-line history in `bc` is really about understanding VT-100
3705 adaptation to make the command-line history implementation perfect for `bc`
3711 all of that, it does not have at least one feature: multi-line pasting from the
3759 It turns out that what I needed was a [async-signal-safe][115] form of what
3770 2. While `longjmp()` is required to be [async-signal-safe][115], if it is
3771 invoked by a signal handler that interrupted a non-[async-signal-safe][115]
3784 (For more information, see the [Async-Signal-Safe Signal Handling][173]
3786 async-signal-safe, they first need to use `BC_SIG_LOCK` to lock signals, and
3800 In this way, C++-style exceptions were implemented in pure C. Not fun, but it
3815 all of the buffered I/O API is basically non-[async-signal-safe][115].
3824 easily-parsable form.
3853 just processes command-line arguments after they are parsed by
3888 [Shunting-Yard Algorithm][109] is designed to generate [Reverse Polish
3929 *always* at a REPL (Read-Eval-Print Loop). `bc` is, first and foremost, an
3959 `bc`'s language is Turing-complete. That means that code needs the ability to
3978 "Cond" labels are so-called because they are used by conditionals.
3984 well-known. Cond labels are easy.
3993 "Exit" labels are so-called because they are used by code "exiting" out of `if`
4022 That extra work is called the [Shunting-Yard algorithm][109], and the form it
4026 [Shunting-Yard algorithm][109]. Go do that before you read on.
4030 In `bc`, the [Shunting-Yard algorithm][109] is implemented with bytecode as the
4192 little-endian order.
4289 * To seed the pseudo-random number generator.
4325 In order to do arbitrary-precision math, as `bc` must do, there must be some way
4326 of representing arbitrary-precision numbers. `BcNum` in [`include/num.h`][184]
4330 applied to arbitrary-precision numbers. It means one piece of the number. It can
4352 between -1 and 1).
4371 scale + (BC_BASE_DIGS - 1) / BC_BASE_DIGS == rdx >> 1
4375 64-bit systems and 4 on other systems.)
4391 little-endian order. This means that the last decimal places are in the limb
4392 stored at index 0, and the most significant digits are stored at index `len-1`.
4411 equal to -1.
4412 * `(rdx >> 1) == len`: the number is greater than -1 and less than 1, not
4463 ### Pseudo-Random Number Generator
4466 manpages about the pseudo-random number generator (PRNG) first; that will help
4471 cryptographically secure PRNG available via command-line, usually
4479 On top of that, `bc` is an arbitrary-precision calculator; if I made it able to
4493 I could, of course, put the entire 255 bits into one massive arbitrary-precision
4526 3. Divide the seed by `2^128` with a `scale` of 128. (For 32-bit systems,
4530 Likewise, the algorithm to convert from a user-supplied number to a seed is:
4539 #### Generating Arbitrary-Precision Numbers
4555 Besides building `bc` in debug mode with the `-g` flag to [`configure.sh`][69],
4556 programmers can also add `-DBC_DEBUG_CODE=1` to the `CFLAGS`. This will enable
4574 First, you can probably use [profile-guided optimization][217] to optimize even
4592 `__builtin_expect((e), 0)`, respectively, on GCC and Clang for better branch
4594 the [GCC documentation][218].)
4599 Also, as stated in the [build manual][219], link-time optimization is excellent
4635 statically-allocated stack (it's just a global array with a set size). Then,
4661 Even the [pseudo-random number generator][179] is included, with extra support
4667 Like signal handling in `bc` proper (see the [Async-Signal-Safe Signal
4712 The encoding of errors is this: if an error happens, the value `0-error` is
4714 `0-num_errors` is an error.
4727 [5]: ./bc/A.1.md#standard-library
4729 [7]: ./bc/A.1.md#extended-library
4730 [8]: #libbc-2
4733 [11]: https://gavinhoward.com/2019/12/values-for-yao/
4734 [12]: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf
4735 [13]: ./build.md#cross-compiling
4741 [19]: https://clang-analyzer.llvm.org/scan-build.html
4744 [22]: https://gavinhoward.com/2019/11/finishing-software/
4750 [28]: https://github.com/rain-1/linenoise-mob
4754 [32]: ../NEWS.md#3-0-0
4798 [76]: ##posix-shell-scripts
4801 [79]: #bc-1
4802 [80]: #dc-1
4803 [81]: ./build.md#build-type
4804 [82]: #fuzzing-1
4807 [85]: #locales-1
4808 [86]: #manuals-1
4818 [96]: #caching-of-numbers
4819 [97]: #error-handling
4831 [109]: https://en.wikipedia.org/wiki/Shunting-yard_algorithm
4832 [110]: #bc-parsing
4836 [114]: #custom-io
4845 [123]: https://tmuxp.git-pull.com/
4846 [124]: #test-suite
4848 [126]: #link-time-optimization
4849 [127]: #fuzzing-performance
4851 [129]: #afl-quickstart
4854 [132]: https://git.yzena.com/gavin/vim-bc
4860 [138]: #test-suite-portability
4863 [141]: #group-tests
4864 [142]: #build-system
4865 [143]: #generated-tests
4866 [144]: #benchmarks-1
4868 [146]: #test-coverage
4869 [147]: #integration-with-the-build-system
4870 [148]: #test-scripts
4871 [149]: #standard-tests
4872 [150]: #script-tests
4873 [151]: #error-tests
4874 [152]: #stdin-tests
4875 [153]: #read-tests
4876 [154]: #other-tests
4877 [155]: #history-tests
4879 [157]: #bcl-test
4882 [160]: #addresssanitizer-and-friends
4883 [161]: #bc-2
4884 [162]: #dc-2
4885 [163]: #alltxt-1
4889 [167]: #alltxt-3
4890 [168]: #errorstxt-1
4891 [169]: #scripts-1
4892 [170]: #scripts-2
4893 [171]: #alltxt-2
4894 [172]: #alltxt-4
4895 [173]: #async-signal-safe-signal-handling
4900 [178]: #math-style
4901 [179]: #pseudo-random-number-generator
4909 [187]: #slabs-and-slab-vectors
4910 [188]: ./build.md#extra-math
4911 [189]: #command-line-history
4913 [191]: #linux-timeconstbc-script
4916 [194]: https://www.valgrind.org/docs/manual/mc-manual.html
4919 [197]: https://clang-analyzer.llvm.org/
4920 [198]: https://unix.stackexchange.com/questions/253349/eintr-is-there-a-rationale-behind-it
4922 [200]: https://skarnet.org/cgi-bin/archive.cgi?2:mss:1607:201701:dfblejammjllfkggpcph
4923 [201]: https://slembcke.github.io/2020/10/12/CustomAllocators.html#1-slab-allocator
4936 [214]: https://gmplib.org/manual/Nomenclature-and-Types
4938 [216]: #main-and-read-functions
4939 [215]: https://www.pcg-random.org/
4940 [216]: https://lemire.me/blog/2017/08/22/testing-non-cryptographic-random-number-generators-my-resu…
4941 [217]: https://en.wikipedia.org/wiki/Profile-guided_optimization
4942 [218]: https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html#index-_005f_005fbuiltin_005fexpect
4946 [222]: https://www.freebsd.org/cgi/man.cgi?query=ministat&apropos=0&sektion=0&manpath=FreeBSD+13.0-…