1# Getting Started 2 3In order to build and test lexical, only a Rust toolchain (1.31+) is required. However, for reasons described below, we highly recommend you install a recent (1.55+) nightly toolchain. 4 5 6```bash 7cargo +nightly build 8cargo +nightly test 9``` 10 11# Code Structure 12 13Functionality is generally made **public** to separate the tests from the implementation, although non-documented members is not stable, and any changes to this code is not considered a breaking change. Tests are separated from the actual implementation, and comprehensively test each individual component. 14 15Furthermore, any unsafe code uses the following conventions: 16 171. Each unsafe function must contain a `# Safety` section. 182. Unsafe operations/calls in unsafe functions must be marked as unsafe, with their safety guarantees clearly documented via a `// SAFETY:` section. 19 20# Dependencies 21 22In order to fully test and develop lexical, a recent, nightly compiler along with following Rust dependencies is required: 23 24- Clippy 25- Rustfmt 26- Miri 27- Valgrind 28- Tarpaulin 29- Fuzz 30- Count 31 32These dependencies may be installed via: 33 34```bash 35rustup toolchain install nightly 36rustup +nightly component add clippy 37rustup +nightly component add rustfmt 38rustup +nightly component add miri 39cargo +nightly install cargo-valgrind 40cargo +nightly install cargo-tarpaulin 41cargo +nightly install cargo-fuzz 42cargo +nightly install cargo-count 43``` 44 45In addition, the following non-Rust dependencies must be installed: 46 47- Python3.6+ 48- python-magic (python-magic-win64 on Windows) 49- Valgrind 50 51# Development Process 52 53The [scripts](https://github.com/Alexhuszagh/minimal_lexical/tree/main/scripts) directory contains numerous scripts for testing, fuzzing, analyzing, and formatting code. Since many development features are nightly-only, this ensures the proper compiler features are used. This requires a nightly compiler installed via Rustup, which can be invoked as `cargo +nightly`. 54 55- [check.sh](https://github.com/Alexhuszagh/minimal_lexical/blob/main/scripts/check.sh): Check rustfmt and clippy without formatting any code. 56- [fmt.sh](https://github.com/Alexhuszagh/minimal_lexical/blob/main/scripts/fmt.sh): Run `cargo fmt` and `cargo clippy` in all projects and workspaces, on nightly. 57- [hooks.sh](https://github.com/Alexhuszagh/minimal_lexical/blob/main/scripts/hooks.sh): Install formatting and lint hooks on commits. 58 59All PRs must pass the following checks: 60 61```bash 62# Check all safety sections and other features are properly documented. 63RUSTFLAGS="--deny warnings" cargo +nightly build --features=lint 64# Ensure all rustfmt and clippy checks pass. 65scripts/check.sh 66``` 67 68# Safety 69 70In order to ensure memory safety even when using unsafe features, we have the following requirements. 71 72- All code with local unsafety must be marked as an `unsafe` function. 73- All unsafe macros must have a `# Safety` section in the documentation. 74- All unsafe functions must have a `# Safety` section in the documentation. 75- All code using `unsafe` functionality must have a `// SAFETY:` section on the previous line, and must contain an `unsafe` block, even in `unsafe` functions. 76- If multiple lines have similar safety guarantees, a `// SAFETY:` section can be used for a block or small segment of code. 77 78In order to very that the safety guarantees are met, any changes to `unsafe` code must be fuzzed, the test suite must be run with Valgrind, and must pass the following commands: 79 80```bash 81# Ensure `unsafe` blocks are used within `unsafe` functions. 82RUSTFLAGS="--deny warnings" cargo +nightly build --features=lint 83# Ensure clippy checks pass for `# Safety` sections. 84cargo +nightly clippy --all-features -- --deny warnings 85``` 86 87# Algorithm Changes 88 89The "docs" directory containing detailed descriptions of algorithms and benchmarks. If you make any substantial changes to an algorithm, you should both update the algorithm description and the provided benchmarks. 90