Lines Matching +full:- +full:- +full:with +full:- +full:zstd
30 Zstd uses a branch-based workflow for making changes to the codebase. Typically, zstd
37 * Checkout your fork of zstd if you have not already
39 git checkout https://github.com/<username>/zstd
40 cd zstd
44 git pull https://github.com/facebook/zstd dev
51 git checkout -b <branch-name>
52 git push origin <branch-name>
57 git add -u && git commit -m <message>
58 git push origin <branch-name>
91 …* Note: if you have been working with a specific user and would like them to review your work, mak…
92 * Submit the pull request and iterate with feedback.
95 … * You will have to iterate on your changes with feedback from other collaborators to reach a point
99 …* Eventually, someone from the zstd team will approve your pull request and not long after merge i…
102 … * Most PRs are linked with one or more Github issues. If this is the case for your PR, make sure
106 … that the change must make it to an official zstd release for it to be meaningful. We recommend
108 … their change makes it to the next release of zstd. Users will often discover bugs in your code or
113 executing it. It usually helps us find many simple bugs. Zstd uses clang's `scan-build` tool for
114 …install it by following the instructions for your OS on https://clang-analyzer.llvm.org/scan-build.
122 In general, you can use `scan-build` to static analyze any build script. For example, to static ana…
126 scan-build make -C contrib/largeNbDicts largeNbDicts
130 `scan-build` is part of our regular CI suite. Other static analyzers are not.
134 - Static analyzers are full of false positive. The signal to noise ratio is actually pretty low.
135 - A good CI policy is "zero-warning tolerance". That means that all issues must be solved, includin…
136 - Multiple static analyzers will feature multiple kind of false positives, sometimes applying to th…
139 - As if that was not enough, the list of false positives change with each version. It's hard enough…
151 Most people will just want to create a PR with the destination set to their local dev
152 branch of zstd. You can then find the status of the tests on the PR's page. You can also
153 re-run tests and cancel running tests from the PR page or from the respective CI's dashboard.
155 Almost all of zstd's CI runs on GitHub Actions (configured at `.github/workflows`), which will auto…
161 ### Third-party CI
163 For these, we use a variety of third-party services (listed below). It is not necessary to set
164 these up on your fork in order to contribute to zstd; however, we do link to instructions for those
168 …-----------|--------------------------------------------------------------------------------------…
169 …-x86 architectures such as PowerPC | https://docs…
170 … | https://www.appveyor.com/blog/2018/10/02/github-apps-integration/ <br> h…
171 … | https://github.com/marketplace/cirrus-ci/ …
172 …ci.com/docs/2.0/getting-started/#setting-up-circleci <br> https://youtu.be/Js3hMUsSZ2c <br> https:…
174 Note: the instructions linked above mostly cover how to set up a repository with CI from scratch.
175 The general idea should be the same for setting up CI on your fork of zstd, but you may have to
177 config files (since zstd already has configs for each of these services).
180 Performance is extremely important for zstd and we only merge pull requests whose performance
181 landscape and corresponding trade-offs have been adequately analyzed, reproduced, and presented.
184 always welcome contributions to improve performance (or worsen performance for the trade-off of
187 1. Zstd isn't as old as gzip but it has been around for time now and its evolution is
191 would-be PR. Of course, just because a topic has already been discussed (and perhaps rejected
197 take the time to run extensive, long-duration, and potentially cross-(os, platform, process, etc)
203 legitimate thing to do as long as it does not harm the overall performance health of Zstd.
204 This is a hard balance to strike but please keep in mind other aspects of Zstd when
205 submitting changes that are clang-specific, windows-specific, etc.
208 Performance microbenchmarking is a tricky subject but also essential for Zstd. We value empirical
214 benchmarking machine. A virtual machine, a machine with shared resources, or your laptop
218 Of course, benchmarking can be done on non-hyper-stable machines as well. You will just have to
233 with the fastest running time) has been the most stable approach.
245 another cpu/memory-intensive application in the background. If you are running benchmarks on your
254 …s, you can "Set Processor Affinity" using https://www.thewindowsclub.com/processor-affinity-windows
255 …pple.com/library/archive/releasenotes/Performance/RN-AffinityAPI/#//apple_ref/doc/uid/TP40006635-C…
259 be enabled by default when building zstd.
260 4. Use a profiler with a good high resolution timer. See the section below on profiling for
268 ### Zstd benchmark
269 The fastest signal you can get regarding your performance changes is via the in-build zstd cli
270 bench option. You can run Zstd as you typically would for your scenario using some set of options
271 and then additionally also specify the `-b#` option. Doing this will run our benchmarking pipeline
275 For example: say you have made a change that you believe improves the speed of zstd level 1. The
277 is `zstd -b`. You might try to do something like this. Note: you can use the `-i` option to
282 $ git checkout <commit-before-your-change>
283 $ make && cp zstd zstd-old
284 $ git checkout <commit-after-your-change>
285 $ make && cp zstd zstd-new
286 $ zstd-old -i5 -b1 <your-test-data>
287 1<your-test-data> : 8990 -> 3992 (2.252), 302.6 MB/s , 626.4 MB/s
288 $ zstd-new -i5 -b1 <your-test-data>
289 1<your-test-data> : 8990 -> 3992 (2.252), 302.8 MB/s , 628.4 MB/s
296 obscured. So unless you see a large performance win (10-15% consistently) using just
305 for level 1 compression on Zstd. Typically this means, you have identified a section of
310 0.0001% of the total running time. Luckily, there are tools to help with this.
319 assembly which most profilers will provide you with.
331 usually be long enough). This way the profiler will have something to work with
335 $ zstd -b1 -i5 <my-data> # this will run for 5 seconds
341 `zstd`
349 zstd and your benchmarking script using debug flags. On mac and linux, this just means
350 you will have to supply the `-g` flag alone with your build script. You might also
351 have to provide the `-fno-omit-frame-pointer` flag
358 This wiki has a pretty detailed tutorial on getting started working with perf so we'll
364 * Use `perf stat -r # <bench-program>` to quickly get some relevant timing and
367 * Perf has a long list of hardware counters that can be viewed with `perf --list`.
370 if you expect the L1 cache misses to decrease with your change, you can look at the
371 counter `L1-dcache-load-misses`
390 The following is a non-exhaustive list of rules employed in zstd code base:
394 with 2 extensions : 64-bit `long long` types, and variadic macros.
396 Sub-project in `contrib/` are allowed to use other conventions.
406 - Reduce dependencies to the minimum possible level.
411 - Within `lib/`, this policy is even more drastic.
416 Other accepted non-symbol headers are `<stddef.h>` and `<limits.h>`.
417 - Within the project, there is a strict hierarchy of dependencies that must be respected.
423 - Functions in `lib/` must use very little stack space,
429 * All public symbols are prefixed with `ZSTD_`
430 + private symbols, with a scope limited to their own unit, are free of this restriction.
437 - `PREFIX_prefix2_camelCase`
438 - `PREFIX_camelCase_extendedQualifier`
439 * Multi-words names generally consist of an action followed by object:
440 - for example : `ZSTD_createCCtx()`
442 - `goBackward` rather than `notGoForward`
455 Any variable that can be `const` (aka. read-only) **must** be `const`.
459 …Conversely, non-const variables are a signal to readers to watch out for modifications later on in…