• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1<html>
2<head>
3<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
4<title>1.8.3 Manual</title>
5</head>
6<body>
7<h1>1.8.3 Manual</h1>
8<hr>
9<a name="Contents"></a><h2>Contents</h2>
10<ol>
11<li><a href="#Chapter1">Introduction</a></li>
12<li><a href="#Chapter2">Version</a></li>
13<li><a href="#Chapter3">Tuning parameter</a></li>
14<li><a href="#Chapter4">Simple Functions</a></li>
15<li><a href="#Chapter5">Advanced Functions</a></li>
16<li><a href="#Chapter6">Streaming Compression Functions</a></li>
17<li><a href="#Chapter7">Streaming Decompression Functions</a></li>
18<li><a href="#Chapter8">Unstable declarations</a></li>
19<li><a href="#Chapter9">Private definitions</a></li>
20<li><a href="#Chapter10">Obsolete Functions</a></li>
21</ol>
22<hr>
23<a name="Chapter1"></a><h2>Introduction</h2><pre>
24  LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core,
25  scalable with multi-cores CPU. It features an extremely fast decoder, with speed in
26  multiple GB/s per core, typically reaching RAM speed limits on multi-core systems.
27
28  The LZ4 compression library provides in-memory compression and decompression functions.
29  Compression can be done in:
30    - a single step (described as Simple Functions)
31    - a single step, reusing a context (described in Advanced Functions)
32    - unbounded multiple steps (described as Streaming compression)
33
34  lz4.h provides block compression functions. It gives full buffer control to user.
35  Decompressing an lz4-compressed block also requires metadata (such as compressed size).
36  Each application is free to encode such metadata in whichever way it wants.
37
38  An additional format, called LZ4 frame specification (doc/lz4_Frame_format.md),
39  take care of encoding standard metadata alongside LZ4-compressed blocks.
40  If your application requires interoperability, it's recommended to use it.
41  A library is provided to take care of it, see lz4frame.h.
42<BR></pre>
43
44<a name="Chapter2"></a><h2>Version</h2><pre></pre>
45
46<pre><b>int LZ4_versionNumber (void);  </b>/**< library version number; useful to check dll version */<b>
47</b></pre><BR>
48<pre><b>const char* LZ4_versionString (void);   </b>/**< library version string; unseful to check dll version */<b>
49</b></pre><BR>
50<a name="Chapter3"></a><h2>Tuning parameter</h2><pre></pre>
51
52<pre><b>#ifndef LZ4_MEMORY_USAGE
53# define LZ4_MEMORY_USAGE 14
54#endif
55</b><p> Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.)
56 Increasing memory usage improves compression ratio
57 Reduced memory usage may improve speed, thanks to cache effect
58 Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache
59
60</p></pre><BR>
61
62<a name="Chapter4"></a><h2>Simple Functions</h2><pre></pre>
63
64<pre><b>int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity);
65</b><p>    Compresses 'srcSize' bytes from buffer 'src'
66    into already allocated 'dst' buffer of size 'dstCapacity'.
67    Compression is guaranteed to succeed if 'dstCapacity' >= LZ4_compressBound(srcSize).
68    It also runs faster, so it's a recommended setting.
69    If the function cannot compress 'src' into a more limited 'dst' budget,
70    compression stops *immediately*, and the function result is zero.
71    Note : as a consequence, 'dst' content is not valid.
72    Note 2 : This function is protected against buffer overflow scenarios (never writes outside 'dst' buffer, nor read outside 'source' buffer).
73        srcSize : max supported value is LZ4_MAX_INPUT_SIZE.
74        dstCapacity : size of buffer 'dst' (which must be already allocated)
75        return  : the number of bytes written into buffer 'dst' (necessarily <= dstCapacity)
76                  or 0 if compression fails
77</p></pre><BR>
78
79<pre><b>int LZ4_decompress_safe (const char* src, char* dst, int compressedSize, int dstCapacity);
80</b><p>    compressedSize : is the exact complete size of the compressed block.
81    dstCapacity : is the size of destination buffer, which must be already allocated.
82    return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
83             If destination buffer is not large enough, decoding will stop and output an error code (negative value).
84             If the source stream is detected malformed, the function will stop decoding and return a negative result.
85             This function is protected against malicious data packets.
86</p></pre><BR>
87
88<a name="Chapter5"></a><h2>Advanced Functions</h2><pre></pre>
89
90<pre><b>int LZ4_compressBound(int inputSize);
91</b><p>    Provides the maximum size that LZ4 compression may output in a "worst case" scenario (input data not compressible)
92    This function is primarily useful for memory allocation purposes (destination buffer size).
93    Macro LZ4_COMPRESSBOUND() is also provided for compilation-time evaluation (stack memory allocation for example).
94    Note that LZ4_compress_default() compresses faster when dstCapacity is >= LZ4_compressBound(srcSize)
95        inputSize  : max supported value is LZ4_MAX_INPUT_SIZE
96        return : maximum output size in a "worst case" scenario
97              or 0, if input size is incorrect (too large or negative)
98</p></pre><BR>
99
100<pre><b>int LZ4_compress_fast (const char* src, char* dst, int srcSize, int dstCapacity, int acceleration);
101</b><p>    Same as LZ4_compress_default(), but allows selection of "acceleration" factor.
102    The larger the acceleration value, the faster the algorithm, but also the lesser the compression.
103    It's a trade-off. It can be fine tuned, with each successive value providing roughly +~3% to speed.
104    An acceleration value of "1" is the same as regular LZ4_compress_default()
105    Values <= 0 will be replaced by ACCELERATION_DEFAULT (currently == 1, see lz4.c).
106</p></pre><BR>
107
108<pre><b>int LZ4_sizeofState(void);
109int LZ4_compress_fast_extState (void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration);
110</b><p>    Same compression function, just using an externally allocated memory space to store compression state.
111    Use LZ4_sizeofState() to know how much memory must be allocated,
112    and allocate it on 8-bytes boundaries (using malloc() typically).
113    Then, provide this buffer as 'void* state' to compression function.
114</p></pre><BR>
115
116<pre><b>int LZ4_compress_destSize (const char* src, char* dst, int* srcSizePtr, int targetDstSize);
117</b><p>  Reverse the logic : compresses as much data as possible from 'src' buffer
118  into already allocated buffer 'dst', of size >= 'targetDestSize'.
119  This function either compresses the entire 'src' content into 'dst' if it's large enough,
120  or fill 'dst' buffer completely with as much data as possible from 'src'.
121  note: acceleration parameter is fixed to "default".
122
123 *srcSizePtr : will be modified to indicate how many bytes where read from 'src' to fill 'dst'.
124               New value is necessarily <= input value.
125 @return : Nb bytes written into 'dst' (necessarily <= targetDestSize)
126           or 0 if compression fails.
127</p></pre><BR>
128
129<pre><b>int LZ4_decompress_fast (const char* src, char* dst, int originalSize);
130</b><p>  This function used to be a bit faster than LZ4_decompress_safe(),
131  though situation has changed in recent versions,
132  and now `LZ4_decompress_safe()` can be as fast and sometimes faster than `LZ4_decompress_fast()`.
133  Moreover, LZ4_decompress_fast() is not protected vs malformed input, as it doesn't perform full validation of compressed data.
134  As a consequence, this function is no longer recommended, and may be deprecated in future versions.
135  It's only remaining specificity is that it can decompress data without knowing its compressed size.
136
137  originalSize : is the uncompressed size to regenerate.
138                 `dst` must be already allocated, its size must be >= 'originalSize' bytes.
139 @return : number of bytes read from source buffer (== compressed size).
140           If the source stream is detected malformed, the function stops decoding and returns a negative result.
141  note : This function requires uncompressed originalSize to be known in advance.
142         The function never writes past the output buffer.
143         However, since it doesn't know its 'src' size, it may read past the intended input.
144         Also, because match offsets are not validated during decoding,
145         reads from 'src' may underflow.
146         Use this function in trusted environment **only**.
147
148</p></pre><BR>
149
150<pre><b>int LZ4_decompress_safe_partial (const char* src, char* dst, int srcSize, int targetOutputSize, int dstCapacity);
151</b><p>  Decompress an LZ4 compressed block, of size 'srcSize' at position 'src',
152  into destination buffer 'dst' of size 'dstCapacity'.
153  Up to 'targetOutputSize' bytes will be decoded.
154  The function stops decoding on reaching this objective,
155  which can boost performance when only the beginning of a block is required.
156
157 @return : the number of bytes decoded in `dst` (necessarily <= dstCapacity)
158           If source stream is detected malformed, function returns a negative result.
159
160  Note : @return can be < targetOutputSize, if compressed block contains less data.
161
162  Note 2 : this function features 2 parameters, targetOutputSize and dstCapacity,
163           and expects targetOutputSize <= dstCapacity.
164           It effectively stops decoding on reaching targetOutputSize,
165           so dstCapacity is kind of redundant.
166           This is because in a previous version of this function,
167           decoding operation would not "break" a sequence in the middle.
168           As a consequence, there was no guarantee that decoding would stop at exactly targetOutputSize,
169           it could write more bytes, though only up to dstCapacity.
170           Some "margin" used to be required for this operation to work properly.
171           This is no longer necessary.
172           The function nonetheless keeps its signature, in an effort to not break API.
173
174</p></pre><BR>
175
176<a name="Chapter6"></a><h2>Streaming Compression Functions</h2><pre></pre>
177
178<pre><b>LZ4_stream_t* LZ4_createStream(void);
179int           LZ4_freeStream (LZ4_stream_t* streamPtr);
180</b><p>  LZ4_createStream() will allocate and initialize an `LZ4_stream_t` structure.
181  LZ4_freeStream() releases its memory.
182
183</p></pre><BR>
184
185<pre><b>void LZ4_resetStream (LZ4_stream_t* streamPtr);
186</b><p>  An LZ4_stream_t structure can be allocated once and re-used multiple times.
187  Use this function to start compressing a new stream.
188
189</p></pre><BR>
190
191<pre><b>int LZ4_loadDict (LZ4_stream_t* streamPtr, const char* dictionary, int dictSize);
192</b><p>  Use this function to load a static dictionary into LZ4_stream_t.
193  Any previous data will be forgotten, only 'dictionary' will remain in memory.
194  Loading a size of 0 is allowed, and is the same as reset.
195 @return : dictionary size, in bytes (necessarily <= 64 KB)
196
197</p></pre><BR>
198
199<pre><b>int LZ4_compress_fast_continue (LZ4_stream_t* streamPtr, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration);
200</b><p>  Compress 'src' content using data from previously compressed blocks, for better compression ratio.
201  'dst' buffer must be already allocated.
202  If dstCapacity >= LZ4_compressBound(srcSize), compression is guaranteed to succeed, and runs faster.
203
204 @return : size of compressed block
205           or 0 if there is an error (typically, cannot fit into 'dst').
206
207  Note 1 : Each invocation to LZ4_compress_fast_continue() generates a new block.
208           Each block has precise boundaries.
209           It's not possible to append blocks together and expect a single invocation of LZ4_decompress_*() to decompress them together.
210           Each block must be decompressed separately, calling LZ4_decompress_*() with associated metadata.
211
212  Note 2 : The previous 64KB of source data is __assumed__ to remain present, unmodified, at same address in memory!
213
214  Note 3 : When input is structured as a double-buffer, each buffer can have any size, including < 64 KB.
215           Make sure that buffers are separated, by at least one byte.
216           This construction ensures that each block only depends on previous block.
217
218  Note 4 : If input buffer is a ring-buffer, it can have any size, including < 64 KB.
219
220  Note 5 : After an error, the stream status is invalid, it can only be reset or freed.
221
222</p></pre><BR>
223
224<pre><b>int LZ4_saveDict (LZ4_stream_t* streamPtr, char* safeBuffer, int maxDictSize);
225</b><p>  If last 64KB data cannot be guaranteed to remain available at its current memory location,
226  save it into a safer place (char* safeBuffer).
227  This is schematically equivalent to a memcpy() followed by LZ4_loadDict(),
228  but is much faster, because LZ4_saveDict() doesn't need to rebuild tables.
229 @return : saved dictionary size in bytes (necessarily <= maxDictSize), or 0 if error.
230
231</p></pre><BR>
232
233<a name="Chapter7"></a><h2>Streaming Decompression Functions</h2><pre>  Bufferless synchronous API
234<BR></pre>
235
236<pre><b>LZ4_streamDecode_t* LZ4_createStreamDecode(void);
237int                 LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream);
238</b><p>  creation / destruction of streaming decompression tracking context.
239  A tracking context can be re-used multiple times.
240
241</p></pre><BR>
242
243<pre><b>int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char* dictionary, int dictSize);
244</b><p>  An LZ4_streamDecode_t context can be allocated once and re-used multiple times.
245  Use this function to start decompression of a new stream of blocks.
246  A dictionary can optionally be set. Use NULL or size 0 for a reset order.
247  Dictionary is presumed stable : it must remain accessible and unmodified during next decompression.
248 @return : 1 if OK, 0 if error
249
250</p></pre><BR>
251
252<pre><b>int LZ4_decoderRingBufferSize(int maxBlockSize);
253#define LZ4_DECODER_RING_BUFFER_SIZE(mbs) (65536 + 14 + (mbs))  </b>/* for static allocation; mbs presumed valid */<b>
254</b><p>  Note : in a ring buffer scenario (optional),
255  blocks are presumed decompressed next to each other
256  up to the moment there is not enough remaining space for next block (remainingSize < maxBlockSize),
257  at which stage it resumes from beginning of ring buffer.
258  When setting such a ring buffer for streaming decompression,
259  provides the minimum size of this ring buffer
260  to be compatible with any source respecting maxBlockSize condition.
261 @return : minimum ring buffer size,
262           or 0 if there is an error (invalid maxBlockSize).
263
264</p></pre><BR>
265
266<pre><b>int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* src, char* dst, int srcSize, int dstCapacity);
267int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* src, char* dst, int originalSize);
268</b><p>  These decoding functions allow decompression of consecutive blocks in "streaming" mode.
269  A block is an unsplittable entity, it must be presented entirely to a decompression function.
270  Decompression functions only accepts one block at a time.
271  The last 64KB of previously decoded data *must* remain available and unmodified at the memory position where they were decoded.
272  If less than 64KB of data has been decoded, all the data must be present.
273
274  Special : if decompression side sets a ring buffer, it must respect one of the following conditions :
275  - Decompression buffer size is _at least_ LZ4_decoderRingBufferSize(maxBlockSize).
276    maxBlockSize is the maximum size of any single block. It can have any value > 16 bytes.
277    In which case, encoding and decoding buffers do not need to be synchronized.
278    Actually, data can be produced by any source compliant with LZ4 format specification, and respecting maxBlockSize.
279  - Synchronized mode :
280    Decompression buffer size is _exactly_ the same as compression buffer size,
281    and follows exactly same update rule (block boundaries at same positions),
282    and decoding function is provided with exact decompressed size of each block (exception for last block of the stream),
283    _then_ decoding & encoding ring buffer can have any size, including small ones ( < 64 KB).
284  - Decompression buffer is larger than encoding buffer, by a minimum of maxBlockSize more bytes.
285    In which case, encoding and decoding buffers do not need to be synchronized,
286    and encoding ring buffer can have any size, including small ones ( < 64 KB).
287
288  Whenever these conditions are not possible,
289  save the last 64KB of decoded data into a safe buffer where it can't be modified during decompression,
290  then indicate where this data is saved using LZ4_setStreamDecode(), before decompressing next block.
291</p></pre><BR>
292
293<pre><b>int LZ4_decompress_safe_usingDict (const char* src, char* dst, int srcSize, int dstCapcity, const char* dictStart, int dictSize);
294int LZ4_decompress_fast_usingDict (const char* src, char* dst, int originalSize, const char* dictStart, int dictSize);
295</b><p>  These decoding functions work the same as
296  a combination of LZ4_setStreamDecode() followed by LZ4_decompress_*_continue()
297  They are stand-alone, and don't need an LZ4_streamDecode_t structure.
298  Dictionary is presumed stable : it must remain accessible and unmodified during next decompression.
299
300</p></pre><BR>
301
302<a name="Chapter8"></a><h2>Unstable declarations</h2><pre>
303 Declarations in this section should be considered unstable.
304 Use at your own peril, etc., etc.
305 They may be removed in the future.
306 Their signatures may change.
307<BR></pre>
308
309<pre><b>void LZ4_resetStream_fast (LZ4_stream_t* streamPtr);
310</b><p>  Use this, like LZ4_resetStream(), to prepare a context for a new chain of
311  calls to a streaming API (e.g., LZ4_compress_fast_continue()).
312
313  Note:
314  Using this in advance of a non- streaming-compression function is redundant,
315  and potentially bad for performance, since they all perform their own custom
316  reset internally.
317
318  Differences from LZ4_resetStream():
319  When an LZ4_stream_t is known to be in a internally coherent state,
320  it can often be prepared for a new compression with almost no work, only
321  sometimes falling back to the full, expensive reset that is always required
322  when the stream is in an indeterminate state (i.e., the reset performed by
323  LZ4_resetStream()).
324
325  LZ4_streams are guaranteed to be in a valid state when:
326  - returned from LZ4_createStream()
327  - reset by LZ4_resetStream()
328  - memset(stream, 0, sizeof(LZ4_stream_t)), though this is discouraged
329  - the stream was in a valid state and was reset by LZ4_resetStream_fast()
330  - the stream was in a valid state and was then used in any compression call
331    that returned success
332  - the stream was in an indeterminate state and was used in a compression
333    call that fully reset the state (e.g., LZ4_compress_fast_extState()) and
334    that returned success
335
336  When a stream isn't known to be in a valid state, it is not safe to pass to
337  any fastReset or streaming function. It must first be cleansed by the full
338  LZ4_resetStream().
339
340</p></pre><BR>
341
342<pre><b>int LZ4_compress_fast_extState_fastReset (void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration);
343</b><p>  A variant of LZ4_compress_fast_extState().
344
345  Using this variant avoids an expensive initialization step. It is only safe
346  to call if the state buffer is known to be correctly initialized already
347  (see above comment on LZ4_resetStream_fast() for a definition of "correctly
348  initialized"). From a high level, the difference is that this function
349  initializes the provided state with a call to something like
350  LZ4_resetStream_fast() while LZ4_compress_fast_extState() starts with a
351  call to LZ4_resetStream().
352
353</p></pre><BR>
354
355<pre><b>void LZ4_attach_dictionary(LZ4_stream_t *working_stream, const LZ4_stream_t *dictionary_stream);
356</b><p>  This is an experimental API that allows for the efficient use of a
357  static dictionary many times.
358
359  Rather than re-loading the dictionary buffer into a working context before
360  each compression, or copying a pre-loaded dictionary's LZ4_stream_t into a
361  working LZ4_stream_t, this function introduces a no-copy setup mechanism,
362  in which the working stream references the dictionary stream in-place.
363
364  Several assumptions are made about the state of the dictionary stream.
365  Currently, only streams which have been prepared by LZ4_loadDict() should
366  be expected to work.
367
368  Alternatively, the provided dictionary stream pointer may be NULL, in which
369  case any existing dictionary stream is unset.
370
371  If a dictionary is provided, it replaces any pre-existing stream history.
372  The dictionary contents are the only history that can be referenced and
373  logically immediately precede the data compressed in the first subsequent
374  compression call.
375
376  The dictionary will only remain attached to the working stream through the
377  first compression call, at the end of which it is cleared. The dictionary
378  stream (and source buffer) must remain in-place / accessible / unchanged
379  through the completion of the first compression call on the stream.
380
381</p></pre><BR>
382
383<a name="Chapter9"></a><h2>Private definitions</h2><pre>
384 Do not use these definitions.
385 They are exposed to allow static allocation of `LZ4_stream_t` and `LZ4_streamDecode_t`.
386 Using these definitions will expose code to API and/or ABI break in future versions of the library.
387<BR></pre>
388
389<pre><b>typedef struct {
390    const uint8_t* externalDict;
391    size_t extDictSize;
392    const uint8_t* prefixEnd;
393    size_t prefixSize;
394} LZ4_streamDecode_t_internal;
395</b></pre><BR>
396<pre><b>typedef struct {
397    const unsigned char* externalDict;
398    size_t extDictSize;
399    const unsigned char* prefixEnd;
400    size_t prefixSize;
401} LZ4_streamDecode_t_internal;
402</b></pre><BR>
403<pre><b>#define LZ4_STREAMSIZE_U64 ((1 << (LZ4_MEMORY_USAGE-3)) + 4)
404#define LZ4_STREAMSIZE     (LZ4_STREAMSIZE_U64 * sizeof(unsigned long long))
405union LZ4_stream_u {
406    unsigned long long table[LZ4_STREAMSIZE_U64];
407    LZ4_stream_t_internal internal_donotuse;
408} ;  </b>/* previously typedef'd to LZ4_stream_t */<b>
409</b><p> information structure to track an LZ4 stream.
410 init this structure before first use.
411 note : only use in association with static linking !
412        this definition is not API/ABI safe,
413        it may change in a future version !
414
415</p></pre><BR>
416
417<pre><b>#define LZ4_STREAMDECODESIZE_U64  4
418#define LZ4_STREAMDECODESIZE     (LZ4_STREAMDECODESIZE_U64 * sizeof(unsigned long long))
419union LZ4_streamDecode_u {
420    unsigned long long table[LZ4_STREAMDECODESIZE_U64];
421    LZ4_streamDecode_t_internal internal_donotuse;
422} ;   </b>/* previously typedef'd to LZ4_streamDecode_t */<b>
423</b><p> information structure to track an LZ4 stream during decompression.
424 init this structure  using LZ4_setStreamDecode (or memset()) before first use
425 note : only use in association with static linking !
426        this definition is not API/ABI safe,
427        and may change in a future version !
428
429</p></pre><BR>
430
431<a name="Chapter10"></a><h2>Obsolete Functions</h2><pre></pre>
432
433<pre><b>#ifdef LZ4_DISABLE_DEPRECATE_WARNINGS
434#  define LZ4_DEPRECATED(message)   </b>/* disable deprecation warnings */<b>
435#else
436#  define LZ4_GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
437#  if defined (__cplusplus) && (__cplusplus >= 201402) </b>/* C++14 or greater */<b>
438#    define LZ4_DEPRECATED(message) [[deprecated(message)]]
439#  elif (LZ4_GCC_VERSION >= 405) || defined(__clang__)
440#    define LZ4_DEPRECATED(message) __attribute__((deprecated(message)))
441#  elif (LZ4_GCC_VERSION >= 301)
442#    define LZ4_DEPRECATED(message) __attribute__((deprecated))
443#  elif defined(_MSC_VER)
444#    define LZ4_DEPRECATED(message) __declspec(deprecated(message))
445#  else
446#    pragma message("WARNING: You need to implement LZ4_DEPRECATED for this compiler")
447#    define LZ4_DEPRECATED(message)
448#  endif
449#endif </b>/* LZ4_DISABLE_DEPRECATE_WARNINGS */<b>
450</b><p>   Should deprecation warnings be a problem,
451   it is generally possible to disable them,
452   typically with -Wno-deprecated-declarations for gcc
453   or _CRT_SECURE_NO_WARNINGS in Visual.
454   Otherwise, it's also possible to define LZ4_DISABLE_DEPRECATE_WARNINGS
455</p></pre><BR>
456
457</html>
458</body>
459