Lines Matching +full:libzstd +full:- +full:size
5 * This source code is licensed under both the BSD-style license (found in the
8 * You may select, at your option, one of the above-listed licenses.
52 * typically with -Wno-deprecated-declarations for gcc or _CRT_SECURE_NO_WARNINGS in Visual.
77 real-time compression scenarios at zlib-level and better compression ratios.
78 The zstd compression library provides in-memory compression and decompression
82 which is currently 22. Levels >= 20, labeled `--ultra`, should be used with
88 - a single step (described as Simple API)
89 - a single step, reusing a context (described as Explicit context)
90 - unbounded multiple steps (described as Streaming compression)
94 - a single step (described as Simple dictionary API)
95 - a single step, reusing a dictionary (described as Bulk-processing
101 Advanced experimental APIs should never be used with a dynamically-linked
106 /*------ Version ------*/
136 /* All magic numbers are supposed read/written to/from files/memory using little-endian convention …
153 * @return : compressed size written into `dst` (<= `dstCapacity),
160 …* `compressedSize` : must be the _exact_ size of some number of compressed and/or skippable frame…
171 * hint : any size >= `ZSTD_frameHeaderSize_max` is large enough.
172 * @return : - decompressed size of `src` frame content, if known
173 * - ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
174 …* - ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too…
176 …* note 2 : decompressed size is an optional field, it may not be present, typically in streaming…
177 * When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
180 * as ZSTD_decompress() only needs an upper bound of decompressed size.
182 …* note 3 : decompressed size is always present when compression is completed using single-pass f…
184 * note 4 : decompressed size can be very large (64-bits value),
187 * note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified.
191 #define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1)
192 #define ZSTD_CONTENTSIZE_ERROR (0ULL - 2)
200 * @return : decompressed size of `src` frame content _if known and not empty_, 0 otherwise. */
207 * `srcSize` must be >= first frame size
208 * @return : the compressed size of the first frame starting at `src`,
216 * maximum compressed size in worst case single-pass scenario.
217 * When invoking `ZSTD_compress()` or any other one-pass compression function,
228 * for example to size a static array on stack.
232 …) ? 0 : (srcSize) + ((srcSize)>>8) + (((srcSize) < (128<<10)) ? (((128<<10) - (srcSize)) >> 11) /*…
233 …_t ZSTD_compressBound(size_t srcSize); /*!< maximum compressed size in worst case single-pass scen…
254 * Note : re-using context is just a speed / resource optimization.
256 * Note 2 : In multi-threaded environments,
305 * __They do not apply to one-shot variants such as ZSTD_compressCCtx()__ .
333 * See ZSTD_CCtx_refCDict() for more info (superseded-by-cdict). */
334 … ZSTD_c_compressionLevel=100, /* Set compression parameters according to pre-defined cLevel table.
347 ZSTD_c_windowLog=101, /* Maximum allowed back-reference distance, expressed as power of 2.
354 … * requires explicitly allowing such size at streaming decompression stage. */
355 ZSTD_c_hashLog=102, /* Size of the initial probe table, as a power of 2.
361 ZSTD_c_chainLog=103, /* Size of the multi-probe search table, as a power of 2.
373 ZSTD_c_minMatch=105, /* Minimum size of searched matches.
374 * Note that Zstandard can still find matches of smaller size,
375 … * it just tweaks its search algorithm to look for this size and larger.
395 … * Attempts to fit compressed block size into approximatively targetCBlockSize.
399 … * This is helpful in low bandwidth streaming environments to improve end-to-end latency,
410 * It increases memory usage and window size.
415 ZSTD_c_ldmHashLog=161, /* Size of the table for long distance matching, as a power of 2.
419 * default: windowlog - 7.
421 ZSTD_c_ldmMinMatch=162, /* Minimum match size for long distance matcher.
425 …ZSTD_c_ldmBucketSizeLog=163, /* Log size of each bucket in the LDM hash table for collision resolu…
430 … * Must be clamped between 0 and (ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN).
431 … * Default is MAX(0, (windowLog - ldmHashLog)), optimizing hash table usage.
437 …ZSTD_c_contentSizeFlag=200, /* Content size will be written into frame header _whenever known_ (de…
438 * Content size must be known at the beginning of compression.
440 … * For streaming scenarios, content size must be provided with ZSTD_CCtx_setPledgedSrcSize() */
441 …ZSTD_c_checksumFlag=201, /* A 32-bits checksum of content is written at end of frame (default:0) */
444 /* multi-threading parameters */
445 …/* These parameters are only active if multi-threading is enabled (compiled with build macro ZSTD_…
446 … * Otherwise, trying to set any other value than default (0) will be a no-op and return an error.
447 * In a situation where it's unknown if the linked library supports multi-threading or not,
457 … * Default value is `0`, aka "single-threaded mode" : no worker is spawned,
459 …ZSTD_c_jobSize=401, /* Size of a compression job. This value is enforced only when nbWorkers …
462 …* Job size must be a minimum of overlap size, or ZSTDMT_JOBSIZE_MIN (= 512 KB), whichever is large…
463 * The minimum size is automatically and transparently enforced. */
464 ZSTD_c_overlapLog=402, /* Control the overlap size, as a fraction of window size.
465 … * The overlap size is an amount of data reloaded from previous job at the beginning of a new job.
470 … * - 0 means "default" : value will be determined by the library, depending on strategy
471 * - 1 means "no overlap"
472 * - 9 means "full overlap", using a full window size.
473 * Each intermediate rank increases/decreases load size by a factor 2 :
531 * - an error status field, which must be tested using ZSTD_isError()
532 * - lower and upper bounds, both inclusive
541 * Exception : when using multi-threading mode (nbWorkers >= 1),
550 * Total input data size to be compressed as a single frame.
555 * In order to mean "unknown content size", pass constant ZSTD_CONTENTSIZE_UNKNOWN.
574 * - The session : will stop compressing current frame, and make CCtx ready to start a new one.
580 * - The parameters : changes all parameters back to "default".
584 * - Both : similar to resetting the session, followed by resetting parameters.
593 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
594 * - The function is always blocking, returns when compression is completed.
597 * @return : compressed size written into `dst` (<= `dstCapacity),
619 ZSTD_d_windowLogMax=100, /* Select a size limit (in power of 2) beyond which
622 …ameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
651 * - an error status field, which must be tested using ZSTD_isError()
652 * - both lower and upper bounds, inclusive
680 size_t size; /**< size of input buffer */ member
681 … pos; /**< position where reading stopped. Will be updated. Necessarily 0 <= pos <= size */
686 size_t size; /**< size of output buffer */ member
687 … pos; /**< position where writing stopped. Will be updated. Necessarily 0 <= pos <= size */
692 /*-***********************************************************************
693 * Streaming compression - HowTo
698 * It is recommended to reuse ZSTD_CStream since it will play nicer with system's memory, by re-usi…
709 * set more specific parameters, the pledged source size, or load a dictionary.
715 * the output buffer is already full, in which case `input.pos < input.size`.
728 * using ZSTD_compressStream2() with ZSTD_e_flush. `output->pos` will be updated.
729 * Note that, if `output->size` is too small, a single invocation with ZSTD_e_flush might not be en…
736 …some data still present within internal buffer (the value is minimal estimation of remaining size),
748 …some data still present within internal buffer (the value is minimal estimation of remaining size),
775 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
776 …* - Compression parameters cannot be changed once compression is started (save a list of exceptio…
777 * - output->pos must be <= dstCapacity, input->pos must be <= srcSize
778 …* - output->pos and input->pos will be updated. They are guaranteed to remain below their respect…
779 * - endOp must be a valid directive
780 …* - When nbWorkers==0 (default), function is blocking : it completes its job before returning to …
781 …* - When nbWorkers>=1, function is non-blocking : it copies a portion of input, distributes jobs …
784 …* - Exception : if the first call requests a ZSTD_e_end directive and provides enough dstCapacity…
785 * - @return provides a minimum amount of data remaining to be flushed from internal buffers
790 * - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
794 * - note: if an operation ends with an error, it may leave @cctx in an undefined state.
796 * In order to be re-employed after an error, a state must be reset,
807 …* They are not required : ZSTD_compressStream*() happily accepts any buffer size, for both input a…
808 * Respecting the recommended size just makes it a bit easier for ZSTD_compressStream*(),
819 ZSTDLIB_API size_t ZSTD_CStreamInSize(void); /**< recommended size for input buffer */
820 ZSTDLIB_API size_t ZSTD_CStreamOutSize(void); /**< recommended size for output buffer. Guarantee …
843 * the next read size (if non-zero and not an error). ZSTD_compressStream2()
844 * returns the minimum nb of bytes left to flush (if non-zero and not an error).
853 /*-***************************************************************************
854 * Streaming decompression - HowTo
861 * @return : recommended first input size
866 * If `input.pos < input.size`, some input has not been consumed.
868 * The function tries to flush all data decoded immediately, respecting output buffer size.
869 * If `output.pos < output.size`, decoder has flushed everything it could.
870 * But if `output.pos == output.size`, there might be some data left within internal buffers.,
876 * the return value is a suggested next input size (just a hint for b…
877 * that will never request more than the remaining frame size.
902 * - `input.pos < input.size`, some input remaining and caller should provide remaining input
904 * - `output.pos < output.size`, decoder finished and flushed all remaining buffers.
905 * - `output.pos == output.size`, potentially uncflushed data present in the internal buffers,
915 * In order to re-use such a state, it must be first reset,
921 ZSTDLIB_API size_t ZSTD_DStreamInSize(void); /*!< recommended size for input buffer */
922 ZSTDLIB_API size_t ZSTD_DStreamOutSize(void); /*!< recommended size for output buffer. Guarantee …
963 …Dict can be created once and shared by multiple threads concurrently, since its usage is read-only.
1017 * It can still be loaded, but as a content-only dictionary. */
1023 * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */
1029 * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */
1036 * - The frame does not require a dictionary to be decoded (most common case).
1037 …* - The frame was built with dictID intentionally removed. Whatever dictionary is necessary is a …
1038 * Note : this use case also happens when using a non-conformant dictionary.
1039 …* - `srcSize` is too small, and as a result, the frame header could not be decoded (only possible…
1040 * - This is not a Zstandard frame.
1053 * In contrast, Prefixes are single-use.
1061 * Special: Loading a NULL (or 0-size) dictionary invalidates previous dictionary,
1062 * meaning "return to no-dictionary mode".
1067 * It's also a CPU consuming operation, with non-negligible impact on latency.
1085 * The parameters ignored are labelled as "superseded-by-cdict" in the ZSTD_cParameter enum docs.
1086 * The ignored parameters will be used again if the CCtx is returned to no-dictionary mode.
1089 * Special : Referencing a NULL CDict means "return to no-dictionary mode".
1096 * Reference a prefix (single-usage dictionary) for next compressed frame.
1107 * ensure that the window size is large enough to contain the entire source.
1110 * It's a CPU consuming operation, with non-negligible impact on latency.
1122 * Special : Adding a NULL (or 0-size) dictionary invalidates any previous dictionary,
1123 * meaning "return to no-dictionary mode".
1125 * which has a non-negligible impact on CPU usage and latency.
1148 * Special: referencing a NULL DDict means "return to no-dictionary mode".
1154 * Reference a prefix (single-usage dictionary) to decompress next frame.
1220 …) ((format) == ZSTD_f_zstd1 ? 5 : 1) /* minimum input size required to query frame header size */
1236 #define ZSTD_SEARCHLOG_MAX (ZSTD_WINDOWLOG_MAX-1)
1251 … * requiring larger than (1<<ZSTD_WINDOWLOG_LIMIT_DEFAULT) window size,
1254 …* The limit does not apply for one-pass decoders (such as ZSTD_decompress()), since no additional …
1265 #define ZSTD_LDM_HASHRATELOG_MAX (ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN)
1274 /* --- Advanced types --- */
1281 * literals in the block of litLength size.
1300 * rep == 1 --> offset == repeat_offset_1
1301 * rep == 2 --> offset == repeat_offset_2
1302 * rep == 3 --> offset == repeat_offset_3
1304 * rep == 1 --> offset == repeat_offset_2
1305 * rep == 2 --> offset == repeat_offset_3
1306 * rep == 3 --> offset == repeat_offset_1 - 1
1321 …unsigned targetLength; /**< acceptable match size for optimal parser (only) : larger == more co…
1326 int contentSizeFlag; /**< 1: content size will be in frame header (when known) */
1327 …int checksumFlag; /**< 1: generate a 32-bits checksum using XXH64 algorithm at end of frame, fo…
1344 …ZSTD_dlm_byRef = 1 /**< Reference dictionary content -- the dictionary buffer must outlive its …
1349 … ZSTD_f_zstd1_magicless = 1 /* Variant of zstd frame format, without initial 4-bytes magic number.
1374 * - The contents of the CDict can be copied into the working context. This
1377 * the compression faster per-byte of input. However, the initial copy of
1382 * - The CDict's tables can be used in-place. In this model, compression is
1384 * tables. However, this model incurs no start-up cost (as long as the
1388 * - The CDict's tables are not used at all, and instead we use the working
1390 * size. See ZSTD_compress_insertDictionary() and ZSTD_compress_usingDict().
1392 * to the input size, and the input size is fairly large to begin with.
1420 ZSTD_ps_enable = 1, /* Force-enable the feature */
1425 * Frame header and size functions
1430 * `srcSize` must be the _exact_ size of this series
1432 * @return : - decompressed size of all data in all successive frames
1433 * - if the decompressed size cannot be determined: ZSTD_CONTENTSIZE_UNKNOWN
1434 * - if an error occurred: ZSTD_CONTENTSIZE_ERROR
1436 …* note 1 : decompressed size is an optional field, that may not be present, especially in stream…
1437 * When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
1439 * note 2 : decompressed size is always present when compression is done with ZSTD_compress()
1440 * note 3 : decompressed size can be very large (64-bits value),
1443 * note 4 : If source is untrusted, decompressed size could be wrong or intentionally modified.
1453 * `srcSize` must be the _exact_ size of this series
1455 * @return : - upper-bound for the decompressed size of all data in all successive frames
1456 * - if an error occurred: ZSTD_CONTENTSIZE_ERROR
1459 …* note 2 : the upper-bound is exact when the decompressed size field is available in every ZSTD …
1461 …* note 3 : when the decompressed size field isn't available, the upper-bound for that frame is c…
1462 * upper-bound = # blocks * min(128 KB, Window_Size)
1468 * @return : size of the Frame Header,
1477 …ZSTD_frameType_e frameType; /* if == ZSTD_skippableFrame, frameContentSize is the size of…
1497 * Zstd supports in-place decompression, where the input and output buffers overlap.
1506 * |---------------------------------------|-----------|----------|
1511 * NOTE: This applies only to single-pass decompression through ZSTD_decompress() or
1513 * NOTE: This function supports multi-frame input.
1516 * @param srcSize The size of the compressed frame(s)
1523 * the compressed frame, compute it from the original size and the blockSizeLog.
1526 * WARNING: This macro does not support multi-frame input, the input must be a single
1529 * @param originalSize The original uncompressed size of the data.
1530 * @param blockSize The block size == MIN(windowSize, ZSTD_BLOCKSIZE_MAX).
1537 …((originalSize) == 0 ? 0 : 3 * (((originalSize) + (blockSize) - 1) / blockSize)) /* 3 bytes per bl…
1547 * `srcSize` : size of the input buffer
1548 * @return : upper-bound for the number of sequences that can be generated
1551 * note : returns number of sequences - to get bytes, multiply by sizeof(ZSTD_Sequence).
1567 * @param outSeqs The output sequences buffer of size @p outSeqsSize
1568 * @param outSeqsSize The size of the output sequences buffer.
1571 * @param src The source buffer to generate sequences from of size @p srcSize.
1572 * @param srcSize The size of the source buffer.
1612 * the block size derived from the cctx, and sequences may be split. This is the default setting.
1622 …* - ZSTD_c_minMatch MUST be set as less than or equal to the smallest match generated by the ma…
1623 …* - ZSTD_c_compressionLevel accordingly adjusts the strength of the entropy coder, as it would …
1624 …* - ZSTD_c_windowLog affects offset validation: this function will return an error at higher de…
1627 …* Note: Repcodes are, as of now, always re-calculated within this function, so ZSTD_Sequence::rep …
1630 * @return : final compressed size, or a ZSTD error code.
1641 * Skippable frames begin with a 4-byte magic number. There are 16 possible choices of magic number,
1646 …* Returns an error if destination buffer is not large enough, if the source size is not representa…
1647 …* with a 4-byte unsigned int, or if the parameter magicVariant is greater than 15 (and therefore i…
1658 * i.e. magicNumber - ZSTD_MAGIC_SKIPPABLE_START. This can be NULL if the caller is not interested
1671 ZSTDLIB_API unsigned ZSTD_isSkippableFrame(const void* buffer, size_t size);
1686 * to compress data of any size using one-shot compression ZSTD_compressCCtx() or ZSTD_compress2()
1691 * Note that the size estimation is specific for one-shot compression,
1703 * Note : only single-threaded compression is supported.
1714 * It will also consider src size to be arbitrarily "large", which is a worst case scenario.
1717 …ams() can be used in tandem with ZSTD_CCtxParams_setParameter(). Only single-threaded compression …
1718 * Note : CStream size estimation is only correct for single-threaded compression.
1720 …* Note 2 : ZSTD_estimateCStreamSize* functions are not compatible with the Block-Level Sequence P…
1721 * Size estimates assume that no external sequence producer is registered.
1723 * ZSTD_DStream memory budget depends on frame's window Size.
1726 * Any frame requesting a window size larger than max specified one will be rejected.
1728 * an internal ?Dict will be created, which additional size is not estimated here.
1729 * In this case, get total size by adding ZSTD_estimate?DictSize
1738 …* ZSTD_estimateCDictSize() will bet that src size is relatively "small", and content is copied, l…
1747 * Initialize an object using a pre-allocated fixed-size buffer.
1749 * Provided pointer *must be 8-bytes aligned*.
1751 * workspaceSize: Use ZSTD_estimate*Size() to determine
1754 * or NULL if error (size too small, incorrect alignment, etc.)
1764 * Limitation 2 : static cctx currently not compatible with multi-threading.
1792 typedef void* (*ZSTD_allocFunction) (void* opaque, size_t size);
1863 …* same as ZSTD_getCParams(), but @return a full `ZSTD_parameters` object instead of sub-component…
1949 * The target average block size is ZSTD_c_jobSize / 2.
1950 * It's possible to modify the job size to increase or decrease
1952 * Once the jobSize is smaller than the window size,
1969 /* Force back-reference distances to remain < windowSize,
1986 * literals compression based on the compression parameters - specifically,
1991 /* User's best guess of source size.
1993 * There is no guarantee that hint is close to actual source size,
2027 * structure for the dictionary that is read-optimized.
2047 * In general, you should expect compression to be faster--sometimes very much
2048 * so--and CDict creation to be slightly slower. Eventually, we will probably
2061 * However, it's possible to increase the @size field,
2075 * this flag is ALWAYS memory safe, and will never access out-of-bounds
2091 * calls. Specifically: (out.size - out.pos) will never grow. This gives the
2101 * Zstd will check that (out.size - out.pos) never grows and return an error
2148 * Set to ZSTD_ps_disable to never use row-based matchfinder.
2149 * Set to ZSTD_ps_enable to force usage of row-based matchfinder.
2152 * the row-based matchfinder based on support for SIMD instructions and the window log.
2164 * results than when the two buffers are non-contiguous. This flag forces zstd
2165 * to always load the prefix in non-contiguous mode, even if it happens to be
2180 * In some situations, zstd uses CDict tables in-place rather than copying them
2184 * when they are used in-place.
2194 * but in the future zstd may conditionally enable this feature via an auto-detection
2205 * is block-by-block: the internal sequence producer will only be called for blocks
2208 * normal (fully-internal) compression operation.
2210 * The user is strongly encouraged to read the full Block-Level Sequence Producer API
2228 * provided through the compressSequences() API or from an external block-level
2260 * - ZSTD_createCCtxParams() : Create a ZSTD_CCtx_params structure
2261 * - ZSTD_CCtxParams_setParameter() : Push parameters one by one into
2265 * - ZSTD_CCtx_setParametersUsingCCtxParams() : Apply parameters to
2269 * - ZSTD_compressStream2() : Do compression using the CCtx.
2270 * - ZSTD_freeCCtxParams() : Free the memory, accept NULL pointer.
2273 * for static allocation of CCtx for single-threaded compression.
2341 * Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always be 0.
2344 ZSTDLIB_STATIC_API unsigned ZSTD_isFrame(const void* buffer, size_t size);
2373 …* Refuses allocating internal buffers for frames requiring a window size larger than provided lim…
2375 …ameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
2402 * checked when the frame content size is known. The data in the ZSTD_outBuffer
2410 * If you need to avoid the input buffer allocation use the buffer-less
2414 * this flag is ALWAYS memory safe, and will never access out-of-bounds
2474 * Forces the decompressor to reject blocks whose content size is
2490 * This instruction is mandatory to decode data without a fully-formed header,
2527 * "0" also disables frame content size field. It may be enabled in the future.
2613 * This is typically useful to skip dictionary loading stage, since it will reuse it in-place.
2639 * Note : (ingested - consumed) is amount of input data buffered internally, not yet compressed.
2695 /* ********************* BLOCK-LEVEL SEQUENCE PRODUCER API *********************
2698 * The Block-Level Sequence Producer API allows users to provide their own custom
2699 * sequence producer which libzstd invokes to process each block. The produced list
2700 * of sequences (literals and matches) is then post-processed by libzstd to produce
2703 * This block-level offload API is a more granular complement of the existing
2704 * frame-level offload API compressSequences() (introduced in v1.5.1). It offers
2705 * an easier migration story for applications already integrated with libzstd: the
2711 * acceleration not available within libzstd itself.
2714 * Block-Level Sequence Producer API.
2719 * arguments to the user-provided function:
2721 * - sequenceProducerState: a pointer to a user-managed state for the sequence
2724 * - outSeqs, outSeqsCapacity: an output buffer for the sequence producer.
2728 * - src, srcSize: an input buffer for the sequence producer to parse.
2731 * - dict, dictSize: a history buffer, which may be empty, which the sequence
2736 * - compressionLevel: a signed integer representing the zstd compression level
2742 * - windowSize: a size_t representing the maximum allowed offset for external
2747 * The user-provided function shall return a size_t representing the number of
2749 * code if it is greater than outSeqsCapacity. The return value must be non-zero
2750 * if srcSize is non-zero. The ZSTD_SEQUENCE_PRODUCER_ERROR macro is provided
2754 * If the user-provided function does not return an error code, the sequences
2758 * - The sum of matchLengths and literalLengths must equal srcSize.
2759 * - All sequences in the parse, except for the final sequence, must have
2762 * - All offsets must respect the windowSize parameter as specified in
2764 * - If the final sequence has matchLength == 0, it must also have offset == 0.
2770 * If the user-provided function returns an error, zstd will either fall back
2792 * COMPRESSION WILL FAIL if it is enabled and the user tries to compress with a block-level
2794 * - Note that ZSTD_c_enableLongDistanceMatching is auto-enabled by default in some
2798 * - As of this writing, ZSTD_c_enableLongDistanceMatching is disabled by default
2800 * check the docs on ZSTD_c_enableLongDistanceMatching whenever the Block-Level Sequence
2805 * - Dictionaries are not currently supported. Compression will *not* fail if the user
2807 * - Stream history is not currently supported. All advanced compression APIs, including
2811 * Third, multi-threading within a single compression is not currently supported. In other words,
2813 * Multi-threading across compressions is fine: simply create one CCtx per thread.
2815 * Long-term, we plan to overcome all three limitations. There is no technical blocker to
2819 #define ZSTD_SEQUENCE_PRODUCER_ERROR ((size_t)(-1))
2831 * Instruct zstd to use a block-level external sequence producer function.
2858 * This is used for accurate size estimation with ZSTD_estimateCCtxSize_usingCCtxParams(),
2874 * Buffer-less and synchronous inner streaming functions (DEPRECATED)
2888 Buffer-less streaming compression (synchronous mode)
2899 - ZSTD_compressContinue() has no internal buffer. It uses externally provided buffers only.
2900 - Interface is synchronous : input is consumed entirely and produces 1+ compressed blocks.
2901 …- Caller must ensure there is enough space in `dst` to store compressed data under worst case scen…
2904 …- ZSTD_compressContinue() presumes prior input ***is still accessible and unmodified*** (up to max…
2906 …- ZSTD_compressContinue() detects that prior input has been overwritten when `src` buffer overlaps.
2916 /*===== Buffer-less streaming compression functions =====*/
2917 ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
2919 ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
2921 ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
2928 ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
2930 ZSTD_DEPRECATED("The buffer-less API is deprecated in favor of the normal streaming API. See docs.")
2941 Buffer-less streaming decompression (synchronous mode)
2956 such as the dictionary ID, content size, or maximum back-reference distance (`windowSize`).
2966 …or that previous contiguous segment is large enough to properly handle maximum back-reference dist…
2969 The most memory efficient way is to use a round buffer of sufficient size.
2970 Sufficient size is determined by invoking ZSTD_decodingBufferSize_min(),
2971 …which can return an error code if required value is too large for current system (in 32-bits mode).
2974 which maximum size is provided in `ZSTD_frameHeader` structure, field `blockSizeMax`.
2978 …There are alternatives possible, for example using two or more buffers of size `windowSize` each, …
2980 Finally, if you control the compression process, you can also ignore all buffer size rules,
2981 as long as the encoder and decoder progress in "lock-step",
3003 Skippable frames allow integration of user-defined data into a flow of concatenated frames.
3006 a) Skippable frame ID - 4 Bytes, Little endian format, any value from 0x184D2A50 to 0x184D2A5F
3007 b) Frame Size - 4 Bytes, Little endian format, unsigned 32-bits
3008 c) Frame Content - any content (User Data) of length equal to Frame Size
3009 For skippable frames ZSTD_getFrameHeader() returns zfhPtr->frameType==ZSTD_skippableFrame.
3013 /*===== Buffer-less streaming decompression functions =====*/
3015 …indowSize, unsigned long long frameContentSize); /**< when frame content size is not known, pass …
3041 - ZSTD_c_format = ZSTD_f_zstd1_magicless
3042 - ZSTD_c_contentSizeFlag = 0
3043 - ZSTD_c_checksumFlag = 0
3044 - ZSTD_c_dictIDFlag = 0
3051 …Frame metadata cost is typically ~12 bytes, which can be non-negligible for very small blocks (< 1…
3055 - Compressing and decompressing require a context structure
3057 - It is necessary to init context before starting
3060 - Block size is limited, it must be <= ZSTD_getBlockSize() <= ZSTD_BLOCKSIZE_MAX == 128 KB
3061 … + If input is larger than a block size, it's necessary to split input data into multiple blocks
3063 …Frame metadata is not that costly, and quickly becomes negligible as source size grows larger than…
3064 …- When a block is considered not compressible enough, ZSTD_compressBlock() result will be 0 (zero)…
3083 …Size); /**< insert uncompressed block into `dctx` history. Useful for multi-blocks decompression.…