Function zstd_sys::ZDICT_finalizeDictionary
source · pub unsafe extern "C" fn ZDICT_finalizeDictionary(
dstDictBuffer: *mut c_void,
maxDictSize: usize,
dictContent: *const c_void,
dictContentSize: usize,
samplesBuffer: *const c_void,
samplesSizes: *const usize,
nbSamples: c_uint,
parameters: ZDICT_params_t,
) -> usize
Expand description
ZDICT_finalizeDictionary(): Given a custom content as a basis for dictionary, and a set of samples, finalize dictionary by adding headers and statistics according to the zstd dictionary format.
Samples must be stored concatenated in a flat buffer samplesBuffer
,
supplied with an array of sizes samplesSizes
, providing the size of each
sample in order. The samples are used to construct the statistics, so they
should be representative of what you will compress with this dictionary.
The compression level can be set in parameters
. You should pass the
compression level you expect to use in production. The statistics for each
compression level differ, so tuning the dictionary for the compression level
can help quite a bit.
You can set an explicit dictionary ID in parameters
, or allow us to pick
a random dictionary ID for you, but we can’t guarantee no collisions.
The dstDictBuffer and the dictContent may overlap, and the content will be appended to the end of the header. If the header + the content doesn’t fit in maxDictSize the beginning of the content is truncated to make room, since it is presumed that the most profitable content is at the end of the dictionary, since that is the cheapest to reference.
maxDictSize
must be >= max(dictContentSize, ZSTD_DICTSIZE_MIN).
@return: size of dictionary stored into dstDictBuffer
(<= maxDictSize
),
or an error code, which can be tested by ZDICT_isError().
Note: ZDICT_finalizeDictionary() will push notifications into stderr if
instructed to, using notificationLevel>0.
NOTE: This function currently may fail in several edge cases including:
* Not enough samples
* Samples are uncompressible
* Samples are all exactly the same