zstd_sys

Function ZDICT_finalizeDictionary

source
pub unsafe extern "C" fn ZDICT_finalizeDictionary(
    dstDictBuffer: *mut c_void,
    maxDictSize: usize,
    dictContent: *const c_void,
    dictContentSize: usize,
    samplesBuffer: *const c_void,
    samplesSizes: *const usize,
    nbSamples: c_uint,
    parameters: ZDICT_params_t,
) -> usize
Expand description

ZDICT_finalizeDictionary(): Given a custom content as a basis for dictionary, and a set of samples, finalize dictionary by adding headers and statistics according to the zstd dictionary format.

Samples must be stored concatenated in a flat buffer samplesBuffer, supplied with an array of sizes samplesSizes, providing the size of each sample in order. The samples are used to construct the statistics, so they should be representative of what you will compress with this dictionary.

The compression level can be set in parameters. You should pass the compression level you expect to use in production. The statistics for each compression level differ, so tuning the dictionary for the compression level can help quite a bit.

You can set an explicit dictionary ID in parameters, or allow us to pick a random dictionary ID for you, but we can’t guarantee no collisions.

The dstDictBuffer and the dictContent may overlap, and the content will be appended to the end of the header. If the header + the content doesn’t fit in maxDictSize the beginning of the content is truncated to make room, since it is presumed that the most profitable content is at the end of the dictionary, since that is the cheapest to reference.

maxDictSize must be >= max(dictContentSize, ZSTD_DICTSIZE_MIN).

@return: size of dictionary stored into dstDictBuffer (<= maxDictSize), or an error code, which can be tested by ZDICT_isError(). Note: ZDICT_finalizeDictionary() will push notifications into stderr if instructed to, using notificationLevel>0. NOTE: This function currently may fail in several edge cases including: * Not enough samples * Samples are uncompressible * Samples are all exactly the same