Key-Value Store (KVS)

Data Structures

	Name
struct	hse_kvs Opaque structure, a pointer to which is a handle to an HSE KVS within a KVDB.

Functions

	Name
hse_err_t	hse_kvdb_kvs_close(struct hse_kvs * kvs) Close an open KVS.
hse_err_t	hse_kvdb_kvs_create(struct hse_kvdb * kvdb, const char * kvs_name, size_t paramc, const char const paramv) Create a KVS within the referenced KVDB.
hse_err_t	hse_kvdb_kvs_drop(struct hse_kvdb * kvdb, const char * kvs_name) Drop a KVS from the referenced KVDB.
hse_err_t	hse_kvdb_kvs_open(struct hse_kvdb * kvdb, const char * kvs_name, const size_t paramc, const char const paramv, struct hse_kvs* * kvs_out) Open a KVS in a KVDB.
hse_err_t	hse_kvs_delete(struct hse_kvs * kvs, unsigned int flags, struct hse_kvdb_txn * txn, const void * key, size_t key_len) Delete the key and its associated value from the KVS.
hse_err_t	hse_kvs_get(struct hse_kvs * kvs, unsigned int flags, struct hse_kvdb_txn * txn, const void * key, size_t key_len, bool * found, void * valbuf, size_t valbuf_sz, size_t * val_len) Retrieve the value for a given key from the KVS.
const char *	hse_kvs_name_get(struct hse_kvs * kvs) Get the name of a KVS.
hse_err_t	hse_kvs_param_get(struct hse_kvs * kvs, const char * param, char * buf, size_t buf_sz, size_t * needed_sz) Get KVS parameter.
hse_err_t	hse_kvs_prefix_delete(struct hse_kvs * kvs, unsigned int flags, struct hse_kvdb_txn * txn, const void * pfx, size_t pfx_len) Delete all key-value pairs matching the key prefix from a KVS storing segmented keys.
hse_err_t	hse_kvs_put(struct hse_kvs * kvs, unsigned int flags, struct hse_kvdb_txn * txn, const void * key, size_t key_len, const void * val, size_t val_len) Put a key-value pair into KVS.

Defines

	Name
	HSE_KVS_COUNT_MAX Maximum number of KVSs contained within one KVDB.
	HSE_KVS_KEY_LEN_MAX Maximum key length.
	HSE_KVS_NAME_LEN_MAX Maximum length of a KVS name.
	HSE_KVS_PFX_LEN_MAX Max key prefix length.
	HSE_KVS_VALUE_LEN_MAX Max value length is 1MiB.

Functions Documentation

function hse_kvdb_kvs_close

hse_err_t hse_kvdb_kvs_close(
    struct hse_kvs * kvs
)

Close an open KVS.

Parameters:

kvs KVS handle from hse_kvdb_kvs_open().

Return: Error status.

Note: This function is not thread safe.

Warning: After invoking this function, calling any other KVS functions will result in undefined behavior unless the KVS is re-opened.

function hse_kvdb_kvs_create

hse_err_t hse_kvdb_kvs_create(
    struct hse_kvdb * kvdb,
    const char * kvs_name,
    size_t paramc,
    const char *const * paramv
)

Create a KVS within the referenced KVDB.

Parameters:

kvdb KVDB handle from hse_kvdb_open().
kvs_name KVS name.
paramc Number of configuration parameters in paramv.
paramv List of parameters in key=value format.

Return: Error status.

Remark:

kvdb must not be NULL.
kvs_name must be non-NULL.
kvs_name must be NULL-terminated.
strlen(kvs_name) must be less than HSE_KVS_NAME_LEN_MAX.
kvs_name must match the following pattern: [-_A-Za-z0-9]+.
kvs_name cannot already exist.

function hse_kvdb_kvs_drop

hse_err_t hse_kvdb_kvs_drop(
    struct hse_kvdb * kvdb,
    const char * kvs_name
)

Drop a KVS from the referenced KVDB.

Parameters:

kvdb KVDB handle from hse_kvdb_open().
kvs_name KVS name.

Return: Error status.

Warning: It is an error to call this function on a KVS that is open.

Remark:

kvdb must not be NULL.
kvs_name must not be NULL.
kvs_name must be NULL-terminated.

function hse_kvdb_kvs_open

hse_err_t hse_kvdb_kvs_open(
    struct hse_kvdb * kvdb,
    const char * kvs_name,
    const size_t paramc,
    const char *const * paramv,
    struct hse_kvs ** kvs_out
)

Open a KVS in a KVDB.

Parameters:

kvdb KVDB handle from hse_kvdb_open().
kvs_name KVS name.
paramc Number of configuration parameters in paramv.
paramv List of parameters in key=value format.
kvs_out Handle to access the opened KVS.

Return: Error status.

Remark:

kvdb must not be NULL.
kvs_name must not be NULL.
kvs_name must be NULL-terminated.
kvs_out must not be NULL.

function hse_kvs_delete

hse_err_t hse_kvs_delete(
    struct hse_kvs * kvs,
    unsigned int flags,
    struct hse_kvdb_txn * txn,
    const void * key,
    size_t key_len
)

Delete the key and its associated value from the KVS.

Parameters:

kvs KVS handle from hse_kvdb_kvs_open().
flags Flags for operation specialization.
txn Transaction context (optional).
key Key to be deleted from kvs.
key_len Length of key.

Return: Error status.

Note: This function is thread safe.

Remark:

kvs must not be NULL.
key must not be NULL.
key_len must be within the range of [1, HSE_KVS_KEY_LEN_MAX].

It is not an error if the key does not exist within the KVS. See Transactions for information on how deletes within transactions are handled.

Flags:

0 - Reserved for future use.

function hse_kvs_get

hse_err_t hse_kvs_get(
    struct hse_kvs * kvs,
    unsigned int flags,
    struct hse_kvdb_txn * txn,
    const void * key,
    size_t key_len,
    bool * found,
    void * valbuf,
    size_t valbuf_sz,
    size_t * val_len
)

Retrieve the value for a given key from the KVS.

Parameters:

kvs KVS handle from hse_kvdb_kvs_open().
flags Flags for operation specialization.
txn Transaction context (optional).
key Key to get from kvs.
key_len Length of key.
found Whether or not key was found.
valbuf Buffer into which the value associated with key will be copied (optional).
valbuf_sz Size of valbuf.
val_len Actual length of value if key was found.

Return: Error status.

Note: This function is thread safe.

Remark:

kvs must not be NULL.
key must not be NULL.
key_len must be within the range of [1, HSE_KVS_KEY_LEN_MAX].
found must not be NULL.
val_len must not be NULL.

If the key exists in the KVS, then the referent of found is set to true. If the caller's value buffer is large enough then the data will be returned. Regardless, the actual length of the value is placed in val_len. See Transactions for information on how gets within transactions are handled.

Flags:

0 - Reserved for future use.

function hse_kvs_name_get

const char * hse_kvs_name_get(
    struct hse_kvs * kvs
)

Get the name of a KVS.

Parameters:

kvs KVS handle from hse_kvdb_kvs_open()

Returns:

NULL if given an invalid KVS handle.

Return: KVS name.

Note: This function is thread safe.

Remark: kvs must not be NULL.

function hse_kvs_param_get

hse_err_t hse_kvs_param_get(
    struct hse_kvs * kvs,
    const char * param,
    char * buf,
    size_t buf_sz,
    size_t * needed_sz
)

Get KVS parameter.

Parameters:

kvs KVS handle from hse_kvdb_kvs_open().
param Parameter name.
buf Buffer for writing stringified value of parameter.
buf_sz Size of buf.
needed_sz Needed size of buf.

Return: Error status.

Note: This function is thread safe.

Remark:

kvs must not be NULL.
param must not be NULL.
param must be a valid parameter.

Puts the stringified version of the parameter value into buf. If buf_sz is NULL, then needed_sz will still be populated.

function hse_kvs_prefix_delete

hse_err_t hse_kvs_prefix_delete(
    struct hse_kvs * kvs,
    unsigned int flags,
    struct hse_kvdb_txn * txn,
    const void * pfx,
    size_t pfx_len
)

Delete all key-value pairs matching the key prefix from a KVS storing segmented keys.

Parameters:

kvs KVS handle from hse_kvdb_kvs_open().
flags Flags for operation specialization.
txn Transaction context (optional).
pfx Prefix of keys to delete.
pfx_len Length of pfx, must equal KVS prefix length.

Return: Error status.

Note: This function is thread safe.

Remark: kvs must not be NULL.

This interface is used to delete an entire range of segmented keys. To do this the caller passes a filter with a length equal to the KVS's key prefix length. It is not an error if no keys exist matching the filter. If there is a filtered iteration in progress, then that iteration can fail if hse_kvs_prefix_delete() is called with a filter matching the iteration.

If hse_kvs_prefix_delete() is called from a transaction context, it affects no key-value mutations that are part of the same transaction. Stated differently, for KVS commands issued within a transaction, all calls to hse_kvs_prefix_delete() are treated as though they were issued serially at the beginning of the transaction regardless of the actual order these commands appeared in.

Flags:

0 - Reserved for future use.

function hse_kvs_put

hse_err_t hse_kvs_put(
    struct hse_kvs * kvs,
    unsigned int flags,
    struct hse_kvdb_txn * txn,
    const void * key,
    size_t key_len,
    const void * val,
    size_t val_len
)

Put a key-value pair into KVS.

Parameters:

kvs KVS handle from hse_kvdb_kvs_open().
flags Flags for operation specialization.
txn Transaction context (optional).
key Key to put into kvs.
key_len Length of key.
val Value associated with key (optional).
val_len Length of value.

Return: Error status

Note: This function is thread safe.

Remark:

kvs must not be NULL.
key must not be NULL.
key_len must be within the range of [1, HSE_KVS_KEY_LEN_MAX].
val_len must be within the range of [0, HSE_KVS_VALUE_LEN_MAX].

If the key already exists in the KVS then the value is effectively overwritten. See Transactions for information on how puts within transactions are handled.

The HSE KVDB attempts to maintain reasonable QoS and for high-throughput clients this results in very short sleep's being inserted into the put path. For some kinds of data (e.g., control metadata) the client may wish to not experience that delay. For relatively low data rate uses, the caller can set the HSE_KVS_PUT_PRIO flag for an hse_kvs_put(). Care should be taken when doing so to ensure that the system does not become overrun. As a rough approximation, doing 1M priority puts per second marked as PRIO is likely an issue. On the other hand, doing 1K small puts per second marked as PRIO is almost certainly fine.

If compression is on by default for the given kvs, then hse_kvs_put() will attempt to compress the value unless the HSE_KVS_PUT_VCOMP_OFF flag is given. Otherwise, the HSE_KVS_PUT_VCOMP_OFF flag is ignored.

If compression is off by default for the given kvs, then hse_kvs_put() will not attempt to compress a value unless the HSE_KVS_PUT_VCOMP_ON flag is given. Otherwise, the HSE_KVS_PUT_VCOMP_ON flag is ignored.

Flags:

HSE_KVS_PUT_PRIO - Operation will not be throttled.
HSE_KVS_PUT_VCOMP_OFF - Value will not be compressed.
HSE_KVS_PUT_VCOMP_ON - Value may be compressed.

Macros Documentation

define HSE_KVS_COUNT_MAX

#define HSE_KVS_COUNT_MAX (256)

Maximum number of KVSs contained within one KVDB.

define HSE_KVS_KEY_LEN_MAX

#define HSE_KVS_KEY_LEN_MAX 1344

Maximum key length.

A common requirement clients have for key length is 1024. Combined with a discriminant and (potentially) a chunk key, this pushes us to 1030 bytes keys. Looking at the packing for the on-media format for data, we can have at most 3 keys of such large size in a 4k page. Lopping off 64-bytes for other data, and we can have 3 keys of 1344 bytes.

Keys need not be NULL-terminated.

define HSE_KVS_NAME_LEN_MAX

#define HSE_KVS_NAME_LEN_MAX 32

Maximum length of a KVS name.

KVS names are NULL-terminated strings. The string plus the NULL-terminator must fit into a HSE_KVS_NAME_LEN_MAX byte buffer.

define HSE_KVS_PFX_LEN_MAX

#define HSE_KVS_PFX_LEN_MAX 32

Max key prefix length.

define HSE_KVS_VALUE_LEN_MAX

#define HSE_KVS_VALUE_LEN_MAX (1024 * 1024)

Max value length is 1MiB.

Values need not be NULL-terminated.

Updated on 17 November 2022 at 15:11:02 CST