ACID & strict serializability versus best effort on a per request basis

[dumblob]

Motivation

Nowadays many setups do actually require a seamless combination of "ACID" key-value pairs and "best effort" key-value pairs (imagine analogy TCP packets and UDP packets treatment) in the same distributed namespace/storage (allowing e.g. transactions over a mix of "ACID" and "best effort" k-v pairs). The goal is to achieve the very minimal possible latency for the "best effort" k-v pairs and strict serializability and full durability for all other k-v pairs.

A naive DB interface for DB clients would then support a "tag" to distinguish between an "ACID" request on a key-value pair (or on a set of pairs) and a "best effort" request on a key-value pair (or on a set of pairs)?

For requests tagged with "best effort", the DB would e.g. ignore some ACID invariants or in the very worst case drop the whole request.

Question: Is it possible to achieve similar behavior with the code from master? How?

[erthink]

Briefly:

1. MDBX always provide serializability. This can't altered or undone, but you could implement in-process batching for "best effort" requests, and "nested write transactions" feature could be useful for such cases.
2. MDBX affords wait-free for readers with read-committed isolation. This can't altered or undone, but you could implement in-process dirty reads (i.e. uncomited) for "best effort" requests.
3. Only "D" (i.e. the durability) in "ACID" could be relaxed by using MDBX API in a regular way. Please see SYNC MODES for more information.

[dumblob]

Thanks! That makes it clear where to draw the line.

3. Only "D" (i.e. the durability) in "ACID" could be relaxed by using MDBX API in a regular way. Please see SYNC MODES for more information.

I saw that but was unable to extract the information whether I can change sync mode "any time" (especially after the DB was opened or a transaction started etc.) - i.e. per-request and if so, whether there are any side effects (like poorer performance due to some caches issues or other inefficiencies).

Could you maybe comment on this as well?

[erthink]

I saw that but was unable to extract the information whether I can change sync mode "any time" (especially after the DB was opened or a transaction started etc.) - i.e. per-request and if so, whether there are any side effects (like poorer performance due to some caches issues or other inefficiencies).

By mdbx_txn_begin_ex() you able to relax durability to MDBX_NOMETASYNC or MDBX_SAFE_NOSYNC for any non-nested write transaction.

[dumblob]

By mdbx_txn_begin_ex() you able to relax durability to MDBX_NOMETASYNC or MDBX_SAFE_NOSYNC for any non-nested write transaction.

Does this affect other threads & processes which run their read transactions at the same time with different MDBX...SYNC options than the current writer?

It shouldn't but with vast majority of DBs always trying to trade safety for performance I can never be sure 😉.

[erthink]

Does this affect other threads & processes which run their read transactions at the same time with different MDBX...SYNC options than the current writer?

No affects, but in case of system/power failure you may lose all non-sycn'ed (unsteady) changes/transactions.

Nonetheless, please read all about the MDBX_SAFE_NOSYNC with references, at least three times.