https://github.com/martinsumner/leveled was mentioned during CodeBEAM in a talk presenting how Riak is used at NHS. It’s a pure-erlang implementation of a key-value store that is designed to replace eleveldb as a backend for Riak.

For local KV storage I tend to use either DETS (single-load use), mnesia (persistant save/load use), or leveldb (for high speed data that might exceed 2 gigs).

Overall I default to mnesia.

Preference I guess. Mnesia provides more features then either rocksdb or lmdb but also has some issues with file loading times, maximum file sizes, repair times on corrupted files, etc. IIRC Klarna built a backend for mnesia in leveldb (which is what rocksdb is based on) in order to solve those issues. If you just need keys and values on a single file system then rocksdb is what I would use. If you want the extra features of mnesia and you don’t have a ton of data to store then mnesia is probably fine.

Aw thanks! Happy to hear you’re enjoying it.

If you just need a quick lookup for KVs and don’t need persistence then ETS is going to be your best bet. I haven’t benchmarked dets or any other “native” elixir solution vs. the rocksdb nif so I can’t say what the performance implications are. Personally I use rocksdb because its “very fast” and its seen tons of use across many companies and solutions. So I trust it to be reliable.

Well DETS (which is what mnesia is based on) has a max 2gb file size so you’ll have to use mnesia’s table fragmentation anyway in that case.

I’m assuming you mean read performance here. I’m also assuming you’ve tuned your caches, compaction threads and such for this? In my experience you shouldn’t have that much issue tuning reads in the “double digit GB” order of magnitude but obviously you know your use case more then me . I’m also not arguing that you wouldn’t get better read performance from LMDB or any other db based on B+trees .

All of that aside, I don’t know of a KV store written in Elixir that I would trust that hasn’t already been mentioned or isn’t based on an underlying DB engine through NIFs.

Check out ETS, here’s the elixir school page https://elixirschool.com/en/lessons/specifics/ets/