ExESDB v0.1.0 available!
I am proud to announce that the first useful release of ExESDB is now available as v0.1.0! This release comes with an adapter for the fantastic Commanded Library AND a Phoenix LiveView demo app. Feel free to check it out

On Hex:

• The BEAM-native Event Store
• ExESDB Commanded Adapter
• ExESDB Gater

On Github

• ExESDB Store
• ExESDB Commanded
• ExESDB Gater

DEMO Video
Comments and Feedback most welcome!

Hi Eiji,

It has taken me a few months to finish things up (but then, what is ‘finished’, right?) but meanwhile, the distributed store is there, a HA proxy is there and…the Commanded adapter is there, too!
It is said that the proof of the pudding is in the eating, so the adapter is being developed and tested in conjunction with a working Phoenix LiveView application: an event-sourced demo app that mimics a dashboard for regulating greenhouses. All data (events) is stored in ExESDB, while Cachex is used for read models, thus creating a true BEAM-only application without the need of any external services like EventStoreDB or PostgreSQL.

I must confess, it feels a little liberating, being able to just spin up a few containers and not having to worry about connectivity with other backend components.

Also, since we remain in the same ecosystem, there is no need to worry about (or waste processing power with) serialization. Serialization will only become a topic if and when we decide to create API’s for clients in other languages..

Hi Tomasz,
Thank you for your feedback.
Next to the advantages you already mentioned (speed, security, no need for extra serialization - events are indeed stored as Erlang terms), I’d like to add the capability to deploy event sourced services as a self-contained BEAM-native release to the edge. This would allow us to leverage for instance The Nerves Project for deployment.
I am also looking in to https://bondy.io for scenarios where you could have 10E+N nodes in the network.
Much of my past work revolved around decentralized and autonomous systems (think Parking Facilities, Vehicles, Agricultural automation, Logistics etc), often in a “spotty” environment, where nodes aren’t always connected. Such systems benefit little from SaaS solutions, if the network is not available. That space could be considered my main motivator to build a BEAM-native event store: as few dependencies on 3rd party services as possible.

The reason for implementing the Commanded Adapter is simple: it is the de-facto event sourcing standard for the BEAM and is as far as I am concerned, feature complete.

When it comes to migrating data from existing stores, I’d argue that’s quite easy, barely an inconvenience: replay the old store and project into the new.
So, indeed: highest points on the agenda are:

1, have Kherpi triggers throw seen events on pg2 (for projections etc…)
2. Dynamic Clustering via Partisan
3. Commanded Adapter
4. Monitoring/Telemetry
…

This seems cool - love seeing more database projects in Elixir!

A couple of random questions from someone who knows very little about CQRS/ES:

Khepri, like Mnesia, is an in-memory database (which also persists to disk). If you’re storing an immutable log, would you eventually run out of memory? Or is the log truncated at some point?

Khepri, as I understand it, is a K/V store built on top of a Raft log. Since the thing you’re storing is, of course, a log, would it make more sense to use Ra directly?

I see you mentioned a large number of nodes. Is the idea here to have many individual Khepri clusters running independently within a large cluster?

What is Khepri’s throughput like? I would imagine you would get better results with aggressive batching.

Thank you for your input, those are some very valid points and concerns.
The main driver for this project is decentralization and in such scenarios, I’d imagine JIT availability and localized sharding functions as a counterweight to throughput. For now, I don’t worry about this too much yet and focus on getting the store operational.
Most of the dedicated Event Stores (I know of) are centralized at the data center level and there is not much literature about decentralized event sourcing. It probably opens a whole different can of worms, but we need tooling to investigate it. ExESDB should be seen in this context.

As an example, imagine the scenario of a parking facility where vehicles enter and exit, people enter and exit, payments are made etc…next, imagine such a facility not being managed by a centralized system, but rather by a mesh of SBCs that perform individual parts of the process. Such a system might consist of a few hundred devices, and indeed, in that mesh there might be a number of individual realms that are responsible for parts of the process.