This solution I presented is definitely on the naive side. The advantage of the is that it’s easy to grok and debug, but Chris’s points are very true.

I found that the more complex tools were too much for my simple cache case and went with pg2 + cachex. This does handle dynamic node membership because the cache process sends out a request for cache load when it starts. Lots of room for improvement but a decent starting point for the naive solution.

If you have a limited number of nodes, the nodes don’t change that often, it’s really just a cache and you don’t mind lost writes, and you have a limited amount of data to replicate then you can probably do something naive. The most basic would be to listen for node up and down events. when you detect a new node has joined you can replicate your data to them. You can probably use pg2 to maintain a registry of cache processes. You can replicate to those processes and have them write into an ets table or something similar. This approach is very naive and overly chatty. Plus it’ll definitely result in lost writes eventually. But it may be ok for your use case.

If any of those caveats are deal breakers then you may want to use an external process like redis or memcached or pull in a more complete solution like lasp or erleans. Otherwise you’re gunna find yourself solving some pretty gnarly distributed systems problems .

FWIW, I used the Cachex + pg2 solution and had 0 issues with it in production since it shipped. I’ve since left that company, but my understanding is that it’s just churning along doing its thing.

So that’s a good endorsement for me, although it depends on your use case.

I’m talking about 4-6 apps, pods in k8s. Yes, 3rd party/external service for it good solution. But it has to be fast and also we have issues with these external services. For example my use case is about loosing dependency on DB(Postgres) so we can be fully (most important features) available even when DB is not available. We are preparing for new client who will use api and some web endpoints with rate about 300 r/s + other clients. So downtime a few seconds can be huge problem for us. But I don’t have budget to get some huge solution for it. Even, I don’t think I need black box solutions when we have power of BEAM.