I’m a little confused with what’s going on in this thread. There’s an implication here that Redis is somehow different than using ETS tables, but there is no reason you can’t just put an ETS table on one node and use it exactly like you would use Redis! You would have no fault tolerance and higher latency, just like Redis. The only reason you would want to actually use Redis is if you wanted its vast array of fancy data structures, which in this case maybe you don’t, seeing as you have already implemented your system on ETS.

Again, it’s perfectly valid to use Redis for its functionality, but “ETS and GenServers” were absolutely designed to be used “cross-node”. OTP literally could not be more designed for that! It’s, like, the entire thing!

This is not a literal answer, but the quintessential example would be something like an exchange/order-matching system: something which processes (financial) transactions. If you want to learn more about that the magic keywords are “LMAX architecture”, plus check out Tigerbeetle for a modern fault-tolerant rendition.

Apologies in advance for nitpicking something you’re probably fully aware of, but CAP and “centralized/distributed” are separate concepts. You can have a distributed consistent system, and you can also have a single-threaded inconsistent system, though this is less common because in a single-threaded system consistency is so easy to achieve.

The big change here is not GenServer vs. Redis though. It’s centralized storage of data (CP) vs. distributed storage of data (eventual consistency / AP)

Your remarks make it sound like there’s no real reason for the number 70 exactly, so I’d suggest not going with a fixed cutoff point, but rather evalute a solution using low and high watermarks. Start creating new channels at a low watermark, but also make it so only the high watermark makes that number of users be a problem in a channel. That way if people are added to a channel after the low watermark is reached isn’t directly a problem. You can tune those numbers against expected latency of coordination between nodes as well as expected arrival rates of users. This makes even more sense if you seem to be moving people around anyways.

The part about people potentially landing in empty channels cannot really be avoided though. There will always be the case of all existing channels being considered full and the new person being put into a new channel. It’s however a matter of how likely you make that case. E.g. if your balancer does its job how likely is it for that case to happen.

I have no idea about scaling persence, but I’m asking because both likely affect performance in distinct ways. There’s no single knob to dialing performance. I’d also argue that performance work is best evaluated by benchmarking your solution besides reading what others have to say.

If you want to use ETS that’s fine, I was just pointing out that Redis is designed to handle what you’re trying to do in a way that ETS and Genservers are not. Your trying to add cross node functionality to tools that are for local use, rather than use a tool that will work cross node by default.

If you were using a sorted set you wouldn’t be making this post because it solves the exact issue your post is about with regards to counting and distributing users to rooms atomically.

Again, you mentioned this above as well. With a sorted set you would just go down the list until you find a vacant spot and then fill it. If no vacant spots exist you create a new room. Sorted sets are ordered by highest counts.

I should also add, one of the reasons I’m pushing Redis so hard is that I was desgning something with ETS/Genservers a year or so ago for a couple of week before I hit a wall and finally tried Redis. Redis made everything 10x easier for me.

Yes.

Redis can give you an atomic, cross‑node count while users join and leave in real time. It won’t match ETS’s raw speed, but building your own synchronization and network‑transfer logic for ETS updates erodes that advantage. At high join/leave volumes, custom distribution code risks race conditions and latency that Redis’s built‑in counters avoid.

You will never get 100%, always correct counts when dealing with hundreds of thousands of users connecting at the same time, but at least with Redis the results will be atomically consistent.

No.

ETS is local, it will always be faster than using an external source.

However, I should also add that although ETS is faster locally you still presumably have to manually process users/counts and send info between nodes. Your custom data distribution logic will not magically be done in an instant, especially at the scale of hundreds of thousands of users constantly joining and leaving.

I should add, your posts “problem” is the below. You have only mentioned speed whilst trying to counter Redis, but I recommended Redis because it answers your posts problem. You’ve changed the argument from “I can’t get the counts correct” to “but it’s gotta be fast”.

I’m not going to reply any further, as I feel I’ve expressed my opinion multiple times at this point and it would be a waste of time for me to continue to push a tool you obviously have no interest in using.

Apologies if I’ve annoyed you by turning this into ETS vs Redis, but I still maintain Redis is the correct tool for the job for track user presence with atomic room counts.