This would be might be quite difficult with sticky sessions since some of your “hard-core” users might end up on a single node, whereas other nodes might stay idle. Like with Justin Bieber and twitter. Imagine you are twitter and several celebrities end up on a single node, how would you spread that kind of load?

EDIT However, now that I’ve thought about it more, I probably confuse your approach with consistent hashing. So you would still be able to redistribute the load after an inefficient assignment of a process to a node, whereas with consistent hashing it’s “written in stone”.

An example: In a system where users play together in a game (say: poker), it would make sense to distribute these games, rather than the individual users.

I wonder how discord servers work. Probably the same way, judging by their libraries and blog posts.

The other main thing I am unsure about here is how the supervision would look/work (Because we want a ‘pool’ of processes where members of these pool all live at different nodes).

I think this would be too difficult to be useful. We can’t possibly (although there are ways, but they are too difficult and probably require specialized hardware) know if the process is timing out or the packet is dropped. Erlang type of supervision should be constrained within a single node, I think, since it seems to have been designed with that in mind.

Have you looked at GitHub - erleans/erleans: Erlang Orleans? It might have some of the properties you want.

One way to do this is at the DNS level. When the initial request comes in to www.example.com, we assign it to a pool of servers, www1.example.com, www2.example.com, … and redirect.
Subsequent requests from the same user will go to the same subdomain. That keeps the traffic sticky to one server. Another option is to do customer-specific subdomains, e.g. foo.example.com, which also handle the case of some customers having more traffic than others. We can grow and shrink the pool of physical machines by updating the DNS, e.g. we start with 100 “shards” and assign them to 20 physical machines.

Another option is the “distributed load balancer” approach, i.e. every front end machine can take traffic from any user, but we proxy back to the user’s home machine. We can set a cookie and use that to direct traffic on subsequent requests.

None of this is Elixir specific, but there are interesting tools to share info in the cluster.

Indeed. Instead of consistent hashing, what process is on what node can simply be looked up in the local process registry, which means that processes can be moved around.

Yes, this seems to be the case, sort of. I read this very good blog article about what problems the encountered while attempting to scale to serving many users. Indeed, a Discord Server is actually a single GenServer, but if there are multiple tens of thousand people connecting to the same one, their socket processes might actually be managed by a different node. To limit inter-node chatter they introduced the Manifold library.

Actually, the ‘Sticky Process’ idea is not really new, and is what everyone has to do when building a stateful system (a game, a chatroom, etc): In these cases, we have one single GenServer per ‘unit of concurrency’. What makes it more difficult though, is what happens when you change to a multi-node system (either for scalability or for fault-tolerancy):

• How do we make sure the sticky process (preferably including its current state) lives on when the node it was hosted on dies?
• How do we redistribute work properly when a node dies? And what when the node comes back?
• It is quite possible that we might want to persist-and-terminate a sticky process when it hasn’t been used for a while. (And how to bring it back up later?)

Erleans definitely looks like it is actually attempting to handle this (or at least the last two points), although its documentation is pretty lackluster at this moment, so I would actually not be sure how to get started with using it.

Erleans definitely looks like it is actually attempting to handle this (or at least the last two points), although its documentation is pretty lackluster at this moment, so I would actually not be sure how to get started with using it.

Maybe Orleans is a cross-platform framework for building robust, scalable distributed applications | Microsoft Orleans Documentation would provide more information. It would at least explain their vocabulary: grains, silos, etc. Erleans tries to adapt orleans to erlang/otp, as far as I understand.