harmon25
Can a GenServer state be too "big" and general application architecture
Hoping the old adage that no question is a dumb question holds true with this one…
When building an application that requires maintaining state, should that state be maintained by a single GenServer process? Or should it be split up across multiple processes?
At what point does the state maintained within a single GenServer become too big? Possible?
Does this just boil down to semantics and code organization?
I guess poor performance would be an indicator one might want to split up the state across more processes. Are their more defined best practices in this regard?
Do mnesia or ETS have a role to play in applications which have a lot of state to maintain, but do not require persistence?
Hope the answers to this question will not only help me better understand and build Elixir applications, but other beginners as well!
Thanks all
Most Liked
sasajuric
I wouldn’t say there is one true way. It really varies from case to case, depending on what you want to achieve.
The benefit of keeping the state in a single process is that you have strong consistency. At any point in time, there can be only one process accessing the state. On the flip side, since all the requests are serialized, that process may become a sequential bottleneck if used frequently by many different processes. Another important downside is that you lose the entire state when the process fails. The consequences of a single error might be larger than you want.
Splitting the state/responsibilities over multiple processes give you a reversal of the properties above. There is no strong consistency: you can’t read a frozen snapshot of states from multiple processes, nor can you atomically change states in them. On the upside, you have possible performance improvements, and better failure isolation: if one process crashes, the others are still running, so you get to keep most of your service.
Hence, I’d say that choosing one or the other depends on the problem at hand. The work that needs to be done atomically and consistently should reside in a single process. It’s better off splitting independent tasks into multiple processes, for better error isolation as well as performance and scalability. It’s of course never as simple or as black/white as this, but these are the general guidelines I’d recommend. Occasionally diverging from that path for good reasons is fine.
One point about this:
Does this just boil down to semantics and code organization?
Processes are not the tool for code organization, but rather the organization of runtime. If you need to organize the code, modules are the way to do it. Let’s say for example that the state of your process becomes quite complex, but you still want to keep it in the same process. Then, consider implementing the state in one or more separate, (usually) purely data-oriented modules, and have GenServer functions just use those modules.
JEG2
Because I had to look that up, I though I would shared that it means, “There ain’t no such thing as a free lunch.”
NobbZ
We have a saying in germany, that there are no dumb questions, but only dumb answers ![]()
I think this is very similar to a discussion we recently had in a programming exercise (imperative) about a global super “object”¹ to hold and maintain state or having multiple small distinct global variables. There hasn’t been a real conclusion, but after half an hour everyone decided to use multiple small states, to reduce levels of idirection.
I don’t know about best practices here, I do not have that much battle proof experience with BEAM, but depending on how your super state is structured I do fear, that it will massively influence garbage collection (and generation ;))
Haven’t used mnesia so far, but using :ets in my erlang applications all the time. Reads and writes are much faster than a GenServer keeping a Map as state. Also there are optimisations in :ets for concurrent reads and writes (even interleaved), which you can’t do with a corresponding GenServer and a Map.
¹: Super-sized-struct with accessors would be a better wording here, since we were using C.
rvirding
ETS is very good for managing a large amount of state when you don’t require persistence. The thing to remember is that it is a datastore and not a database so it does not support transactions, or very,very,very limited transactions. Very limited. So if you need to control access to an ETS table then you need to wrap it with a process.
Depending on what you need a GenServer might the right way to go. Seeing you are keeping the data in an ETS table the Genserver process itself will not get very large. This is basically what Mnesia does except that it provides a very large set of features, for example transactions, replication, distribution, persistence. It gives you a distributed ACID database.
TANSTAAFL which means you have to decide what you need and what you are willing to pay for.
One thing to remember with ETS tables is that as they are stored “outside” all processes accessing them means copying data between the process and the table.
Robert
gregvaughn
You had to look it up? Never read Heinlein? Hand in your nerd card. 







