Fl4m3Ph03n1x

Fl4m3Ph03n1x

Save state in a different proccess?

Background

I was re-reading Elixir in Action yesterday and I came across a curious quote from @sasajuric:

If the code of your error-kernel process is complex, consider splitting it into two processes: one that holds state, and another that does the actual work. The former process then becomes extremely simple and is unlikely to crash, whereas the worker process can be removed from the error kernel (because it no longer maintains critical state).

Separating state into different processes

I get that by doing this you would gain error isolation - if the worker blows up, you don’t lose your state. Great, right?

But why use a clumsy, boilerplaty, GenServer for the task? I see some issues with this approach:

  1. The State process can still crash
  2. Communication with a GenServer is slow compared with using an ETS table
  3. Your GenServer state process is a single point of access (meaning it is a bottleneck) and a single point of failure.

Is this approach really viable?

I don’t think I have ever seen a case where this makes sense. What I have seen, is people saving state in an ETS table which then has a dump_process that writes the table’s content into disk every X minutes.

This approach is considerably faster than using a GenServer and works wonderfully with concurrent accesses.

Question

  1. So, why on earth would I save my critical state in a GenServer process? (examples would be appreciated)
  2. Isn’t using ETS tables a lot better?

disclaimer: the author may have given this hint because he only introduces ETS tables in a later chapter.

Marked As Solved

sasajuric

sasajuric

Author of Elixir In Action

The main reason why I didn’t mention ETS tables is indeed because they have not yet been introduced in the book at that point.

Using ETS table to store the snapshot of the state is definitely a valid option. If there’s no other logic in the process, then my goto approach is to use an ETS table, for the same reasons you mention.

That said, ETS tables might not always suffice, simply because they have limited support for atomic, isolated operations. So there might be some cases where the state process might need to be a GenServer, and you’ll need to serialize some actions (sometimes only writes, sometimes everything) through it.

So more generally, the part of the book you quoted is about thinking in terms of isolating error effects (which is the name of the chapter), while keeping the state in ETS tables would fall into a “beyond GenServer” category (which is the name of the next chapter :smile: )

Also Liked

keathley

keathley

Any process can crash in a system. Thats not really the point. The point is that a simple KV store in a GenServer would be the opposite of “clumsy”. It would be so simple that the likelihood of crashing is very low.

I agree with your points about ETS allowing for more performance long term. But there’s overhead in having ETS tables lying around so if you don’t need it then its waste. The idea that always using ets is just better than using a genserver isn’t a solid general rule.

In practice we tend to use a combination of the two. A GenServer will start an ets table and process all writes to that ets table. That way writes are applied serially. However all reading is done in the client process. As an aside, this is why the “boilerplate” of a GenServer is useful. All of this logic is hidden behind that “boilerplate” and is transparent to the end user of the api.

Neither situation is that difficult to test if you want to test in isolation. The api can take an optional name argument and look up the process and ets tables with that name. In production you would use the default name. Thats pattern tends to be less egregious than using something like a mock.

LostKobrakai

LostKobrakai

You’re not wrong, but adding ets to the mix is also one more moving piece in your system and it’s more complex than e.g. a simple Agent, which basically does nothing but read state. Not to mention the write side if you need that as well.

Fl4m3Ph03n1x

Fl4m3Ph03n1x

So, assuming that my state has a low number of accesses per second, and that using an ETS would provide me with the same failsafe resilience as using a process, I would still be better off using an ETS because it would allow me to easily get concurrent access in the future without ever worrying with bottlenecks.

My point is, if you are going to separate state from work, do it with an ETS table as it grants you more benefits on the long term (meaning you already have concurrent access to state when needed).

dimitarvp

dimitarvp

As an extra – and more recent – example, @gyson created Ane which combines :ets and Erlang’s recent :atomics module. Haven’t had a chance to use it yet but it looks like a solid compromise between different tradeoffs with a lot of speed and code readability to be gained.

Other options I’d consider:

As others have said, the chance of such simple processes crashing is pretty low – unless you cache things that rely on volatile external APIs or resources (and even then defensive coding can and will save you).

As for caching libraries and primitives, nowadays we have more choice. Evaluate your tradeoffs and take a pick.

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

We're in Beta

About us Mission Statement