highmountaintea
How to write a caching server in Elixir
There is an oracle on the moon, who can give profound answers to all questions. The oracle is all knowing, but requires a minute or two to ponder each question before answering. Luckily he has 4 brains, so he can ponder 4 questions simultaneously. Unfortunately the oracle has very bad memory, so if you ask the same question over and over again, he will have to think about it every time.
How do we write an OracleServer in Elixir that can query the oracle for answers? It needs to do the following:
- allow multiple people to ask questions at once, because the oracle has 4 brains
- cache all the answers, to avoid asking repeat questions
- if person A asks a question, and person B asks the same question while the oracle is still pondering it, OracleServer would smartly avoid sending the same question to the oracle again, but informs both A and B when the answer comes back
- the answer is not always 42
Obviously it’s a thinly disguised way to describe a well known problem. I am familiar with solutions in other languages, but am having trouble figuring out how to do it in Elixir.
Marked As Solved
kartheek
Cachex has get, put, fetch, clear, etc.
-
getreturns value if it is present in cache, nil if its a cache miss -
fetchexecutes fallback function on cache miss.
For example - lets have a fallback function which returns same value after delay of 5 seconds.
iex(1)> import Cachex.Spec
iex(2)> Cachex.start_link(:my_cache, [ fallback: fallback(default: fn x -> :timer.sleep(5000)
...(2)> x end) ])
iex(3)> Cachex.get(:my_cache, 1) #get will return nil as there is no entry
{:ok, nil}
fetch will invoke fallback function when there is a miss - fallback function is executed and value is put into the cache after delay of 5 seconds.
iex(4)> Cachex.fetch(:my_cache, 1) # 5 sec delay due to timer.sleep in fallback function
{:commit, 1}
iex(5)> Cachex.get(:my_cache, 1) # get will return value as it exists in cache
{:ok, 1}
Now fetch will return value from cache without executing fallback function
iex(6)> Cachex.fetch(:my_cache, 1) # returns immediately, fallback not invoked
{:ok, 1}
Also Liked
kartheek
Is this is the same thing you are looking for ?
https://hexdocs.pm/cachex/reactive-warming.html#courier
As of v3, fallbacks changed quite significantly to provide the guarantee that only a single fallback will fire for a given key, even if more processes ask for the same key before the fallback is complete. The internal
Courierservice will queue these requests up, then resolve them all with the results retrieved by the first. This ensures that you don’t have stray processes calling for the same thing (which is especially bad if they’re talking to a database, etc.). You can think of this as a per-key queue at a high level, with a short circuit involved to avoid executing too often.
The new
Courierservice in Cachex v3 will actually queue the second and third calls to fire after the first one, rather than firing them all at once. What’s even better; the moment the first call resolves, the second and third will immediately resolve with the same results. This ensures that your fallback only fires a single time, regardless of the number of processes awaiting the result. This change in behaviour means that the code above would result in"key"having a single value of1as the second and third never fire. Although this results in a behaviour change above, it should basically never affect you in the same way as the code above is deliberately designed to highlight the changes.
Cachex takes care of these things automatically.
axelson
Ah, I meant using the pool in combination with Cachex and not in the worker processes. Although I now realize that was not very clear. The reason that using a pool is important with Cachex for your use-case is that If you’re using the fallback function with Cachex then you could have more than 5 fallback functions running at the same time if you’re trying to fetch more than 5 keys at the same time. By using a pool you can explicitly limit the number of fallback functions that are currently accessing the oracle on the moon.
cmo
You need to keep track of the question and who has asked it. When you have the answer, you send it to the references you kept track of. When a GenServer receives a call, you have the return address. Keep track of it.
awaiting = %{”Why?" => [pid<0.90.0>, pid<0.765.0>]}
Do you want all the questions & answers kept in memory, on disk or in a db?
You’re going to need a worker pool. Do you want to write that too or use a library?
cmo
See how he keeps track of it in this Port example.
highmountaintea
Thanks for your detailed explanations. Yes, this will work for the scenario I am describing.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"