ddoronin

ddoronin

Trying to understand gc_interval in Nebulex

Hey folks! I’m trying to understand gc_interval in Nebulex.

# GC interval for pushing new generation: 12 hrs
gc_interval: :timer.hours(12),

I know how generations work in .Net and JVM, but I struggle to understand what that parameter means and how I should choose the optimal timeout?

Let’s say I have a cache of short-living objects with TTL 5mins, what is the best gc_interval?
Another case, I cache the HTML file, the TTL is 1 day, what is the best gc_interval?

Marked As Solved

ruslandoga

ruslandoga

I checked nebulex source code, and it’s a purely garbage collection concept. One way to do garbage collection (i.e. deleting stale values from the cache backed by an ets table) is to create a timer for each new entry and delete them when the timer fires. Another is to run a cleanup process that deletes all stale values in one big scan every other interval. And then there is the way nebulex uses, which is to have multiple ets tables and release whole tables when the time comes. So every gc_interval the following happens:

  • tables before garbage collection: [ets1, ets0]
  • gc_interval timer fires, a new ets table is created [ets2, ets1, ets0]
  • oldest table is dropped [ets2, ets1], since nebulex seems to keep at most two tables at a time, all entries of ets0 are now “garbage collected”

So the value of gc_interval seems to depend on how long you can tolerate stale values occupying memory on one hand (so it should be as little as possible after ttl expires) and on the other hand tables shouldn’t be deleted while there are active entries in them so gc_interval >= ttl (you can cache a value right before gc_interval timer fires so lifetime(old_ets) should be at least ttl).


IMHO this approach might be overkill in some scenarios. Personally, I’ve never had issues with “naive” cleanup techniques.

Where Next?

Popular in Questions Top

vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement