darkmarmot

darkmarmot

Garbage Collection and memory implications of ETS?

The documentation that I’ve been able to find notes that ETS tables are not garbage collected.

I realize that to full free the memory I’d need to delete the table.

But, if I perform many updates on keys in the tables, does that generate additional garbage that remains until the full table is deleted – such that I’d need to occasionally create a copy of an existing table and flip to it?

Side question: since data is copied out of ETS into the requesting process, I assume I should make sure that any data stored in ETS is small at the key level? (as in, I shouldn’t store a 10MB map against a single ETS key).

Thanks,
Scott S.

Most Liked

sasajuric

sasajuric

Author of Elixir In Action

No. Any data which is not needed anymore due to update/delete of table rows will be immediately released. This is why ETS can in some cases work better than regular data structures, because it doesn’t put any pressure on the garbage collector, and can therefore give you a more consistent latency (i.e. less variations due to unexpected GCs). I blogged about that technique here.

In general I’d recommend carefully choosing keys, and not storing needless data there, precisely because of the reason you mention. More generally, ETS or not, I wouldn’t expect a lookup by a 10MB key to perform well :slight_smile: As usual, measuring is the best way to verify if performance is satisfactory :slight_smile:

darkmarmot

darkmarmot

Thanks for the great explanation!!!

I was considering using ETS tables to implement something like in-memory indexes to other data – and wanted to make sure I wasn’t going to get into too much trouble :slight_smile:

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
Kagamiiiii
Student & New to elixir. Nice language. I want to convert a english character, e.g. “a”, which is stored in a variable, to it’s asci...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

We're in Beta

About us Mission Statement