patrickdavey

patrickdavey

Caching strategies (limit by memory usage not number of entries)

I’m coming to Elixir & Phoenix from a Rails background. In Rails it’s pretty common to use Redis for caching, where you just set a memory limit and and allow old entries to be expired.

I was looking at caching in Elixir and it seems like cachex limits are set to the number of entries you allow. This is all fine if the data you’re caching is all about the same size, but, not great (as in our case) if different entries could be extremely different in size.

What are people doing here? Can you create multiple Cachex providers and keep similarish data in each of them (so that you can limit by number correctly), or do tools like Redis and friends still get used?

Most Liked

D4no0

D4no0

You can still use redis for caching here, people use usually cachex to avoid adding external dependencies and having network latency (that a lot of time defeats the propose of having cache in the first place).

As for the limit, since the caching is part of the runtime from where you are running the app (and sharing the memory), I don’t think a feature that would limit memory usage would make much sense, moreover I don’t think this can be reliably measured from within the application code (where cachex operates from).

mudasobwa

mudasobwa

Creator of Cure

It cannot, specifically taking into account that the Erlang VM is smart enough to not duplicate the memory for the same objects.

t1 = {1, 2}
m1 = %{a: t1}
m2 = %{a: t1}
:erts_debug.same(m1.a, m2.a)
#⇒ true
LostKobrakai

LostKobrakai

Also :erts_debug – being a debug tool – doesn’t include data on the shared binary heap as well, being asked for size of a term.

mudasobwa

mudasobwa

Creator of Cure

Answering the question stated, partitioning the data per expected size and starting several Cachex.start/2 instances with different limits, one per each “size range” would do.

Where Next?

Popular in Questions Top

vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
William
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement