alexgleason
Pre-generating user feeds in a Twitter-like microblogging platform
Hey guys, I’m developing Pleroma and preparing it to scale. Of course the bottleneck is not Elixir, but the Postgres database.
It runs an SQL query to generate each user’s home feed, and this query is expensive and slow.
In the Ruby world, Mastodon solves this problem with Redis. A background worker pre-generates feeds for each user and stores them in memory, ready to be served at any time.
I’m wondering what your opinions are on options in Elixir. Is it reasonable to use ETS to solve this problem?
First Post!
dimitarvp
It definitely is but it really depends on the shape of the data and how quickly does it expire and it’s especially dependent on how well can you segment this data – I mean in terms of unique caching keys.
Have you looked at Cachex?
If you are thinking about having multi-node caches I feel you’d be going against the wind here and maybe should just use Redis indeed.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"