hubertlepicki
Soft rate-limiting incoming API requests / connections approach by slowing them down?
Hey all,
I have a slightly unusual problem. One of our clients has a public API, that is built on Elixir / Phoenix / Postgres.
I am thinking of rate-limiting the access to API because we are getting hundreds of requests per second already and a lot of clients seem to have bult very lazy clients that just hit our API every second or so.
While I can fairly easily implement something like plug-attack, which would return 401 / 403 / custom status code instantly, and I actually tried this approach, this fails to prevent attempts to contact the API, and also breaks some of the customers code that consumes the API in a naive loops.
I wonder if I get myself into trouble by implementing a Plug or cowboy handler that pauses instead of returning status code instantly for these abusive clients.
So, when user goes above rate limit, we simply :timer.sleep(10_000) in a Plug or similar.
Has someone here tried such approach, or maybe knows of a tools that I can use in front of my API to facilitate this behavior?
I am slightly worried about doing that on the Elixir side, as this would mean maintaining these connections in memory, and also possibly hitting system limits on open file descriptors etc. But maybe it’d be simply OK.
Marked As Solved
rjk
I think it’s perfectly fine to return 429 status codes if it’s really excessive from their side. It sort of should always be in place if you want to make sure you stay up for all your clients. It’s the only official back pressure you can use towards (REST) API clients.
That said, if you want to try out delaying the requests I agree it’s not smart to do it within elixir/BEAM space as you will have to tweak and monitor a lot before you’re really sure it works as intended. (You will drain max conns and tcp somaxconn buffers quite fast if you’ve got a fast client) At the same time it’s a very implicit way to deal with things and therefore very hard to debug for both sides (later on).
If you want to experiment with delays or other forms of rate limiting I would look into something like the following:
But I would still advise to go the 429 status route because it’s well known and very explicit.
Goodluck
Also Liked
al2o3cr
This is going to tie up a Cowboy acceptor process; if enough requests come in that trip the ratelimit, you’ll eventually end up with EVERY acceptor sleeping and everything else waiting.
The customer’s code is already broken, this is just making it more apparent. If they’re requesting data over the network, they need to be prepared to handle network failures (or at least retry).
chasers
Yeah I think your concerns are valid here. In general it sounds like not such a good idea. This is odd behavior, and not explicit. You’re going to be getting clients who wonder wtf is going on when your API is super slow.
IMO, do it like you normally would and fix the your clients client. Or just make it so you can set each client at a different rate limit.
l00ker
Here’s a link to a blog post by Chris McCord in 2017 where he discusses building API rate limiting via ETS. You might find some useful ideas there as well.
Popular in Questions
Other popular topics
Latest Phoenix Threads
Latest on Elixir Forum
Sponsor Spotlight
Enabling companies to succeed by building software people love.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Courses that'll move you from confusion to "Aha, now I get it!"