_jonas
Using Process.sleep for rate limiting external requests?
Hey I’ve got a quick beginner question:
I’m making requests to an external API using Finch.
The API is rate limited, and when the limit is hit, it returns a 429 response which helpfully comes with a Retry-After header that declares a time in seconds after which the request can be retried.
My naive approach would be to route all the external API calls through a GenServer, which when encountering the rate limit, calls Process.sleep/1 on the value (*1000) returned by the Retry-After header, so that all subsequent requests are queued in the message queue of the GenServer process and successfully complete after the process finishes sleeping and the rate limit is lifted.
I am hesitating because of this line in the Process docs for sleep/1:
Use this function with extreme care. For almost all situations where you would use
sleep/1in Elixir, there is likely a more correct, faster and precise way of achieving the same with message passing.
Is this a situation where sleep/1 would be appropriate or is there indeed a better way?
Marked As Solved
dwark
It’s basically the same thing except that Process.sleep also handles :infinity,
dunno if it’s a common pattern or not.
Your current method will block your GenServer from handling system messages under the hood: your GenServer code is just a part of it.
You probably want a GenServer state with a queue of some sorts (could be a simple list for lifo processing if you don’t care about the order in which they arrive versus are being handled) in combination with a wait flag.
If wait is true, simply prepend the incoming request to the list of pending requests (or use a :queue). If wait is false and queue is empty, process the request by calling dispatch_request/1. If queue is non-empty, queue the request and start processing the queue.
dispatch_request/1 should return either:
{:ok, response}or{:wait, milliseconds}if it sees a 429 response.
In the first case, if queue is non-empty, you process the next entry.
In the latter case, you queue the (failed) request, set wait to true and use Process.send_after/4 to send yourself a :resume message at a later point in time.
Something like that anyway.
Also Liked
dimitarvp
I keep shilling for this technique regularly now: GenServer.reply: Don't Call Us, We'll Call You
You do not block the GenServer while at the same time you make the caller wait. It’s ideal IMO because the caller doesn’t have to invent extra logic to match on a return value and do Process.sleep; this pattern will just block it until the GenServer sends it what it needs (which it will not do immediately due to the rate-limiting constraints).
jswanner
I would not recommend using Process.sleep for this, if for nothing else there are bookkeeping tasks that are done under the hood that rely on message passing and you’re pausing all of that while sleeping.
I would use :queue to enqueue requests you want to have wait (storing the from value and eventually replying with GenServer.reply/2). The calling processes will be blocked the whole time (or until they time out), but you don’t need the GenServer to be sleeping for that
jswanner
For sure, it’s a matter of “quick and dirty” (Process.sleep), or more robust (GenServer.reply). It’s not just about the bookkeeping thing I mentioned, you might also want to introspect this process from time to time, to see how big the queue is or whatever, and you can’t do that if the process is blocking.
Another question: do you need a queue or can you load shed (drop requests)?
arcanemachine
I’m guessing by the lack of responses that this is one of those valid use cases.
From what I understand about the BEAM (which is not nearly enough…), making a process sleep like this is not a problem from an efficiency standpoint (i.e. the runtime can easily handle it).
I think that warning is to caution against naive time-based timeouts when there is a better way to do that. In this case however, it seems perfectly legitimate (to this newbie, at least) to do what you are doing since 1) there’s no more direct way of doing it, and 2) you know the exact amount of time you need to wait before performing the request again.
dwark
Probably, but I am not sure I would recurse by calling myself again unless its with a flag that says that it’s a retry, in which case it either succeeds or fails. Otherwise you may end up recursing endlessly depending on the responses you get …
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"