victorolinasc
Constant CPU cost algorithm
Hi everybody!
I have an expensive CPU action that I want to serve through a public internet endpoint. It doesn’t matter what it does, just that it usually takes some time (seconds) and that it costs CPU (say a few percent of CPU usage). For a real use-case you can think of an expensive hashing algorithm like PBKDF or any other on a signin endpoint.
What I want to avoid is being hit by a flood of requests and suffer from resource starvation where everything would become unresponsive. Like the signin case, a public endpoint without authentication makes it a perfect target for attacks that want to bring the server down. Think that all network solutions are in place like rate-limiting, WAFs and so on.
I thought about 2 strategies in the BEAM:
1- Using a pool of processes. I’ve implemented this with poolboy and works fine but is hard to tune it. I have to benchmark the cost of the function in a production server and reach a pool size that will be a good enough “sharing” of resources. I am thinking about switching to wpool and having a bigger than needed pool with a callback module that would check CPU usage before dispatching to the pool. This seems to me very unreliable and prone to error… If we get several concurrent requests and the CPU usage from all of these is under control, then they would all start at the same time and the CPU would spike anyway…
2 - Using a slave node for doing just this operation and fight with the emulator flags to have it use other logical cores. Suppose I have 8 cpus available, I could dedicate 2 or 3 to the slave node and the rest to the main system. Though, with this strategy, it would still be vulnerable to resource starvation and make all calls to it fail which is not my intention here. I’d rather have timeouts than a denial of service.
So, I’d like to know if there are any other algorithms or strategies to deal with this. I appreciate your time 
Most Liked
benwilson512
Have you thought about measures to prevent clients from misbehaving? For example with public signin pages, rate limits or tools like ReCaptcha can limit the impact an individual client can have.
chasers
Ah … yeah I had an API product for years that was designed to be async. So people would POST jobs with a callback URL. We’d hit that with results, or with an endpoint to gather results when the job was finished. It worked well and people understood why we had to do that based on the type of work we were doing.
Other than that, I would limit concurrency with a pool. So the whole system can have N number of concurrent jobs. You don’t even need poolboy for this … just a Dynamic Supervisor limiting the number of children.
I personally wouldn’t try and base it off of CPU. I’m sure it can be done I guess but I’d start with concurrency limiting I think to play with that. Maybe then have a failsafe based on CPU if you want to push the boundaries a bit more.
vfsoraki
How much time can a user wait for his/her response? I mean if it takes 10 seconds normally for a process to finish, can the user wait like 10 minutes for it? Is it crucial for the system to do this calculation in a synchronized manner for the system to do other jobs?
I mean, if a user can wait a long time, then maybe you can queue these jobs and process them with a job processor.
This way, if the system is flooded, only the queue grows, CPU is not maxed out. You can monitor the queue length to see if your system is doing fine, or it is lagging behind. If you notice the system is not handling the load, then maybe you can scale your job processors for a limited time to clear the queue or scale your system to handle more load. You can also notify your users that you are under load. Maybe you cal also analyze your queue and remove some malicious jobs, if there are any.
You can do more fine-tuning too. Having multiple queues (high/low priorities), different job processors for them and somewhat easily horizontal scaling are to name a few.
There are a number of queue job processing libraries available for Elixir, a simple search reveals them.
derek-zhou
If the job takes seconds to complete in the fast path, then you can consider to use a job queue system like:
Let the general public to use a normal queue and paying customers to use a higher priority queue. This can scale up to many nodes.
victorolinasc
Awesome! Thanks a lot for the info! Will try to play with uwieger/jobs A very interesting library to have under my set of tools.
Thank you all for your replies. We have tested the process pool approach and it is handling it pretty good up to now. Though the subject keeps interesting me and I will try other approaches.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"