sezaru

sezaru

Process pool with custom availability logic

I have a GenServer (I will call it A) that I connect to a external source (Actually it is a WebSockex process since I connect using websocket) and provide an API to communicate with that source.

A normal API call for A would be something like subscribe_to_channel(channel, callback) which means that it would subscribe that connection to a specific channel and pass all messages from that channel to the callback.

This works great, but the external source limits the number of channel subscriptions to a certain number per connection, let’s say 10. So, after 10 calls to subscribe_to_channel with specific channels, I would need to create a new A process with a new connection and use it to subscribe to new channels.

I was thinking in how to automatize this, what comes to mind is something like poolboy, where I can tell it to initialize like 5 A processes in a pool (and, as result, 5 websocket connections to the external source) and let it handle the subscribe_to_channel calls.

The idea is that A would have another method called available_subscriptions which returns the number of available subscriptions this process can handle. So, for example, if A have 1 channel subscribed, then available_subscriptions will return 9, for 2 channels, it will return 8, and so on.

From that, poolboy could check that function and decide to send the subscribe call to the process in the pool with the least subscriptions currently.

Now, as far as I know, I don’t think I can configure a behavior like that in poolboy, so my question is, is there any library that implements something like that? Or do I need to implement my custom pool with the above described logic?

Thanks!

Marked As Solved

tty

tty

AFAIK with poolboy you are on your own and have to implement your custom pooler. Process poolers and message queues are the two most commonly (re)written applications in Erlang/Elixir. Welcome to the club!

At last count we have 4 different implementations of process pooling in our production code. Yes, 4.

Where Next?

Popular in Questions Top

aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement