sagar_k317

sagar_k317

What abstraction to use for Producer - consumer model with queue implementation

Hi! I am trying to build a solution and I need guidance as to what type of abstractions to use.

I have a pool of users and operators connected to my Phoenix app via channels. I have entries in my database for each operator and user along with their current state. Now I need to implement a basic FIFO queue of users. Whenever an operator’s state is ‘available’, I need to allocate the first member of the user queue to this operator and pop him out of the queue. The operator is now in a different state and will do some activities with this user through channels. The number of users can be less or more than the agents at any time.
I’d like to know if I should use an external library, like OPQ https://github.com/fredwu/opq, or implement Genstage on my own, or if there’s better solution to this entire problem.
Also, in order to make processing fast, I’m thinking of maintaining states of operators and users using Agent and then asynchronously update in the database.

Any sort of guidance will be really helpful.

Most Liked

axelson

axelson

Scenic Core Team

That’s an interesting problem statement. At first reading it seems like it might be good to model this with processes (using either GenServer or Agent). Can you explain more why you need to persist the operators state in the database? Could that instead be stored entirely in memory?

Based on my understanding of your description, OPQ doesn’t seem like a good fit because it simply has generic worker processes, whereas you need to assign users from the queue to a specific available operator.

I think what you primarily need is a FIFO queue of users, then whenever an operator becomes available, it can request the next user from the queue. That way the operators don’t need to talk to each other, instead they just need to pull items off of the shared queue. Depending on your scaling needs a single GenServer holding the queue of users should work fine (Look at the erlang queue module).

dimitarvp

dimitarvp

Have you looked at Oban?

Or Broadway?

Where Next?

Popular in Questions Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Phillipp
Hey, I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1140 51847 244
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement