Questions & Help>Questions

Is there a limit to creating process transactions in Poolboy?

/phoenix#erlang#worker-pools
GusGA

GusGA

Is there a limit to creating process transactions in Poolboy?

Is there a limit to creating process transactions in Poolboy?

The problem:

I upload a file with several ids, for each id it creates a poolboy transaction. There may be up to 10000 processes waiting to run and the machine crash after a while.

I need a limit on those processes, I work with 150 poolboy workers.

Maybe it’s not the best way enqueue processes in poolboy, but for now I dont know a better implementation.

Thanks in advance

2 comments
/phoenix#erlang#worker-pools
3 920 2

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

@GusGA yeah it sounds like you want a proper queue here of some kind or another. 10_000 processes each trying to checkout a poolboy process is a lot of contention. Each process will only wait the pool_timeout amount. Poolboy is great for holding on to resources, but it isn’t a queueing system.

I would consider GenStage, Flow, or one of the persisted worker queue libraries depending on your persistence needs.

2
Post #3
OvermindDL1

OvermindDL1

This sounds like precisely the use-case for GenStage/Flow. :slight_smile:

But in poolboy limits, I’m unsure? I really wouldn’t think so…

1
Post #2

Where Next?

View thread on forum (has 2 responses!)
phoenix
erlang
worker-pools
Home Questions & Help>Questions
/phoenix#erlang#worker-pools
3 922 2
Last post

Popular in Questions Top

vonH
Questions & Help>Questions

Why would I choose Elixir as a general purpose programming language?
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
#functional-programming#use-cases
65 34758 13
New
sergio_101
Questions & Help>Questions

Pattern matching against a string
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
#pattern-matching
27 26685 14
New
pgiesin
Questions & Help>Questions

Ecto :database Not Found
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
#ecto#postgres#troubleshooting#ecto-migration
7 14214 22
New
_russellb
Questions & Help>Questions

Web scraping tools
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
#web-scraping
123 19127 45
New
sacepums
Questions & Help>Questions

Converting date string to Ecto.Date
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
/phoenix#ecto#dates
14 13473 4
New
minhajuddin
Questions & Help>Questions

Using List.first instead of Enum.at(0)
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
#code-style
76 33273 14
New
ashish173
Questions & Help>Questions

Difference in between :utc_datetime and :naive_datetime in Ecto
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
#ecto#timestamps
79 27907 15
New
Codball
Questions & Help>Questions

Mix format on save VSCode
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
#mix
29 13345 4
New
romenigld
Questions & Help>Questions

Docker run Error loading shared library libstdc++.so.6 and libgcc_s.so.1
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
/phoenix#deployment#docker-compose
30 19932 15
New
idi527
Questions & Help>Questions

How I would create a multiline string without a trailing line break with Elixir?
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
#strings
33 13891 10
New

Other popular topics Top

yawaramin
Questions & Help>Questions

Dialyzer: suppress warning on a specific function
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
#dialyzer
25 13690 6
New
aalberti333
Questions & Help>Questions

Enum.map over list of key/value pairs with a map as the value
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
#enummap
7 19677 6
New
Trevoke
Questions & Help>Questions

(EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
I have an umbrella app. Some of the apps inside depend on other apps in the umbrella, unsurprisingly. I'm writing a test for one of t...
#otp#umbrella#testing#exunit#errors
26 21431 11
New
chrismccord
News>Phoenix News

Phoenix v1.5.0 released!
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
/phoenix#phoenix-release
379 28447 94
New
nsuchy
Questions & Help>Questions

IEX in Windows Powershell?
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
#iex#microsoft-windows#windows#powershell
15 29309 4
New
vertexbuffer
Questions & Help>Questions

Deleting item from a list
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
7 24097 4
New
chrismccord
News>Phoenix News

Phoenix v1.3.0-rc.0 released
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
/phoenix#phoenix-release
434 29347 97
New
shahryarjb
Questions & Help>Questions

How to convert map to string (separated with ,)
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
#maps#strings
15 15491 2
New
AstonJ
Chat & Discussions>Wikis

Phoenix LiveView Info
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
/phoenix#wiki#stickies#liveview
220 25307 74
New
aesmail
News>Announcing

Kaffy - a quick and flexible admin interface for phoenix applications
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
/phoenix#ecto#library#admin-tools#kaffy
301 23683 97
New
Back to top

Latest Phoenix Threads

Phoenix Forum ❯

Questions & Help>Questions

Questions Questions ❯

Latest on Elixir Forum

Elixir Forum ❯

Sponsor Spotlight

Dashbit

Build Elixir applications with speed and confidence.

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Manning

Develop your skills with books, videos, and courses.

Potions

The deployment platform built for Elixir: PaaS ease, VPS control.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Forums

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Manning

Develop your skills with books, videos, and courses.

Potions

The deployment platform built for Elixir: PaaS ease, VPS control.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement