lessless

lessless

How to start dynamic supervisor workers on application startup?

Hello,

In our app, we have to start dynamic amount of workers and we can’t just define them as a part of supervision tree in the application’s start callback, because of using :simple_one_for_one strategy

use Application

 def start(_type, _args) do
    children = [
      supervisor(Registry, [:unique, Postman.AccountRegistry])
    ]
    account_supervisors = Enum.map(accounts(), &account_supervisor/1)

    Supervisor.start_link(children ++ account_supervisors, [strategy: :one_for_one, name: Postman.Supervisor])
  end

  defp account_supervisors(account),
    do: Enum.map(accounts, &(supervisor(Postman.Processor.AccountSupervisor, [&1.name], id: &1.name)))


defmodule Postman.Processor.AccountSupervisor do
  use Supervisor

  def start_link(account_name) do
    Supervisor.start_link(__MODULE__, nil, name: via_tuple(account_name))
  end

  def start_child(account_name, max_demand),
    do: via_tuple(account_name) |> Supervisor.start_child([account_name, max_demand])

  def init(_) do
    supervise(
      [worker(Postman.Processor.Stage.Account, [])],
      strategy: :simple_one_for_one
    )
  end

  defp via_tuple(account_name),
    do: {:via, Registry, {Postman.AccountRegistry, "#{account_name}-supervisor"}}

end

That means to spawn new workers with AccountSupervisor.start_child/2 we have to wait until supervisor finishes its initialization.

The only way around that I can think of is to add GenStage that will start after Supervisors and which solely job will be to spawn children. But is spawning order guaranteed, i.e. is it synchronous?

Maybe there is a whole better way get around that problem?

Most Liked

josevalim

josevalim

Creator of Elixir

Yup, this is the way to go. You don’t need a GenServer though, a Task would suffice. And you want to pass worker(Test.Starter, [], restart: :transient) so it is not restarted right after it terminates.

idi527

idi527

I don’t quite understand your problem, but why can’t you just put a genserver at the end of the children list, which will start all your accounts and terminate? Is there really a need for genstage?

application.ex

defmodule Test.Application do
  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      supervisor(Registry, [:unique, Test.Registry]),
      supervisor(Test.Account.Supervisor, []),
      worker(Test.Account.Starter, [], restart: :transient)
    ]

    opts = [strategy: :one_for_one, name: Test.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

account/supervisor.ex

defmodule Test.Account.Supervisor do
  use Supervisor

  def start_link do
    Supervisor.start_link(__MODULE__, nil, name: __MODULE__)
  end

  def start_child(account) do
    Supervisor.start_child(__MODULE__, [account])
  end

  def init(_) do
    children = [
      worker(Test.Account, [])
    ]

    supervise(children, strategy: :simple_one_for_one)
  end
end

account/starter.ex

defmodule Test.Starter do
  use GenServer

  def start_link do
    GenServer.start_link(__MODULE__, nil)
  end

  def init(_) do
    send(self(), :start_children)
    {:ok, nil}
  end

  def handle_info(:start_children, _) do
    accounts = [:a, :b, :c]
    Enum.each(accounts, fn account ->
      Test.Account.Supervisor.start_child(account)
    end)
    {:stop, "iz kil"}
  end
end

Anyway, it’s just a suggestion. I also think there is a simpler way.

lessless

lessless

@idi527 you’re right - it should be GenServer there.

Thanks a lot, folks!

Where Next?

Popular in Questions Top

belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New

Other popular topics Top

9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New

We're in Beta

About us Mission Statement