Fl4m3Ph03n1x
Supervision tree conflict in an umbrella app
Background
I have an umbrella app that has many smaller apps inside. One of this apps, called A, needs to be able to spin and supervise another app, called B.
B, being an app in its own right, exposes a public API and has a GenServer, responsible for receiving requests that it then redirects to the logic modules and such.
Issue
So, I have two requirements:
- I must be able to launch
Bindependently and have it work as a normal standalone app. -
Amust be able to haveBin its children and restart/manage it, should such a need arise.
The problem I have here, is that with my code I can either achieve 1 or 2, but not both.
Code
So, the following is the important code for app B:
application.ex
defmodule B.Application do
@moduledoc false
use Application
alias B.Server
alias Plug.Cowboy
@test_port 8082
@spec start(any, nil | maybe_improper_list | map) :: {:error, any} | {:ok, pid}
def start(_type, args) do
# B.Server is a module containing GenServer logic and callbacks
children = children([Server])
opts = [strategy: :one_for_one, name: B.Supervisor]
Supervisor.start_link(children, opts)
end
end
server.ex (simplified)
defmodule B.Server do
use GenServer
alias B.HTTPClient
#############
# Callbacks #
#############
@spec start_link(any) :: :ignore | {:error, any} | {:ok, pid}
def start_link(_args), do: GenServer.start_link(__MODULE__, nil, name: __MODULE__)
@impl GenServer
@spec init(nil) :: {:ok, %{}}
def init(nil), do: {:ok, %{}}
@impl GenServer
def handle_call({:place_order, order}, _from, _state), do:
{:reply, HTTPClient.place_order(order), %{}}
@impl GenServer
def handle_call({:delete_order, order_id}, _from, _state), do:
{:reply, HTTPClient.delete_order(order_id), %{}}
@impl GenServer
def handle_call({:get_all_orders, item_name}, _from, _state), do:
{:reply, HTTPClient.get_all_orders(item_name), %{}}
##############
# Public API #
##############
def get_all_orders(item_name), do:
GenServer.call(__MODULE__, {:get_all_orders, item_name})
def place_order(order), do:
GenServer.call(__MODULE__, {:place_order, order})
def delete_order(order_id), do:
GenServer.call(__MODULE__, {:delete_order, order_id})
end
And here is the entrypoint of B
b.ex
defmodule B do
@moduledoc """
Port for http client.
"""
alias B.Server
defdelegate place_order(order), to: Server
defdelegate delete_order(order_id), to: Server
defdelegate get_all_orders(item_name), to: Server
@doc false
defdelegate child_spec(args), to: Server
end
b.ex is basically a facade for the Server, with some extra context information such as specs, type definitions, etc (omitted here for the sake of brevity).
How does A manage the lifecycle?
It is my understanding that supervision trees are specified in the application.ex file of apps. So, from my understanding, I have created this application file for A:
defmodule A.Application do
@moduledoc false
use Application
alias B
def start(_type, _args) do
children = [B]
opts = [strategy: :one_for_one, name: A.Supervisor]
Supervisor.start_link(children, opts)
end
end
Which should work, except it doesn’t.
When inside A's folder, if I run iex -S mix, instead of having a nice launch I get the following error:
** (Mix) Could not start application a: A.Application.start(:normal, []) returned an error: shutdown: failed to start child: B.Server
** (EXIT) already started: #PID<0.329.0>
My current understanding of the issue is that A's application.ex file is conflicting with B's application file.
Questions
- How do I fix this conflict?
Marked As Solved
Fl4m3Ph03n1x
After several posts from you guys I have settled in what I believe is a good organization for the project:
-
:manager,:storeand:auction_housewill be statefull libraries aka @LostKobrakai. They will be used like parsing project @al2o3cr mentioned. -
:cliand:web_interfacewill be real applications that will have a:modkey in theirapplicationfunction inmix.exs. - When using
mix releaseI will have a release for the cli app and one for the web_interface app.
I think this structure give me the benefits of umbrella’s organization, while still giving me the benefits of self healing that I so much value in elixir.
Thank you everyone for your help!
Also Liked
LostKobrakai
There is explicitly no level above applications and therefore no supervision of any kind. If any application* crashes the whole beam instance exits. The only way to recover from that is using system level supervisors like e.g. systemd on linux or setting up erlang’s heart to try to restart the whole instance from the outside.
An application crashing is the very end of trying to self-heal from within the beam. The application itself stopping because the root process crashed is basically the equivalent of: Restarts didn’t help, now it’s time to stop trying.
By the above logic this dependency is irrelevant, as when :b crashes it will take down the whole beam instance including :a anyways.
You need to adjust your mental model of applications. Applications are groups of code and maybe a set of stateful processes started when starting an application. That’s it. Besides order of startup (based on dependencies between applications) applications stand in no hierarchy to each other and there is also no supervision of any kind. If any application* fails the whole beam instance fails.
Stateless applications are often called libraries or library applications, so maybe thinking of stateful applications as libraries with state might make it more obvious.
Supervision trees are a completely different thing. Here you’re dealing with processes, supervision and restarts, the possibility to self heal and so on. Resilience in your system comes from splitting up code execution into different processes, while splitting code into different applications is mostly for organization of code and/or functionality.
- Tech. there’s
:transientand:temporaryapplications as well. Those are rarely used however.
al2o3cr
This is why the top-level process of an application is a Supervisor and not an application GenServer; barring weird hardware errors, the only reason Supervisor will exit is if its children are restarting too often - see the :max_restarts and :max_seconds options.
I’ve seen this happen in production, but it was because the supervised process had a bug and got a MatchError when running init - so no amount of restarting would help.
al2o3cr
Passing B as a child spec means “Call B.child_spec/1 and use that”; that’s delegated to B.Server.child_spec, which returns a spec with name: B.Server.
There can only be one process on the node named B.Server, so when A.Supervisor tries to start B it fails with :already_started - because the umbrella app plumbing already starts apps listed as in_umbrella: true dependencies.
Amust be able to haveBin its children and restart/manage it, should such a need arise.
Regarding your original question, AFAIK there’s nothing stopping a process in A from monitoring etc a process in B.
LostKobrakai
Applications are not processes. Both can be started and stopped (granted by different API), but that’s where the similarities end.
Which applications are available or even started on a given beam instance is defined by the applications list (in elixir usually implicitly built from all applications in a mix project, their deps and extra_applications in the mix.exs).
This is almost completely separate to processes and supervision trees. Supervisors manage processes and processes only. Those processes can run code from the same application or code from other available applications.
The only place where supervision tree (processes) and applications meet is when a callback module (commonly MyApp.Application) is defined for an given application (making it stateful), which then must start a process when called and return its pid to the beam. If that process of the returned pid crashes the application is considered not working and therefore stopped by the beam (no retries or anything). Usually the process returned here is the root level supervisor for many other processes of an application, but it doesn’t need to be.
al2o3cr
The process I’d consider “application B” is the one started by Supervisor.start_link/2 in B.Application.start/2, named B.Supervisor. That supervisor is responsible for restarting the process named B.Server if it crashes.
IIRC if B.Supervisor crashes too many times too fast then The System Is Down, but there may be ways to alert about that.
The mix.exs file for project A declares {:b, in_umbrella: true} alongside its other dependencies.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"