alex88
Genserver isn't restarted on graceful shutdown
I’ve a very simple genserver, I start it like this:
defmodule MyApp.MatchKiller do
use GenServer
def start_link() do
GenServer.start_link(__MODULE__, [], name: :match_killer)
end
and in my app start function I call
Swarm.register_name("match_killer", MyApp.MatchKiller, :start_link, [], 15_000)
So I start two iex sessions and the process is started and registered globally:
[debug] [swarm on a@AlessandrosiMac] [tracker:handle_call] registering "match_killer" as process started by Elixir.MyApp.MatchKiller.start_link/0 with args []
[debug] [swarm on a@AlessandrosiMac] [tracker:do_track] starting "match_killer" on a@AlessandrosiMac
[debug] [swarm on a@AlessandrosiMac] [tracker:do_track] started "match_killer" on a@AlessandrosiMac
[debug] [swarm on b@AlessandrosiMac] [tracker:sync_registry] local tracker is missing "match_killer", adding to registry
[info] [swarm on b@AlessandrosiMac] [tracker:awaiting_sync_ack] local synchronization with a@AlessandrosiMac complete!
[info] [swarm on b@AlessandrosiMac] [tracker:resolve_pending_sync_requests] pending sync requests cleared
[debug] [swarm on b@AlessandrosiMac] [tracker:handle_call] registering "match_killer" as process started by Elixir.MyApp.MatchKiller.start_link/0 with args []
[debug] [swarm on b@AlessandrosiMac] [tracker:do_track] found "match_killer" already registered on a@AlessandrosiMac
now, since the process is running on a if I ctrl-c a on node a the process is restarted on host b:
[debug] [swarm on b@AlessandrosiMac] [tracker:handle_topology_change] topology change (nodedown for a@AlessandrosiMac)
[debug] [swarm on b@AlessandrosiMac] [tracker:handle_topology_change] restarting "match_killer" on b@AlessandrosiMac
[debug] [swarm on b@AlessandrosiMac] [tracker:do_track] starting "match_killer" on b@AlessandrosiMac
[debug] [swarm on b@AlessandrosiMac] [tracker:do_track] started "match_killer" on b@AlessandrosiMac
So far so good, the problem is that if instead gracefully shutdown sending a SIGTERM the process is just shut down (this is a when I send SIGTERM to b which is running the process):
iex(a@AlessandrosiMac)10> [debug] [swarm on a@AlessandrosiMac] [tracker:handle_monitor] "match_killer" is down: :shutdown
[info] [swarm on a@AlessandrosiMac] [tracker:nodedown] nodedown b@AlessandrosiMac
[debug] [swarm on a@AlessandrosiMac] [tracker:handle_topology_change] topology change (nodedown for b@AlessandrosiMac)
[info] [swarm on a@AlessandrosiMac] [tracker:handle_topology_change] topology change complete
if I then restart b (while a is still running) the process is restarted on b because probably it sees that it’s not running.
the problem here is that when I scale down the nodes it won’t restart the process and it won’t transfer its state. Am I missing something?
Most Liked
balena
Hey @alex88,
I don’t know if you still have this problem, but recently I had exactly the same problem at my side using Swarm and decided that it needs a very simple feature from the regular Supervisor implementation: a restart flag, which indicates how your process should live, cluster-wise. The restart parameter assumes the same values as those passed to Supervisor child spec (check hexdocs.pm):
-
:permanentmeans a handoff will always occur, -
:transientmeans that the process will be shut down only if the termination reason is other than:normal | :shutdown | {shutdown, term}, -
:temporarymeans it is never restarted, only when the node gets down abruptly (generating a:DOWNevent with reason:noconnection). This is the default, compatible with the original implementation.
I provided a PR, waiting for approval (or not). It’s here: https://github.com/bitwalker/swarm/pull/139







