pedromvieira

pedromvieira

How do you ensure that only one “copy” of a worker will be active in a multi node otp application?

How to ensure that only one “copy” of worker will be active in a multi node otp application?
We created some tasks to update KPIs that will trigger on a set timed interval (ex: 60 seconds).
That’s why to avoid duplication and extra resources usage, only one per worker type need to be up running accross all nodes.

defmodule MyApp.Application do
  @moduledoc """
  Application Settings.
  """

  use Application

  def start(_type, _args) do
    import Supervisor.Spec

    children = [
      supervisor(MyApp.Repo, []),
      supervisor(MyApp.Endpoint, []),
      worker(Guardian.DB.Token.SweeperServer, []),
      worker(MyApp.Services.UserAgents.Server, []),
      worker(MyApp.Services.IPs.Server, [])
    ]

    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Most Liked

keathley

keathley

This really depends on what you mean when you say, “ensure”. How important is it that you only ever have one copy of a worker? There are loads of libraries out there to help solve this problem: swarm, syn, gproc, or even :global. Each of these libraries have different tradeoffs and guarantees. They handle things like network partitions and node failures differently. It’ll really come down to the kinds of guarantees you need. If you have a small and relatively stable set of nodes and can tolerate occasionally having “duplicate” workers then I’d look at swarm. If you need somewhat stricter guarantees then you might look at gproc. If you need even stricter guarantees you might want to use raft or better yet use an external data store like redis. My intuition (which should be taken with a massive grain of salt since I don’t know your exact use case) is that you’re probably best off using something like swarm. You can use “at least once” messaging guarantees over whatever transport you’re using and work to make your downstream service idempotent.

peerreynders

peerreynders

Would Raft be applicable to your use case?

Where Next?

Popular in Questions Top

yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New

Other popular topics Top

yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

We're in Beta

About us Mission Statement