unfode

unfode

How to achieve atomicity (all or nothing)?

Suppose I have two states. In C, I can mutate them atomically (either all or none are mutated) using locks:

bool atomic_mutation() {
  lock(state1_lock);
  lock(state2_lock);

  bool success = mutate1(state1);
  if (!success) {
    unlock(state1_lock);
    unlock(state2_lock);
    return false;
  }

  success = mutate2(state2);
  if (!success) {
    // revert mutate1(state1)
    unlock(state1_lock);
    unlock(state2_lock);
    return false;
  }

  unlock(state1_lock);
  unlock(state2_lock);
  return true;
}

I’m new to Elixir. What I’ve learned is that I normally use an Agent to manage a state. But how to achieve atomicity shown above in Elixir? Thanks!

Edit: I fixed the bug in the C code above as pointed out by @al2o3cr

Marked As Solved

al2o3cr

al2o3cr

The BEAM provides the infrastructure, but you need to write the code to glue things together into a consistent distributed system - and to be clear, once you have TWO GenServers you’re trying to make change together you’re in distributed-system territory.

For instance, here’s a very basic “table with a lock” GenServer (see below for notes):

defmodule TableWithLock do
  use GenServer

  defstruct [:data, :owner]

  def unlock(pid), do: GenServer.call(pid, :unlock)
  def lock(pid), do: GenServer.call(pid, :lock)
  def update(pid, fun), do: GenServer.call(pid, {:update, fun})

  @impl true
  def init(data) do
    {:ok, %__MODULE__{data: data, owner: nil}}
  end

  @impl true
  def handle_call(:lock, {pid, _tag}, state) do
    cond do
      is_nil(state.owner) ->
        # unlocked, pid now owns lock
        {:reply, :ok, %{state | owner: pid}}

      state.owner == pid ->
        # already locked by pid
        {:reply, :ok, state}

      true ->
        # locked by another process
        raise "oh no lock contention"
    end
  end

  def handle_call(:unlock, {pid, _tag}, state) do
    cond do
      is_nil(state.owner) ->
        # already not locked
        raise "somebody already unlocked this???"

      state.owner == pid ->
        # locked by the caller
        {:reply, :ok, %{state | owner: nil}}

      true ->
        # locked by somebody else
        raise "unlocking somebody else's lock"
    end
  end

  def handle_call({:update, fun}, {pid, _tag}, state) do
    cond do
      is_nil(state.owner) ->
        # not locked
        raise "not locked"

      state.owner == pid ->
        # locked by the caller
        result = fun.(state.data)
        {:reply, result, %{state | data: result}}

      true ->
        # locked by somebody else
        raise "updater not holding the lock"
    end
  end
end

{:ok, pid1} = GenServer.start_link(TableWithLock, [1,2,3])

:ok = TableWithLock.lock(pid1)

result = TableWithLock.update(pid1, fn data -> Enum.map(data, & &1*2) end)

:ok = TableWithLock.unlock(pid1)

IO.inspect(result)

There are a LOT of places where this could be work better / handle concurrency better:

  • crashing the table when a second process tries to take the lock is not realistic. A better implementation would keep a queue of pids that are currently trying to take the lock in lock and pick the next one to reply to in unlock.
  • crashing the table on bogus unlocks isn’t realistic either. An alternative would be to return something from handle_call that the implementation of unlock/1 could use to crash the calling process, since unlocking a table that you haven’t locked is a logic error
  • if a process dies while holding the lock, it will never be unlocked. Tools like Process.monitor can help with this, at the cost of additional complexity.

Expanding this setup to TWO tables adds some extra complications:

  • if process A takes lock 1 and then tries to take lock 2, while at the same time process B takes lock 2 and tries to take lock 1 the system is in a classic DEADLOCK situation. The default 5s timeout on GenServer.call will eventually pick a winner, but real systems will detect this and complain

  • coordinating changes to ensure that they either all appear or all do not is still just as tricky as always. You’d need a third process to coordinate the TableWithLocks and roll back changes if a future change fails.

    Note that even the code in your example does not produce atomicity - if mutate2 returns false, the changes from mutate1 are still visible.

    Solving this problem correctly is capital-H Hard and the solutions are highly sensitive to exactly what tradeoffs your particular application can tolerate.

Also Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

I would simply use the database itself for this, it will provide a ton of dedicated tooling for it.

codeanpeace

codeanpeace

If you’re using Ecto, it has a built in way of achieving this via Ecto.Multi

Ecto.Multi is a data structure for grouping multiple Repo operations.

Ecto.Multi makes it possible to pack operations that should be performed in a single database transaction…

defmodule PasswordManager do
  alias Ecto.Multi

  def reset(account, params) do
    Multi.new()
    |> Multi.update(:account, Account.password_reset_changeset(account, params))
    |> Multi.insert(:log, Log.password_reset_changeset(account, params))
    |> Multi.delete_all(:sessions, Ecto.assoc(account, :sessions))
  end
end
LostKobrakai

LostKobrakai

How do you make sure multiple actors changes are actually independent?

In the end you’ll surely get to the fact that immutability prevents certain optimizations, but generally the question should be how much those matter to the endproduct.

dimitarvp

dimitarvp

I am not sure OP wanted distributed locks, I think they simply were looking for a way to achieve what other languages achieve with a lot of dancing on their forehead while singing “Hallelujah!” (i.e. mutexes, semaphores, condition variables and all the other horrors).

If there’s no distributed requirement then simply serializing access to certain pieces of data – or external resources – via a GenServer is plenty enough. And even if there are distributed requirements then you can just use Oban Pro with the right uniqueness parameters set. Done.

Normally I don’t pursue after OP accepted an answer but you might be doing disservice to them here by giving them a “lock equivalent”. And I am not sure it’s much of an equivalent with so many disclaimers.

Erlang BEAM VM’s philosophy is to avoid locking and just serialize access to things in a singular process. If there are multiple tables (whatever that means?) involved then why not access them in your GenServer message handler, one after another? A GenServer will never process more than one message at a time anyway. You can connect to 3 different databases and 5 different Amazon S3 buckets and 8 separate message queues with zero consistency and contention problems – integrate the saga pattern in your function (like this library does) and you’re done. Problem solved!

It works flawlessly unless you are stepping in the territory of C, C++, Rust et. al. in which case, well, you know, just code in them.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hey welcome! The short answer is that if you need shared state and you need to control changes to that state then yeah you would use an agent or GenServer (more generally) and put the state inside that.

In general though as a functional language much of your program design will be avoiding such shared state at all.

Where Next?

Popular in Questions Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New

We're in Beta

About us Mission Statement