jmurphyweb

Understanding concurrent tasks in context of BEAM schedulers & CPU cores

Hi,

I’m just reading Elixir In Action and have a question about running concurrent tasks in relation to number of cores available.

From what I can understand, a scheduler can only run one CPU-bound task at a time. So spawning less processes than there are schedulers should show a similar amount of time a CPU-bound task takes to run.

I put together my first ever benchmark test using Benchee. But the results were not as expected. Spawning 4 tasks took a lot longer than spawning 1 task even though Benchee reckons I have 8 Cores available.

Can anyone shed any light on what’s happening here?

TEST

Assumptions:

Number of BEAM schedulers equals the number of cores c on my machine.
cpu_bound = fn -> 1..5_000_000 |> Enum.map(&(&1 + 1)) end executes a “CPU bound task” which takes some amount of time t on my machine to run.

Hypothesis:
I’d expect that spawning tasks which run cpu_bound to take roughly t time until the number of tasks spawned increases above the number of schedulers available.

Result:

iex(1)> Bench.run()
Operating System: macOS
CPU Information: Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz
Number of Available Cores: 8
Available memory: 16 GB
Elixir 1.10.1
Erlang 22.2.6

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 10 s
memory time: 2 s
parallel: 1

Name             ips        average  deviation         median         99th %
spawn_1          1.94      516.54 ms     ±1.19%      517.21 ms      528.99 ms
spawn_2          1.68      595.94 ms     ±1.36%      593.47 ms      614.60 ms
spawn_4          1.26      794.28 ms     ±2.83%      800.31 ms      821.24 ms
spawn_8          0.77     1302.10 ms     ±1.03%     1306.02 ms     1317.07 ms

Comparison:
spawn_1          1.94
spawn_2          1.68 - 1.15x slower +79.40 ms
spawn_4          1.26 - 1.54x slower +277.74 ms
spawn_8          0.77 - 2.52x slower +785.55 ms

And the code:

defmodule Bench do
  def run do
    Benchee.run(
      %{
        "spawn_1" => fn -> MultiProc.async(1..1) end,
        "spawn_2" => fn -> MultiProc.async(1..2) end,
        "spawn_4" => fn -> MultiProc.async(1..4) end,
        "spawn_8" => fn -> MultiProc.async(1..8) end
      },
      time: 10,
      memory_time: 2
    )
  end
end

defmodule MultiProc do
  def async(range \\ 1..4) do
    self = self()

    range
    |> Enum.map(fn x ->
      spawn(fn ->
        1..5_000_000 |> Enum.map(&(&1 + 1))
        send(self, x)
      end)
    end)
    |> Enum.map(&receive_message(&1))
  end

  def receive_message(_msg) do
    receive do
      n -> n
    end
  end
end

5 comments

#book-elixir-in-action

4 2574 5

2020-05-04 11:37:19 UTC

Most Liked

hauleth

This one is quite important if you want to write performant software (and partially this is also culprit of SPECTRE attacks on Intel’s x86 CPUs).

So in short - there is difference in access time between each part of memory in computer. Disk is the slowest and RAM is faster. In old days it was how it was working. However to speedup computation even further the access to RAM was still too slow. So CPU added cache on their own, which is the quickest memory store possible (except direct access to data stored in registers). However the point is that the quickest is the access the less memory you have. So if you are reading some data, then CPU stores that in it’s cache (often a little bit more than you wanted just in case it is sequential read, ex. array), which allows you to process data faster. However when you jump between unrelated (non sequential) parts of memory (like in linked lists) then the CPU will slow down as on each step it need to fetch data from slower RAM again.

Also as each CPU can execute only one process at the time (obviously) then each time you want to swap to another process you need to store all of information used by that process (all CPU registers) in RAM and then load data for “new” process so they will “feel” that nothing have changed (except time).

There is even more, like branch prediction that helps modern CPUs with even faster computations (but with problems like mentioned SPECTRE) or SIMD which is instruction-level parallelism for performing computations on multiple data at once.

In general it is very interesting topic that has huge impact on performance, even in languages like Erlang that lives quite far from the “bare metal”. In IT it is generally known as “law of leaky abstractions” where hiding some information about underlying systems still has impact on your code anyway (other example is NFS which simulates local filesystem while in reality dealing with much harder stuff - networking).

Hard to tell, but the general rule is that processes in Erlang aren’t meant to be used as sources of performance improvements (in old days Erlang was single threaded, always), but as a “error containments”, so failure of a single process will not affect the rest of the system. Another thing is that Erlang is not the best thing out there for CPU-intensive tasks.

Post #4

hauleth

This is partially true, as if you would have more schedulers than cores then it will not be true. In general one CPU cannot run more than one task at the time, and you can think about schedulers as “virtual CPUs”.

You have forgot about scheduling and cache locality. Each process will have N reductions (“virtual CPU units”) assigned and when it uses them (or will call receive) will be exempted and other process will take its place. The problem there is that all data this process is currently using need to be cached, all registers need to be saved somewhere (BEAM is register-based VM), etc. This is not negligible amount of work that need to be done.

Another thing that you have forgot is Amdahl’s law which mean that the possible speedup depends on the joining operation.

Another thing is that it also highly depends on what is your machine doing in the meantime, as BEAM do not “own” CPUs, so the underlying host OS can do their own scheduling in the meantime.

Post #2

dimitarvp

This is not strictly true. The BEAM preemptively pauses and switches processes. It’s true that latency can suffer (very minimally in my experience) if you spawn 100 CPU-bound tasks on a 4-core machine but that won’t cause a deadlock in the meaning of how an infinite loop C/C++ program would do it.

See what the author of Benchee has to say on parallel benchmarks. My takeaway is that it is simply not made to benchmark parallel workloads.

Post #6