It’s important to realize that on GenServer.call/3 processing in the calling process blocks:

• an asynchronous message is sent
• the calling process enters a blocking receive with a timeout
• the calling process unblocks when either the timeout expires (failing the call) or when the awaited return message arrives

So the calling process isn’t doing anything between the sent and return message.

because we don’t know when the messages get lost in the air.

Sometimes it makes more sense to monitor the process responsible and using a timeout to determine whether the result will be available (in time). That way the casting process doesn’t block (but the process interaction protocol is more work).

You are still running sequentially.

You are spawning a single process then entering a blocking receive and waiting for the result from that process. Only after the blocking receive is exited can the code advance and spawn the next process.

Now the BEAM can handle a million processes but given the circumstances it isn’t efficient.

Have a look at Task.Supervisor.async_stream/6. It allows you to constrain the number of tasks to the number of schedulers/cores to have the most efficient execution.

https://medium.com/@dinojoaocosta/elixir-findings-asynchronous-task-streams-7f6336227ea

It’s kind of difficult to judge without access to your code - if for some reasons you used blocking operations those 12 processes may not be generating enough work to warrant more cores.

This script floods the cores of my 2016 MBP with work:

$ elixir demo.exs
Schedulers online: 8
Seconds: 265
[
  {1001295697, [{19001, 52697}]},
  {1001288794, [{10874, 92081}]},
  {1001249680, [{10421, 96080}]},
  {1001244420, [{24420, 41001}]},
  {1001240352, [{25011, 40032}]},
  {1001193790, [{10310, 97109}]},
  {1001147422, [{24002, 41711}]},
  {1001139282, [{11022, 90831}]},
  {1001127442, [{14021, 71402}]},
  {1001020252, [{20012, 50021}]},
  {1000812510, [{12510, 80001}]},
  {1000520064, [{20004, 50016}]},
  {1000520010, [{20010, 50001}]},
  {1000425010, [{25010, 40001}]},
  {1000407528, [{12504, 80007}]},
  {1000250010, [{20001, 50010}]},
  {1000198206, [{10206, 98001}]},
  {1000191991, [{10991, 91001}]},
  {1000174288, [{14288, 70001}]}
]
$ 

cores417×594 50.7 KB

Note:

This is true for any message sent. Erlang messaging between processes is like sending mail to a friend.

If you don’t receive a reply, you’ll never know if your letter got lost or the reply, or if your friend just didn’t sent it.

But still for such a thing I consider something like GenStage or Flow is probably the best approach.

I am awestruck at the improvement! From 30 minutes to 7 minutes. Thanks again for this wonderful thing!

It gives us the freedom to decide the number of concurrent processes too (as you very generously highlighted). Just to make sure I am not leaving any other stone un-turned, here is what I am doing:

arg_n..arg_k 
    |> Task.async_stream(&Mix.Tasks.Boss.vamp_check/1, max_concurrency: System.schedulers_online) 
    |> Enum.map(fn {:ok, _result} -> nil end)

It is printing the result in vamp_check itself hence the nil in Enum.map.

(To be noted that my laptop has System.schedulers_online=12)