One thing that’s not helping concurrency: doing work in init means that start_link takes longer to return. Consider moving the code from ProcessEngine.init to a handle_continue callback.

Another thing that isn’t helping concurrency: init and execute both need to make calls to singleton processes (ProcessModelService and ProcessService)

With a nearly 100x increase based on 10x more input, I’d start by carefully looking at where data’s being collected in the code; all it takes is one List.append that’s called per-ProcessEngine to make things quadratic.

Some other random thoughts:

• terminate doesn’t do any formatting on the reason argument, so if you call Process.exit(some_pid, :shutdown) (eg) you’ll get :shutdown as an argument - and the ProcessEngine target will crash!

• harping on the same point: terminate is not guaranteed, there are lots of (admittedly uncommon) scenarios where it will not run before the process disappears. Consider checkpointing the state during intermediate steps, if “resuming” is important.

• consider extracting type-specific code like this to a per-step (or per-type) “callback module”

Instead of checking type everywhere, you’d define a single map:

@callback_modules %{
  decision: DecisionCallbacks,
  service: ServiceCallbacks,
  send: SendCallbacks,
  ...
}

A method like complete_able simplifies to:

defp complete_able(t) do
  @callback_modules[t.type].complete_able(t)
end

DecisionCallbacks then defines:

defmodule DecisionCallbacks do
  def complete_able(_task), do: true

  ...more functions used elsewhere in ProcessEngine...
end

This would also be a valuable place to utilize a behaviour to ensure that the callback modules implement a consistent interface.

A further, even more dynamic approach, would have a field on Task that contains the callback module name. That would allow tasks with the same type to have different callbacks. Whether that’s a bug or a feature will depend on your specific needs

One general note about both variants: they make the code easier to read, but also obscure it from parts of Elixir’s static analysis.

For instance, you would get a compiler error if you wrote DecisionCallbacks.complete_able() (calling with the wrong arity) explicitly, but writing @callback_modules[t.type].complete_able() will only crash at RUNTIME.