yolo007wizard

Need help with app architecture - how to dynamically run code?

Problem Statement:

I am trying to build an ‘action runner’ for judge json rules. In a nut shell, say a json defined rule matches with action string:

{
...rule
    "action": "collect_signature"
}

I’m trying to think of best architecture to run that ‘collect_signature’ code. It could be a module name or a script file. The intention is for rules to trigger actions in a dynamic way so actions are easy to maintain separately.

Desired Requirements:

the action code has its own dependencies/hex packages
adding new actions does not require app reload/recompile/redeploy of the ‘action runner’ i.e. the actions are dynamically loaded and run somehow.

Idea 1:

standalone exs scripts - Actions are defined in standalone exs files with their own Mix.installs. I tried this already by using Code.require_file/1 to dynamically run an exs file however ran into dependency errors. The ‘action runner’ runtime dependencies had conflicts with the loaded exs dependencies. Need a way to get around that. This seems the most promising however.

Idea 2:

Umbrella project where each action is a separate application. I don’t have experience with umbrella projects but this might be a feasible path. It seems a bit overkill however and I think the action runner would have to be reloaded every time a new action is created (which is a big con).

Idea 3:

Similar to umbrella, load actions as private local packages via a dependency path. The ‘action runner’ would then dynamically load dependencies in its mix.exs. Sudo code like:

defp deps do
    for each action_name in actions folder return:
        {:action_name, ">= 0.0.0", path: "/actions/action_name"}
end

Then maybe add a Mix task ‘new action’ that loads up the boiler plate for a new action in the correct folder. The main con with this is same as #2 - the action runner would have to be recompiled with every new action.

17 comments

#exs

8 1112 17

2023-09-09 17:53:12 UTC

Most Liked

axelson

Scenic Core Team

When I tried running multiple notes before I used the Erlang :peer module.

Here’s a few snippets from a semi-defunct unreleased project I have:

    {:ok, node} =
      :peer.start_link(%{
        host: :localhost,
        name: 'peer_node',
        args: peer_args()
      })

    IO.puts("DONE Starting peer node!")
    add_code_paths(node)
    ensure_applications_started(node)

  defp add_code_paths(node) do
    rpc(node, :code, :add_paths, [:code.get_path()])
  end

  @apps_to_start [
    # :iex,
    :logger,
    # :file_system,
    # :jason,
    :erlexec,
    :runtime_tools,
    # :inets,
    :stdlib,
    :crypto,
    # :hex,
    :elixir,
    # :public_key,
    # :gviz,
    # :mix,
    # :gettext,
    :kernel,
    :ssl,
    :compiler
    # :asn1,
  ]

  defp ensure_applications_started(node) do
    rpc(node, Application, :ensure_all_started, [:mix])
    rpc(node, Mix, :env, [Mix.env()])

    # for {app_name, _, _} <- Application.loaded_applications() do
    #   rpc(node, Application, :ensure_all_started, [app_name])
    # end
    for app_name <- @apps_to_start do
      IO.inspect(app_name, label: "starting app_name")

      rpc(node, Application, :ensure_all_started, [app_name])
      |> IO.inspect(label: "started #{inspect(app_name)}")
    end
  end

  defp rpc(node, module, function, args) do
    :rpc.block_call(node, module, function, args)
  end

  defp peer_args do
    Enum.join(
      [
        "-loader inet -hosts 127.0.0.1",
        "-setcookie #{:erlang.get_cookie()}",
        # "-env MIX_BUILD_ROOT /tmp/gviz_build"
      ],
      " "
    )
    |> to_charlist()
  end

Post #10

yolo007wizard

@LostKobrakai All the debugging above is for the :peer approach in PeerNode
@D4no0 instead of ‘best’ perhaps I should have said ‘easiest’ so far

I’m learning and trying to understand more. If both :peer and CLI approaches start a new beam instance, why is one considered better/worse?

Post #17

derek-zhou

Or you can do what Livebook does. Use a separate BEAM node.

Post #2

al2o3cr

Not sure if it’s the problem, but note that Node.spawn expects a function as its second argument and you’re trying to pass the result of Code.require_file there instead. Maybe you need a wrapping fn -> ... end?

Post #4