elvanja
Load all modules implementing a behaviour in escript
Hi, I’m trying to see if there’s a way to find all the modules that implement a behaviour, but from an escript.
This is one way I tried extracting those:
defp modules_implementing_behaviour(behaviour) do
for {module, _} <- :code.all_loaded(),
behaviour in (module.module_info(:attributes)
|> Keyword.get_values(:behaviour)
|> List.flatten()) do
module
end
end
Another attempt was this (also required adding :mix to :external_applications in mix.exs):
# copied from https://github.com/findmypast/behaviour-introspection
defp modules_implementing_behaviour(behaviour) do
# Ensure the current projects code path is loaded
Mix.Task.run("loadpaths", [])
# Fetch all .beam files
Path.wildcard(Path.join([Mix.Project.build_path(), "**/ebin/**/*.beam"]))
# Parse the BEAM for behaviour implementations
|> Stream.map(fn path ->
{:ok, {mod, chunks}} = :beam_lib.chunks('#{path}', [:attributes])
{mod, get_in(chunks, [:attributes, :behaviour])}
end)
# Filter out behaviours we don't care about and duplicates
|> Stream.filter(fn {_mod, all_behaviours} ->
is_list(all_behaviours) && behaviour in all_behaviours
end)
|> Stream.map(fn {module, _} -> module end)
|> Enum.uniq()
end
None of those actually work like that. The first one did kind of work if I did Code.ensure_loaded?(AModuleThatImplementsTargetBehaviour), but it defeats the purpose since I’d need to manually list all the related modules.
The problem I’m trying to solve is a project that knows how to perform a data sync between two databases. It has a number of sync scenarios to choose from. The whole idea is to create an escript that accepts a param that determines which data sync scenario we want, starts the application, finds that scenario and executes it. Not even sure that the escript is the correct way to do it. Thought of releases but can’t find a nice way to wrap it all up as an executable. Docker is something that would work but I was hoping to find a way without that. I can even make it a service that exposes some API and can be triggered via http, but that seems like an overkill for something so simple.
Anyway, thank you for your time 
Marked As Solved
dimitarvp
This got interesting to me because I’ve struggled with it in the past so I made a more complete example. I have a few meduim-sized Elixir projects whose source customers from 2018 and earlier have allowed me to keep. I tested the below module on them and it works quite fine in iex at least.
Curious to find out if it works for you. I realize your case is a bit different but, again, I feel this code is a good first step. Would ditching the escript idea for a Mix task work?
defmodule Behaviours do
def modules_belonging_to_namespace(namespace)
when is_binary(namespace) do
:code.all_available()
|> Enum.filter(fn {name, _file, _loaded} ->
String.starts_with?(to_string(name), "Elixir." <> namespace)
end)
|> Enum.map(fn {name, _file, _loaded} ->
name |> List.to_atom()
end)
end
def ensure_all_modules_loaded(namespace)
when is_binary(namespace) do
modules_belonging_to_namespace(namespace)
|> :code.ensure_modules_loaded()
end
def behaviours(mod)
when is_atom(mod) do
# Convert resulting list to [] if list of behaviours is nil
mod.module_info
|> get_in([:attributes, :behaviour])
|> List.wrap()
end
def implementors(namespace, behaviour)
when is_binary(namespace) and is_atom(behaviour) do
ensure_all_modules_loaded(namespace)
modules_belonging_to_namespace(namespace)
|> Enum.filter(& behaviour in behaviours(&1))
end
end
Also Liked
elvanja
Nice, thank you! That’s https://github.com/elvanja/escript_testbed/pull/2 (for reference). And you were right, it is abusing the protocols indeed
There’s even a bit simpler way with protocols, added to comments on the PR. Cool, today I learned!
elvanja
I’ve created a small project to demonstrate the problem, see https://github.com/elvanja/escript_testbed. One funny thing I noticed is that running it as a mix task also doesn’t work, unless I change one of the scenario modules. In that case, that module is compiled, and mix task is able to find it. But, if there are no changes, there’s also no result. So even in mix tasks I’m not getting the code in lib unless directly specified somehow.
dimitarvp
I think you are sorted.
I again used the module I proposed above and put a note that you need OTP 23 for the PR to work.
elvanja
Yes @dimitarvp, with OTP 23 and :code.all_available() (tried it with your Behaviours module idea) it seems to be working, both from mix task and as escript executable! Nice 
P.S. Just saw your Load all modules implementing a behaviour in escript reply and related PR. That’s mostly what I tried locally, thanks! I’m probably gonna form it a bit differently, since I don’t need that behaviour module except in CLI, but still, the idea works. Great work man!
lud
I sent you a pull request implementing what I meant. both list and sync commands work, (though I commented out a lot of code instead of configuring the repos, did not see the makefile before reading @dimitarvp posts)








