elvanja

elvanja

Load all modules implementing a behaviour in escript

Hi, I’m trying to see if there’s a way to find all the modules that implement a behaviour, but from an escript.
This is one way I tried extracting those:

  defp modules_implementing_behaviour(behaviour) do
    for {module, _} <- :code.all_loaded(),
        behaviour in (module.module_info(:attributes)
                       |> Keyword.get_values(:behaviour)
                       |> List.flatten()) do
      module
    end
  end

Another attempt was this (also required adding :mix to :external_applications in mix.exs):

  # copied from https://github.com/findmypast/behaviour-introspection
  defp modules_implementing_behaviour(behaviour) do
    # Ensure the current projects code path is loaded
    Mix.Task.run("loadpaths", [])

    # Fetch all .beam files
    Path.wildcard(Path.join([Mix.Project.build_path(), "**/ebin/**/*.beam"]))
    # Parse the BEAM for behaviour implementations
    |> Stream.map(fn path ->
      {:ok, {mod, chunks}} = :beam_lib.chunks('#{path}', [:attributes])
      {mod, get_in(chunks, [:attributes, :behaviour])}
    end)
    # Filter out behaviours we don't care about and duplicates
    |> Stream.filter(fn {_mod, all_behaviours} ->
      is_list(all_behaviours) && behaviour in all_behaviours
    end)
    |> Stream.map(fn {module, _} -> module end)
    |> Enum.uniq()
  end

None of those actually work like that. The first one did kind of work if I did Code.ensure_loaded?(AModuleThatImplementsTargetBehaviour), but it defeats the purpose since I’d need to manually list all the related modules.

The problem I’m trying to solve is a project that knows how to perform a data sync between two databases. It has a number of sync scenarios to choose from. The whole idea is to create an escript that accepts a param that determines which data sync scenario we want, starts the application, finds that scenario and executes it. Not even sure that the escript is the correct way to do it. Thought of releases but can’t find a nice way to wrap it all up as an executable. Docker is something that would work but I was hoping to find a way without that. I can even make it a service that exposes some API and can be triggered via http, but that seems like an overkill for something so simple.

Anyway, thank you for your time :slight_smile:

Marked As Solved

dimitarvp

dimitarvp

This got interesting to me because I’ve struggled with it in the past so I made a more complete example. I have a few meduim-sized Elixir projects whose source customers from 2018 and earlier have allowed me to keep. I tested the below module on them and it works quite fine in iex at least.

Curious to find out if it works for you. I realize your case is a bit different but, again, I feel this code is a good first step. Would ditching the escript idea for a Mix task work?

defmodule Behaviours do
  def modules_belonging_to_namespace(namespace)
  when is_binary(namespace) do
    :code.all_available()
    |> Enum.filter(fn {name, _file, _loaded} ->
      String.starts_with?(to_string(name), "Elixir." <> namespace)
    end)
    |> Enum.map(fn {name, _file, _loaded} ->
      name |> List.to_atom()
    end)
  end

  def ensure_all_modules_loaded(namespace)
  when is_binary(namespace) do
    modules_belonging_to_namespace(namespace)
    |> :code.ensure_modules_loaded()
  end

  def behaviours(mod)
  when is_atom(mod) do
    # Convert resulting list to [] if list of behaviours is nil
    mod.module_info
    |> get_in([:attributes, :behaviour])
    |> List.wrap()
  end

  def implementors(namespace, behaviour)
  when is_binary(namespace) and is_atom(behaviour) do
    ensure_all_modules_loaded(namespace)

    modules_belonging_to_namespace(namespace)
    |> Enum.filter(& behaviour in behaviours(&1))
  end
end

Also Liked

elvanja

elvanja

Nice, thank you! That’s https://github.com/elvanja/escript_testbed/pull/2 (for reference). And you were right, it is abusing the protocols indeed :smile: There’s even a bit simpler way with protocols, added to comments on the PR. Cool, today I learned!

elvanja

elvanja

I’ve created a small project to demonstrate the problem, see https://github.com/elvanja/escript_testbed. One funny thing I noticed is that running it as a mix task also doesn’t work, unless I change one of the scenario modules. In that case, that module is compiled, and mix task is able to find it. But, if there are no changes, there’s also no result. So even in mix tasks I’m not getting the code in lib unless directly specified somehow.

dimitarvp

dimitarvp

I think you are sorted. :slight_smile: I again used the module I proposed above and put a note that you need OTP 23 for the PR to work.

elvanja

elvanja

Yes @dimitarvp, with OTP 23 and :code.all_available() (tried it with your Behaviours module idea) it seems to be working, both from mix task and as escript executable! Nice :smile:

P.S. Just saw your Load all modules implementing a behaviour in escript reply and related PR. That’s mostly what I tried locally, thanks! I’m probably gonna form it a bit differently, since I don’t need that behaviour module except in CLI, but still, the idea works. Great work man!

lud

lud

I sent you a pull request implementing what I meant. both list and sync commands work, (though I commented out a lot of code instead of configuring the repos, did not see the makefile before reading @dimitarvp posts)

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New

We're in Beta

About us Mission Statement