PatrickSachs

Accessing a module in the main app from a dependency

Hello there,

I have the following scenario:

I have a library my_lib that is used in my_app. my_lib has a configuration file option that accepts a list of modules. Based on the list of modules provided, a Plug is generated.

The config in my_app is defined like this:

config :my_lib,
  modules: [MyApp.Plugs.Plug1, MyApp.Plugs.Plug2]

The code in my_lib looks like this:

modules = Application.get_env(:my_lib, :modules)
for module <- modules do
  quote do
    forward "/module/#{unquote module}", to: unquote(module)
  end
end

This compiles, but a request to /module/Elixir.MyApp.Plugs.Plug1 just yields a 404 error. Interestingly, I also get a warning in VS Code in the for line, that modules cannot be iterated since it is nil, which to my understanding cannot be the case since I defined a default value of [] for it the mix.exs file of my_lib (I can also certainly confirm that it is not nil, since logging it prints the expected value).

def application do
  [
    mod: {MyLib.Application, []},
    env: [modules: []]
  ]
end

At this point I assumed that I either made an error in my meta-programming or since my_lib is compiled before my_app, that its modules are simply not yet available.
According to the documentation Code.ensure_compiled/1 would be a good candidate to wait for the compilation of these modules. So I added a call to this function and printed out its results, which were quite baffling to me:

modules = Application.get_env(:my_lib, :modules)
for module <- modules do
  compiled = Code.ensure_compiled(module)
  IO.puts("ensure_compiled #{inspect module} -> #{inspect compiled}")
  # = ensure_compiled MyApp.Plugs.Plug1 -> {:error, :nofile}
  # = ensure_compiled MyApp.Plugs.Plug2 -> {:error, :nofile}
  quote do
    forward "/module/#{unquote module}", to: unquote(module)
  end
end

Apparently these modules do not exist at all. However, calling this function directly in IEx, the call resolves successfully:

iex(1)> Code.ensure_loaded MyApp.Plugs.Plug1
{:module, MyApp.Plugs.Plug1}

If I had to take another guess the module resolution fails since my_lib is looking for a source file in its own lib folder and does not take other applications (my_app) into account.

Is there any way to use modules of the “parent” app during compilation? If so, how would I go about that?
What did I do wrong that Plug did not throw an error or return a 500, but just silently ignore my module (well, or just works for that matter )?
Did I set the default config value incorrectly? It seems to work when executing, but since VS Code (using the elixir-ls extension) complains about it, I guess there must be something not quite right with it.

Thank you for your time! I realize that this is quite the wall of text

11 comments

#metaprogramming #plug #compile #macro #elixir-ls #config

6 1550 11

2020-12-16 00:21:22 UTC

Marked As Solved

kip

ex_cldr Core Team

Since macros are primarily about manipulating code, not executing it, I’m not sure how your expectation could be met.

Nevertheless, you can evaluate AST at compile time with Code.eval_quoted/3 although as the documentation says:

Warning: Calling this function inside a macro is considered bad practice as
it will attempt to evaluate runtime values at compile time. Macro arguments are
typically transformed by unquoting them into the returned quoted expressions
(instead of evaluated).

Using your example:

defmacro __using__(opts) do  
  options = Code.eval_quoted(opts)
  modules = Keyword.get(options, :modules)

  quote do
    def get_modules(), do: unquote(modules)
  end
end

The benefit here isn’t really enough to warrant using Code.eval_quoted/3 so perhaps reverting closer to your original idea may be clearer since it too will get resolved at compile time (but in a later compiler expansion):

defmacro __using__(opts) do
  quote do
    modules = unquote(opts[:modules])
    @modules Enum.map(modules, &R.normalize_module/1)

    def get_modules(), do: @modules
  end
end

Post #9

Also Liked

kip

ex_cldr Core Team

Remember that everything in the quote block is actually runtime execution. Outside the quote is compile time execution. Therefore you are probably after:

defmacro __using__(opts) do  
  modules = 
    opts
    |> Keyword.get(:modules)
    |> Enum.map(&MyLib.normalize_module/1)

  quote do
    def get_modules(), do: unquote(modules)
  end
end

Do note however that macros both receive and return AST. So opts[:modules] may not be what you expect since it will be AST.

You may find that you need the following signature for MyLib.normalize_module/1:

def normalize_module({:__aliases__, _meta, [module_name]}) do
  module_name
end

or alternatively:

def normalize_module(module_ast) do
  Macro.expand(module_ast, __ENV__)
end

Post #7

al2o3cr

I don’t think the code being compiled in my_lib can see the code in my_app at all - it would massively complicate dependency tracking if files in the application could cause libraries to recompile.

Have you considered an approach like Ecto.Repo where callers say use MyLib, modules: [...etc...] in a module in my_app? The for loop and friends stay in my_lib but the use of them happens when my_app is compiled.

Post #2

kip

ex_cldr Core Team

Changing conn to var!(conn) will compile cleanly I believe.

Post #5

PatrickSachs

I see, for some reason I always looked at them as a way of executing code at compile time, but this makes a lot more sense.

True, I never even considered the fact that a user might want to e.g. use a runtime configuration value. Then I guess I’ll budge and do the data transformation fully at runtime.

defmacro __using__(opts) do
   modules = opts[:modules]

   quote do
    def get_modules() do 
      unquote(modules)
      |> Enum.map(modules, &R.normalize_module/1)
    end
  end
end

I initially wanted to avoid this to save performance at runtime, but this doesn’t seem to be a good way to do it. Maybe this is a case of premature optimization being evil again