schnittchen

schnittchen

Excessive recompilations when nothing substantially changed

Hi,

I’m trying to track down why my testing experience is such a pain because there’s always many files being compiled… I’ve used the inotify trick from Understanding Elixir's recompilation · Milhouse to see which files these are (is there a simpler way?) and tried to understand from mix xref why those files were compiled.

So there’s runtime, compile time and exports dependencies, I learned from mix help xref. So when I do a trivial change like adding a newline in a source file, only compile time dependencies should trigger a recompile.
If I change file A, all files having a (transitive) compile time dependency on A would need to be recompiled.

This lead me to use mix xref graph --label compile --only-direct because my reasoning is that this would show me all non-transitive compile time dependencies, and I should be able to traverse from the changed file to every compiled file via a graph node connection in the output of this command.

Here’s a snippet from the output, the email.ex file has only one dependency shown.

lib/my_app/email.ex
└── lib/my_app/views/mailer_view.ex (compile)

The email file was recompiled when I made a trivial change in a file which, according to mix xref (with the invocation above), was not connected at all to the email file.

I must have some misconception here.

Most Liked

schnittchen

schnittchen

Thanks for the advice, I quickly found the culprit:

I had compile time dependencies from the router to two controllers. Because those controllers had a transitive run time dependency back to the router (redirects using route helpers) and from there to every other controller, changing any controller required the router to be recompiled.

The fix:

# change this:
get "/", MyApp.FooController, :index
# to this:
scope alias: MyApp do
  get "/", FooController, :show
end

simply because now the fully qualified module name of the controller it no longer present in the router.

Two other things I did/tried:

elixirc_options: [verbose: true]

in the project/0 callback of the MixFile, which made the effective dependency chain more obvious.

I also tried setting

config :phoenix, :plug_init_mode, :runtime

not only in dev mode, but in test mode as well, assuming that plug initialization would otherwise happen at compile time and that that would also be the case for the controllers being routed to (since they are plugs as well), but I was wrong here, the default :compile does not make all controllers compile time dependencies of the router.

schnittchen

schnittchen

I got tired of analyzing the output of mix xref so I wrote a thing… If you run into this problem please check out GitHub - schnittchen/why_did_recompile and help me improve it

axelson

axelson

Scenic Core Team

Have you given DepViz a try?

I spoke about it at Code BEAM BR last year (in english): https://www.codebeambr.com/video/12

That is not quite accurate, an alias will only cause a run-time dependency. However, compile-time dependencies are transitive so if A->compile-time->B and B->run-time->C, then when C changes, A will need to be recompiled.

riebeekn

riebeekn

Was revisiting this a bit today and realized I forgot to mention that in addition to the sugggestions above we found adding GitHub - sasa1977/boundary: Manage and restrain cross-module dependencies in Elixir projects to be super helpful. It helped surface some poorly defined boundaries we had going on.

axelson

axelson

Scenic Core Team

Not quite, you have that reversed. Now C has a compile-dependency on both A and B. The compile-time dep on B is direct, but the compile-time dep on A is transitive (because A depends on B at compile-time). This means that if the a.ex file or b.ex file is changed then c.ex needs to be recompiled (it generally doesn’t make too much difference but compilation dependencies are tracked on a per-file basis even though the dependencies are introduced on a per-module basis).

I’m pretty sure this is the blog post you are referring to:
https://medium.com/multiverse-tech/how-to-speed-up-your-elixir-compile-times-part-2-test-your-understanding-f6ff3de5eb5d

I highly highly recommend it to anyone struggling with recompilation in Elixir. It’s well worth the time to read through and test out the exercises.

Also a shameless plug for anyone that wants to narrow down where they might want to focus their recompilation efforts I wrote a tool called DepViz:

Where Next?

Popular in Questions Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement