Sorry. I meant that you shouldn’t see a difference between mix run -e "" across Elixir versions. But I expect mix run -e "" to indeed have some additional time as it has to start all apps (and you have ~100 of them!)

I spent 15 minutes optimizing app loading and it now loads apps concurrently! If you are willing to give it a try, you can find it in the v1.12 branch too. Since you have 6 cores, I am hoping it will at least go down to 0.2s.

So hopefully this and the upcoming phoenix change shaves ~1.1s from your times. I will keep investigating!

EDIT 1: The 15 minutes actually became 1h because it turns out the code was reaching to a central process. I was able to optimize it by adding a fast paths to apps that come from deps (which are the huge majority). In a sample project I created with 50 deps, times went from 180ms to 50ms!

EDIT 2: I got an app with 150 deps and app loading time went from 920ms to 120ms!

Elixir v1.12.2 is out with the improvements in this thread.

Hi @marcandre!

One of the biggest changes in Elixir v1.11 was the tracking of application calls. We will now warn if you call an application you don’t depend on. This means that the loading of applications, which was done after compilation, is now done before compilation. This is the process of parsing .app files and loading them into memory. This probably explains a chunk of your time, since disabling app loading saves 0.8s. If you disable it, they are loaded and started at once later on.

Some extra suggestions:

1. Can you please measure the times for mix run -e ""? You are not supposed to see a large difference on those. Btw, what is the Elixir version you are measuring against?

2. You can use MIX_DEBUG=1 to track the time spent on each task

3. Phoenix master (upcoming v1.6) no longer requires the :phoenix compiler and that should save some additional time

4. mix compilexxx takes a lot of time because we need to go through all applications, load all existing mix tasks, and see if any of them may match. This is probably another action we can parallelize.

That doesn’t mean we can’t optimize things though. Application loading, for instance, is not parallel today, and we should be able to do so somewhat trivially.

As an appreciation for your work, I would be willing to jump into this codebase, if you could share it with me, or jump into a call and explore some ideas with you. Or perhaps sharing the mix.exs + mix.lock can be enough to reproduce a lot of it (you can erase lib and run again to check). Just let me know.

It will be on v1.12.2, probably in a week or two.

Just one note on this. Some of the work in the :phoenix compiler is still necessary but it should be a bit faster. In particular, the work that is done by the :phoenix compiler is to look into view templates directories to find if new templates have been added. It is done by this function:

Maybe there is a faster way to compute this.

Btw, I think I no longer need access to the repo, as my home-made 150 deps monster lock file reproduced the slow downs locally. Thanks for checking though!