mstalker
Parallel compiler cannot find files sometimes
I’m getting an intermittent compilation error when building a project in Docker:
== Compilation error in file lib/sentry/logger.ex ==
** (MatchError) no match of right hand side value: {:error, :enoent}
(elixir) lib/kernel/parallel_compiler.ex:198: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6
could not compile dependency :sentry, "mix compile" failed. You can recompile this dependency with "mix deps.compile sentry", update it with "mix deps.update sentry" or clean it with "mix deps.clean sentry"
Kernel.ParallelCompiler.spawn_workers/6 calls :elixir_compiler.file/2, which calls file:read_file/1, which returns {error, enoent} if it can’t find a file.
:elixir_compiler.file/2 tries to pattern-match…
{ok, Bin} = file:read_file(File)
…fails, and raises the MatchError we see above.
Here are a few thoughts:
- I could just do what the instructions tell me to do
…and recompile the dependency with mix deps.compile sentry, but this only happens intermittently, so I don’t think the problem is with how I’m compiling. This is part of an automated build, too, and recompiling one dependency occasionally isn’t ideal. - I’m using the most recent version of Sentry, so
mix deps.update sentrywon’t help. - I don’t think
mix deps.clean sentryis the answer because I’m starting off with a clean slate; I’m building the project while building a Docker image. Bothdeps/and_build/are in my.dockerignorefile. - This only seems to happen with the Sentry library.
- The issue is intermittent. On most attempts to build the project and the Docker image, everything compiles correctly.
- Even though the error was in
lib/sentry/logger.exin the code snippet above, this error sometimes springs up in other files in the Sentry library. - My Docker image is running Erlang/OTP 20, and Elixir 1.6.1 compiled with OTP 20, on Debian GNU/Linux 8.
Do you have any suggestions for how to debug this? I’m a bit flummoxed. There’s an existing issue on the Sentry project that describes the problem, but I thought it might actually be a bug in the Elixir compiler, so I posted here, just in case 
Most Liked
easco
Are you seeing the problem on a Mac?
Using my mix project and docker files (which are unrelated to Sentry) on both an Mac and on our Linux-based build server, I’ve seen similar compile problems “frequently” when running in the dev environment on my Mac laptop, but never when running the container on the build server. The Mac often finishes without error - but will fail once out of 4 or 5 times.
I suspect that it may be related to the synchronization and timing between the Macintosh file system, and the container’s virtual file system. You may choose to read https://docs.docker.com/docker-for-mac/osxfs/#performance-issues-solutions-and-roadmap
But please understand that this is just a suspicion on my part - I have no verifiable evidence to prove or disprove.
josevalim
One idea is to use strace at the moment of compilation and trace all of the system calls happening on lib/sentry/logger.ex. Something like:
$ mix deps.get
$ enable strace on deps/sentry
$ mix deps.compile
$ disable strace
$ mix compile
For all purposes, it seems the file exists in disk. So we need to figure out if for some reasons docker is saying it does not. In case docker does say it exists and Erlang still says it doesn’t, then there is something else at play.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"