mstalker

mstalker

Parallel compiler cannot find files sometimes

I’m getting an intermittent compilation error when building a project in Docker:

== Compilation error in file lib/sentry/logger.ex ==
** (MatchError) no match of right hand side value: {:error, :enoent}
    (elixir) lib/kernel/parallel_compiler.ex:198: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6
could not compile dependency :sentry, "mix compile" failed. You can recompile this dependency with "mix deps.compile sentry", update it with "mix deps.update sentry" or clean it with "mix deps.clean sentry"

Kernel.ParallelCompiler.spawn_workers/6 calls :elixir_compiler.file/2, which calls file:read_file/1, which returns {error, enoent} if it can’t find a file.

:elixir_compiler.file/2 tries to pattern-match…

{ok, Bin} = file:read_file(File)

…fails, and raises the MatchError we see above.

Here are a few thoughts:

  • I could just do what the instructions tell me to do :slight_smile: …and recompile the dependency with mix deps.compile sentry, but this only happens intermittently, so I don’t think the problem is with how I’m compiling. This is part of an automated build, too, and recompiling one dependency occasionally isn’t ideal.
  • I’m using the most recent version of Sentry, so mix deps.update sentry won’t help.
  • I don’t think mix deps.clean sentry is the answer because I’m starting off with a clean slate; I’m building the project while building a Docker image. Both deps/ and _build/ are in my .dockerignore file.
  • This only seems to happen with the Sentry library.
  • The issue is intermittent. On most attempts to build the project and the Docker image, everything compiles correctly.
  • Even though the error was in lib/sentry/logger.ex in the code snippet above, this error sometimes springs up in other files in the Sentry library.
  • My Docker image is running Erlang/OTP 20, and Elixir 1.6.1 compiled with OTP 20, on Debian GNU/Linux 8.

Do you have any suggestions for how to debug this? I’m a bit flummoxed. There’s an existing issue on the Sentry project that describes the problem, but I thought it might actually be a bug in the Elixir compiler, so I posted here, just in case :smile:

Most Liked

easco

easco

Are you seeing the problem on a Mac?

Using my mix project and docker files (which are unrelated to Sentry) on both an Mac and on our Linux-based build server, I’ve seen similar compile problems “frequently” when running in the dev environment on my Mac laptop, but never when running the container on the build server. The Mac often finishes without error - but will fail once out of 4 or 5 times.

I suspect that it may be related to the synchronization and timing between the Macintosh file system, and the container’s virtual file system. You may choose to read https://docs.docker.com/docker-for-mac/osxfs/#performance-issues-solutions-and-roadmap

But please understand that this is just a suspicion on my part - I have no verifiable evidence to prove or disprove.

josevalim

josevalim

Creator of Elixir

One idea is to use strace at the moment of compilation and trace all of the system calls happening on lib/sentry/logger.ex. Something like:

$ mix deps.get
$ enable strace on deps/sentry
$ mix deps.compile
$ disable strace
$ mix compile

For all purposes, it seems the file exists in disk. So we need to figure out if for some reasons docker is saying it does not. In case docker does say it exists and Erlang still says it doesn’t, then there is something else at play.

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New

We're in Beta

About us Mission Statement