akash-akya

akash-akya

Cross Compiling with NIF as compilation dependency

I am trying to add support for Nerves in Vix (an Image Processing library) and facing a blocker related to cross-compilation.

Background

Vix uses NIF to communicate with libvips library. It uses libvips introspection and elixir macros to generate code and documentation for all the libvips operations. Since macro expansion happen during elixir code compilation, Vix loads NIF during the code compilation.

Issue

When the compilation target and runtime target is same, it just loads the NIF elixir_make compiled. But in case of Nerves they both differ. For example in my case, I am trying to create firmware in macos targeting RPi3. During the elixir code compilation it tries to load the NIF which is created for RPi3 and fails. In other words, how to handle native dependencies during cross compilation.

Anyone faced similar issue, or have any workarounds?

Few possible workarounds I can think of

  • just create firmware on the target itself. i.e. create the firmware (compile) in the RPi3 itself.
  • slightly ugly, but we could hack around priv/* directory by replacing NIF files to match target after code compilation
  • dynamically load different NIF file (:erlang.load_nif(nif_path, 0)) based on current architecture. Again, this is ugly since I have to juggle between multiple NIF files for different targets.

Most Liked

fhunleth

fhunleth

Co-author of Nerves

Looking at your project, I don’t think this is the issue, but let me start with a general statement since this comes up periodically.

For elixir_make projects, there are few important things to make NIFs work well with Nerves:

  1. Use macros like CC and CFLAGS instead of hardcoding calls to gcc. A lot of projects already do this, so sometimes this is really easy. See Environment variables — nerves v1.10.4.
  2. Put outputs in $(MIX_APP_PATH)/priv to keep host and target binaries separated. Nerves uses Mix’s MIX_TARGET feature which will create a new build directory for each target you compile for.
  3. Use Mix’s $(ERL_EI_INCLUDE_DIR) and $(ERL_EI_LIBDIR) instead of calling Erlang in the Makefile to figure out where Erlang header files and libraries are.

The standard procedure is to copy/paste a Makefile from another Elixir project to make it work with Nerves. For an example, see circuits_spi’s Makefile for a NIF example.

Nerves uses Buildroot to handle complicated C/C++ libraries. I took a look at libvips, and they made a few design decisions that I honestly don’t find enjoyable to work around. If I were required to use it, I’d do what was suggested and make a custom Nerves system to build libvips. The Buildroot package is at Buildroot libvips. Once that’s done, the Elixir Vix library would need to properly detect that it exists. That may work.

This is a long way of saying this:

  1. Give up trying to manually cross-compile libvips for Nerves
  2. If compiling with Nerves (checking $(CROSSCOMPILE) in the Makefile is probably easiest), make sure that the mode is PLATFORM_PROVIDED_LIBVIPS. The pkg-config call should look in the right place.
  3. If crosscompiling and not using a platform-provided libvips or it can’t be found, tell the user to build a custom Nerves system and BR2_PACKAGE_LIBVIPS=y to their nerves_defconfig. While at it, the user may want to also enable other libraries since libvips optionally works with so many.

Unfortunately, this will make Vix less interesting to Nerves users since it won’t work with the official Nerves systems, but it should work for anyone who needs it.

Where Next?

Popular in Questions Top

freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

Other popular topics Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New

We're in Beta

About us Mission Statement