tjhanley

tjhanley

Mix deps.get - with Artifactory?

I am currently evaluating Elixir for building tools for my company. I like how mix manages dependencies. I have been looking around to see how to configure mix to look at something like Artifactory in our own private cloud for installing libraries we create. I see you can use git: and github: to specify remotes. Is there something like this for Artifactory?

Most Liked

OvermindDL1

OvermindDL1

I’ve personally not seen one for artifactory but it is extensible in mix.deps as I recall, you’d just specify your own SCM module (maybe a manager too?) and you can use it to specify dep locations too as I recall. I’m unsure on the details as I’ve not made one, but maybe @josevalim or someone will pop in to clarify how with links to documentation? :slight_smile:

rmertz92

rmertz92

Artifactoy has had an open jira item since about 2016 to support this, that appears to now be on the 2021 roadmap.

Additionally, I was able to host elixir client libs in a generic artifactory repo by using a modified version of the mix hex.registry build that my much-more-experienced co-worker helped me with. It borrows from the source of mix hex.registry build but uses the existing metadata files and modifies them instead of re-writing them.

Pipeline:

  • create directory structure outlined by hexpm here
  • download meta data files from the artifactory repo (names, versions, packages/*)
  • add new .tar files to tarballs/
  • use our custom mix task (containerized in a CI pipeline) to update the metadata files
  • upload new .tar files and updated metadata files
chulkilee

chulkilee

For some reasons - such as auditing and network requirement, etc. - having Elixir support in such artifact/package management software is often wanted, even elixir package is just source code package. Note that go is adopting “package” system and Artifactory and Nexus support Go as well…

These are tickets for Elixir, for the reference.

For related discussion and resource:

rmertz92

rmertz92

@matreyes Unfortunately, I’m not at liberty to share the mix task itself, but here’s some more info that should help.

First, take a look at the docs here: https://hex.pm/docs/self_hosting around self-hosting. Follow the guide, particularly, the section around “Building the registry”. Take a look at the contents of the dir you created - you’ll see some folders and files (names, versions, public_key, and packages/*). These are the “metadata” files that I was talking about. There is also the tarballs directory (more on that below)

If you’re hosting the registry yourself, have direct access to the server hosting it, and you want to add a new package, you can simply add the generated tar to the tarballs/ dir and re-run the mix hex.registry build ... command, just like what is outlined in the link above.

This is where the artifactory difficulty comes in - when you’re updating a self-hosted registry, all of your tar files need to be present. All those metadata files I mentioned - they get regenerated based on the contents of the tarballs/ dir. If you’re using a CI pipeline, for example, to publish client libs that means you need to pull the contents of your entire registry (all those metadata files + all the tarballs) into your pipeline’s workspace, add your new tarballs, re-build the registry and then push it back up to wherever you pulled it from (whether thats artifactory, or somewhere else). That is what drove the solution I mentioned above - as the registry grows, pulling down all those tarballs is 1) completely unnecessary and 2) will become intractable over time.

This is where the custom mix task comes in - the biggest difference between our mix task and the mix task mix hex.registry build ... is that we don’t re-generate the metadata files, we update them. All we need to pull down from artifactory are the metadata files into our CI pipeline’s workspace - we don’t need any of the existing tarballs. Then we add the new tarballs locally, run the custom mix task, and push it all up to artifactory.

There is an obvious simpler solution if you’re not tied to artifactory and simply want to just host a private registry somewhere yourself - on the machine you’re hosting your registry, just write a simple API in front of it to accept tarballs for your packages, add them in the tarballs/ dir and run the mix hex.registry build ... command.

The solution I described was necessary simply because 1) we’re tied to artifactory and 2) we can’t set up any custom scripts/tasks to run on artifactory when things are uploaded to a repository.

Our custom mix task is simply a modified, self-serving version of the source of the mix hex.registry build ... command. I would suggest spending some time looking over the source here: https://github.com/hexpm/hex/blob/v0.21.2/lib/mix/tasks/hex.registry.ex as well as the hex docs here: https://hexdocs.pm/hex/Mix.Tasks.Hex.Registry.html

Hope that helps! :blush:

fcabestre

fcabestre

Artifactory is for binary artifacts which can be linked at compile time with your application, like JAR files in Java. AFAIK, there is no such thing as linkable binaries in Elixir. Like with Golang the dependency manager gets sources which are in some way vendored and compiled along your application. In short, I don’t see more use of Artifactory for Elixir than for Golang for example.

Where Next?

Popular in Questions Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New

Other popular topics Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement