cheerfulstoic
Conflicting dependencies and use of the ~> operator
I feel like Elixir is getting big enough and old enough that I’m starting to experience problems with conflicting dependencies. An example from an application that I’m currently working with:
I went to try out the merquery library. This application uses version 1.0.2 of the jq library (the latest version, but from over five years ago) which in turn specifies poison ~> 4.0. The merquery library that I was adding specifies flint ~> 0.6 and flint 0.6.0, in turn, specifies poison ~> 6.0
This was actually just the beginning of the headaches and I had to chase down a few situations, making a couple of forks in the process to versions which didn’t conflict.
For a while I’ve been thinking that it’s wise practice to follow a general rule:
- When specifying dependencies for an application, prefer the
~>operator (either x.y or x.y.z generally) - When specifying dependencies for a library, prefer the
>=operator
I think pretty much everybody will agree with the first point. My thought on the second being: new versions of your dependencies will come along and, if there are bugs, you can fix your library (or if something is too difficult or you don’t have time you can add a ~> or <= constraint). But you shouldn’t stand in the way of applications wanting to use your library with newer versions of shared dependencies because probably most of the time they will work
Absolutely I think that you should consider your libraries use of each dependency. If you’re including jason and all you do is very simple usage of Jason.encode and Jason.decode, then the likelyhood that a major version will break things is low and you should probably use >=. If you’re using a library in a complex and intricate way, then you should consider if you should limit the versions you allow.
But I think that most people writing a library are used to using the ~> operator from building applications.
So my suggestions (given that this is the ideas/suggestions/proposals area of the forum):
- have some advice in the documentation
- if people are mostly in agreement, build a bot which goes to Elixir dependencies (especially more popular ones) and automatically suggests changes, along with a detailed explanation of the whys involved and things that the maintainer should consider on if they should or should not accept the change.
I’m happy to work on these things (and others) because I think it will be more and more of a big deal to help make Elixir development smoother in the long run (I’ve used npm enough to see how bad things can get 
)
Also, this post from the Ruby world has some good discussion on the topic as well.
Most Liked
zachdaniel
You can override dependencies if necessary, but I think there is something major missing from overriding dependencies, specifically I think you ought to be able to say why you are overriding a dependency, and then mix can tell you when you don’t need to anymore.
For example:
{:jq, "~> x.x", override: [:merquery]}
says “I know that merquery wants a different version, but I’m overriding it”. Then if later you update merquery, and no longer need the override, mix will instruct you to remove it. Additionally, if you add some new dependency that wants an older version of jq, you have to acknowledge that you are also overriding that dependency.
This makes it a bit safer to use override as a consumer of libraries, which I think helps alleviate this issue a bit as well.
zachdaniel
Made a PR to the library author guidelines with some of this in mind: Advise library authors on how best to depend on child dependencies by zachdaniel · Pull Request #14080 · elixir-lang/elixir · GitHub
ericmj
There is more to it than just changing the library name. There are lots of global names that would have to be modified to ensure uniqueness, application names, module names, process names, ets table names etc.
zachdaniel
I’m suggesting that this is built into the mix dependency resolver. Right now the following happens all the time:
- I want to add
footo my app. I already havebarandbaz. foodepends on a newer version ofbar.- I can’t update
barbecausebazdepends on it. - I check how they
bazandfoousebar, and confirm that its fine to just override. - So I add
{:bar, "~> ...", override: true} - Someone else adds
buzzto the app, which also depends on an old version ofbar. - Problem 1: We never find out that we just overrode
barfor the sake ofbuzzas well. - Next,
fooreleases an update that depends on more stuff from the old version ofbar. - Problem 2: mix tells us we can update
foo, so we update it and have bugs.
If instead of override: true, I could say:
{:bar, "~> x.x", override: [foo: "x.x.x"]}
which would say “this override only overrides the dependency that foo at exactly version x.x.x has on bar”, then we are protected from any of those accidental changes.
Adding buzz would produce an appropriate dependency conflict warning, solving Problem 1. We can then go look at the code/docs and decide if we want to override the bar dependency for that version of buzz as well.
foo won’t appear to be automatically upgradeable, solving Problem 2.
al2o3cr
FWIW, >= in libraries can cause its own headaches, especially if the package being referenced doesn’t follow semver.
For instance, here’s a pair of issues from standardrb that were ultimately driven by interactions between >= and already-locked versions:
Popular in Proposals: Ideas
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"