Fl4m3Ph03n1x
Mock is crashing process in umbella project
Background
I have an umbrella project, where I run mix test from the root.
In one of the apps, I am mocking the File module using the Mock library.
Problem
The issue here is that when I run mix test the process dies, with no error message to show:
Manager.Impl.Store.ReaderTest [test/unit/store/reader_test.exs]
* test list_syndicates/1 returns the list of all known syndicates [L#496]** (EXIT from #PID<0.98.0>) killed
Code
The code of the test is as follows:
defmodule Manager.Impl.Store.ReaderTest do
use ExUnit.Case, async: false
alias Manager.Impl.Store.Reader
import Mock
setup do
%{
file_io: File,
paths: [syndicates: ["syndicates.json"]]
}
end
describe "lists syndicates" do
test_with_mock "returns the list of all known syndicates", %{paths: paths} = deps, File, [],
read: fn _filename -> {:ok, "[\"utc\"]"} end do
# Act
actual = Reader.list_syndicates(deps)
expected = {:ok, [%Syndicate{name: "UTC", id: :utc, catalog: []}]}
expected_path = Path.join(paths[:syndicates])
# Assert
assert actual == expected
assert_called(File.read(expected_path))
end
end
end
In comparison, the follow test (which does not use mock) works just fine:
defmodule Manager.Impl.Store.ReaderTest do
@moduledoc false
use ExUnit.Case, async: false
alias Manager.Impl.Store.Reader
import Mock
setup do
%{
paths: [syndicates: ["syndicates.json"]]
}
end
describe "list_syndicates/1" do
defmodule FileMockListSyndicates do
@moduledoc false
def read(path) do
assert path == "syndicates.json"
{:ok, "[\"utc\"]"}
end
end
setup do
%{
file_io: Manager.Impl.Store.ReaderTest.FileMockListSyndicates
}
end
test "returns the list of all known syndicates",
%{paths: paths} = deps do
# Act
actual = FileSystem.list_syndicates(deps)
expected = {:ok, [%Syndicate{name: "UTC", id: :utc, catalog: []}]}
# Assert
assert actual == expected
end
end
end
To me this is rather surprising. One alternative crashes the process with no error message, while the other makes everything work.
To me, this indicates one of three problems:
- A problem with the library Mock
- A problem with my setup of the library
- A problem with the test that causes the process to crash
I believe the second and third options to be the most probable, but without any information about the error, I can’t be sure. The process simply dies.
Question
Why is my process dying, and how can I fix it?
Marked As Solved
Fl4m3Ph03n1x
At the time of this writing, I have tested all major mocking libraries (Mock, Mox and Mimic).
Both Mock and Mimic rely on :meck underneath. Rather surprisingly, I get the exact same issue with Mimic as I got with Mock, i.e., the process crashes without warning.
This leaves me to the only logical conclusion, which is that the problem I am facing is related to the underlying system that servers as a base for both libraries: :meck. This issue does not happen using Mox.
I have therefore decided not to use Mock (nor Mimic) for the application and I am instead injecting the dependencies directly into the functions that need them. This last approach is very lightweight and does allow for async: true which gives a noticeable speed increase when running mix test.
Because I am only using a very small portion of the external system’s API (2 - 3 functions) this fits well with my needs. However, If I were to use all functions from said API (let’s say 20) this solution would be rather difficult to manage.
I don’t expect the affected modules to evolve in such a direction, so for the time being, I am rather happy.
Here is a sample test for those searching for inspiration (using File as an example):
setup do
%{
paths: [products: ["products.json"]]
}
end
test "returns list of available products from given syndicate", %{paths: paths} = deps do
read_fn = fn filename ->
assert filename == Path.join(paths[:products])
{:ok, "[{\"name\": \"utc\"}]"}
end
deps = Map.put(deps, :io, %{read: read_fn})
syndicate = Syndicate.new(name: "UTC", id: :utc)
assert FileSystem.list_products(syndicate, deps) ==
{:ok, [Product.new(%{"name" => "Bananas"})]}
end
Also Liked
LostKobrakai
Mock depends on :meck, which does swap out the complete module within the runtime – as in make the VM unload the existing module and load the module with the mock code. That architecture cannot support concurrent tests. The best improvement to get would be better errors or disclaimers.
What you call an issue is imo a good driver for well rounded mocking.
There’s the guideline of “don’t mock what you don’t own”. You don’t own the API of File – the core team does. Elixir does well with not doing backwards incompatible changes, but they could always add new return values and such. You might not become aware of those additional return values, so you won’t be testing for those, which might break your code in production while even well setup tests – working on an incomplete assumption of the interface – would suggest everything is fine.
Instead you want to create your own interface (in the form of a behaviour), around the actual usecases you have for interacting with the filesystem. Let’s call it MyApp.FileStorage. Then you own the interface between your code and the underlying implementation using File’s API (MyApp.FileStorage.LocalFiles), as well as the implementation you use in tests (MyApp.FileStorage.Mock).
Changes in File’s APIs then no longer affect your mocked interface MyApp.FileStorage. They only affect the implementation MyApp.FileStorage.LocalFiles, which you hopefully tested separately without mocks to ensure it works correctly. Those tests hopefully fail before you push to production. All tests using the mock implementation would be unaffected.
One sideeffect of that approach is also that your interface might becomes smaller. Instead of the whole File API you’ll likely shrink the mocked interface to a few more select things your codebase actually needs, potentially even shrinking the number of possible parameters and return values as well. Complex tasks, which require multiple calls to File API might become a single callback on your behaviour, again simplifying the interface and how much work it would be to mock.
LostKobrakai
I certainly have made my point and I’ll stop adding comments in regards to that going forward. Though I’d argue that my points will aid in having more code under control, and therefore being able to be tested as much as possible, rather than less. The gaps will be exactly the impossible to cover gaps one has with any approach. So I think we do align on the goal.
LostKobrakai
So you cannot use automated tests – that’s a real and valid limitation. But even if you cannot have that specific benefit of testing the production implementation (which you couldn’t test anyways) you still get all the benefits of having the mocked interface not mapping to that external http API.
The only thing changes in that http api can break is your implementation of your own interface with said http API. It however cannot change your interface itself and to a certain degree your mock can get away not being affected as well.
There’s a few ways your external http api can change:
- Endpoints change, but there are alternatives – potentially doing multiple requests – still using the same data as before, ultimately returning all the date you need.
- HTTP API is the interface → You need to adjust all code/tests/mocks interacting with that externally supplied interface
- You have your own interface → You adjust the implementation with the http API only.
- Endpoints change and either need more data than before or no longer return data you depended on before
- HTTP API is the interface → You need to adjust all code/tests/mocks interacting with that externally supplied interface
- You have your own interface → You need to adjust your interface by the minimal amount of changes the http api changes pushed onto your interface (might just be a subset of the http api changes)
- Stateful interaction between multiple API calls change
- HTTP API is the interface → You need to adjust all code/tests/mocks interacting with that externally supplied interface
- You have your own interface → If those changes are contained within a single callback of your interface, then only the implementation with the http api might need to change. If it affects interaction between multiple callbacks of the interface, then any interaction with your interface might need adjustment. You might however also be able to just adjust the implementation with the http api by introducing your own stateful handling correcting for the changes vs how your interface is used.
Actual real life changes might be a combination of those cases.
Having you own interface here is therefore an effective means of limiting the effects external changes can have onto your system. That’s essentially the idea around anti-corruption layers you might be reading about in hexagonal or onion architecture, though they usually explain that in regards to data not computation.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"