SeanShubin
How do I test things like IO, File, and System?
New to elixir, looking for guidance regarding how to test things like IO, File, and System
Whenever I learn a new language, I always make sure I know how to test it by implementing the following “Hello, world!” application.
defmodule HelloApp do
def main() do
time_unit = :microsecond
microseconds_before = System.monotonic_time time_unit
target = System.argv |> List.first |> File.read!
IO.puts "Hello, #{target}!"
microseconds_after = System.monotonic_time time_unit
microseconds_duration = microseconds_after - microseconds_before
IO.puts "Took #{microseconds_duration} microseconds"
end
end
I intentionally choose the requirements for this application because they have the most extreme combination of easy-to-write yet hard-to-test that I can think of.
As it is in any language, the trick is to isolate the side effects behind an abstraction of some kind.
With Elixir this seems harder to do than normal, but perhaps that is simply because I don’t know the proper language constructs to hide the side effects behind an abstraction.
I don’t see a way to swap out System, IO, and File with stub versions because they are sitting in a global namespace.
I did look at ExUnit.CaptureIO, but rejected it for 2 reasons.
First reason is that as the capture is happening globally, I become vulnerable to one test affecting another.
Second reason is that CaptureIO is not generalizable to File and System.
Ideally I would like each test to validate the implementation in a sandbox independent of other tests with all the side-effecting modules swapped out.
I was able to get 100% test coverage by inverting the dependencies, so it looks like this may indeed be the right answer.
However, since this is my first ever Elixir program, I want to check with people with more Elixir experience than me to make sure I am on the right track.
What do you think of this solution to getting IO, File, and System under 100% test coverage?
Can you guide me to a better solution?
For the implementation, I invert each side-effecting dependency
defmodule HelloAppInverted1 do
def main(collaborators) do
system = collaborators.system
file = collaborators.file
io = collaborators.io
time_unit = :microsecond
microseconds_before = system.monotonic_time.(time_unit)
target = system.argv.() |> List.first |> file.read!.()
io.puts.("Hello, #{target}!")
microseconds_after = system.monotonic_time.(time_unit)
microseconds_duration = microseconds_after - microseconds_before
io.puts.("Took #{microseconds_duration} microseconds")
end
end
For production, I configure to use the real thing:
defmodule HelloAppEntry1 do
def main() do
system = %{
:monotonic_time => &System.monotonic_time/1,
:argv => &System.argv/0
}
file = %{
:read! => &File.read!/1
}
io = %{
:puts => &IO.puts/1
}
collaborators = %{
:system => system,
:file => file,
:io => io
}
HelloAppInverted1.main(collaborators)
end
end
For testing, I configure to use stubs:
defmodule HelloAppInverted1Test do
use ExUnit.Case
@moduletag timeout: 1_000
test "say hello to world" do
tester = create_tester(
%{
:command_line_arguments => ["configuration.txt"],
:remaining_monotonic_time_values => [1000, 1234],
:file_contents_by_name => %{
"configuration.txt" => "world"
},
:lines_emitted => []
}
)
tester.run.()
assert tester.lines_emitted.() == ["Hello, world!", "Took 234 microseconds"]
end
def create_tester(initial_state) do
state_process = create_process(initial_state)
monotonic_time = fn time_unit ->
send(state_process, {:get_monotonic_time, time_unit, self()})
receive do
x -> x
end
end
argv = fn ->
send(state_process, {:get_argv, self()})
receive do
x -> x
end
end
read! = fn file_name ->
send(state_process, {:get_file_contents, file_name, self()})
receive do
x -> x
end
end
puts = fn output_string ->
send(state_process, {:puts, output_string})
end
lines_emitted = fn ->
send(state_process, {:get_lines_emitted, self()})
receive do
x -> x
end
end
system = %{
:monotonic_time => monotonic_time,
:argv => argv
}
file = %{
:read! => read!
}
io = %{
:puts => puts
}
collaborators = %{
:system => system,
:file => file,
:io => io
}
run = fn ->
HelloAppInverted1.main(collaborators)
end
%{
:run => run,
:lines_emitted => lines_emitted
}
end
def create_process(state) do
spawn_link(fn -> loop(state) end)
end
def consume_monotonic_time(state) do
[time_value | remaining_time_values ] = state.remaining_monotonic_time_values
new_state = Map.replace(state, :remaining_monotonic_time_values, remaining_time_values)
{new_state, time_value}
end
def append_line(state, line) do
new_lines_emitted = [line | state.lines_emitted]
Map.replace(state, :lines_emitted, new_lines_emitted)
end
def loop(state)do
%{
:command_line_arguments => ["configuration.txt"],
:remaining_monotonic_time_values => [1000, 1234],
:file_contents_by_name => %{
"configuration.txt" => "world"
},
:lines_emitted => []
}
receive do
{:get_lines_emitted, caller} ->
send(caller, Enum.reverse(state.lines_emitted))
loop(state)
{:get_monotonic_time, :microsecond, caller} ->
{new_state, monotonic_time_value} = consume_monotonic_time(state)
send(caller, monotonic_time_value)
loop(new_state)
{:get_argv, caller} ->
send(caller, state.command_line_arguments)
loop(state)
{:get_file_contents, file_name, caller} ->
file_contents = state.file_contents_by_name[file_name]
send(caller, file_contents)
loop(state)
{:puts, line} ->
new_state = append_line(state, line)
loop(new_state)
x ->
raise "unmatched pattern #{inspect x}"
end
end
end
- edit, I said production twice, the last snippet was for testing, not production
Most Liked
benwilson512
Hey @SeanShubin welcome!
While true, I think there’s more to say here. As a functional language, the first thing to do wouldn’t be to reach for an abstraction, it would be to reach for extraction (I couldn’t resist). Basically, work to extract the code with side effects from the logic of your program.
Then much of your logic can be tested just with pure inputs and outputs. In particular the values you’re reaching for like the path to read or write a file to should definitely not be produced in the same function that actually does the writing or reading to the file.
sodapopcan
Hello @SeanShubin and welcome.
Your target audience is a bit unclear here. Your writing style is attracting responses from seasoned Elixir developers and seasoned Elixir developers are well aware they are not working in a purely functional language. Erlang was developed to solve a practical problem and its functional properties emerged as a consequence of that. It’s never been academic. Elixir developers generally favour pure functions for the most part and push all side-effects to a particular boundary which isn’t necessarily the top boundary (depending on how you think about your system).
I’d say go for it if you want. You will generally find people around here encouraging others to code however they see fit. If you look through some prominent open source Elixir projects there are all sorts of different styles.
dimitarvp
FWIW, what you are describing is more in the lane of integration / end-to-end testing than unit testing. Having pure functions that don’t rely on side effects and testing them with various inputs is what I would do in an imperative / OOP language as well, not only in an FP one (like Elixir). Testing those and/or their modules is what’s unit testing.
If you anticipate wildcard events – like time jumps due to DST (which just happened last night in Europe) – then craft a specific test that mimics the scenario?
As for files, I can partially side with you because it’s not a one-minute job to imitate conditions with no disk space remaining, exhausted file descriptors etc. but with the help of the already mentioned Mox – or Patch – you can find the underlying Erlang code or dig in the docs for the right error responses when these things happen and just return them straight away in a mock.
(Me and many others in the Elixir community use this technique to a crushing success, very often, e.g. when you have an API client contacting 3rd party servers you can just directly return 404, 500 or a validation error in your mock. Or 429 for too many responses etc. You can exercise the unhappy paths very easily. Not super quickly, mind you, it’s still a job where you must apply rigor and cover your expected error conditions exhaustively. But it’s possible and it’s done regularly in commercial projects.)
I don’t think there’s a magic silver bullet here. As programming matures as a profession / discipline, people at large mostly understand that 99% of your code should be pure and the rest 1% should be well-covered with integration tests utilizing mocking.
benwilson512
The thing is, for all practical purposes, I already have this. For stubbing out side effects involving interaction with APIs Mox basically is what you’re asking for with DI, and for interacting with the DB you can either do the same thing or you can use Ecto sandboxes. Personally I prefer Ecto sandboxes because unless you’re doing very simple things with a DB the stubs would be pretty complicated to code.
GitHub - jjh42/mock: Mocking library for Elixir language uses meck (an erlang library) to do actual code substitution which seems to be what you’re referring to in your last paragraph.
Languages with a strong reliance on DI aren’t my main wheelhouse so I’m sure I could be missing some nuance, but to me Mox seems like it is exactly what you’re looking for.
SeanShubin
Thanks! I will definitely check it out.







