SeanShubin

SeanShubin

How do I test things like IO, File, and System?

New to elixir, looking for guidance regarding how to test things like IO, File, and System

Whenever I learn a new language, I always make sure I know how to test it by implementing the following “Hello, world!” application.

defmodule HelloApp do
  def main() do
    time_unit = :microsecond
    microseconds_before = System.monotonic_time time_unit
    target = System.argv |> List.first |> File.read!
    IO.puts "Hello, #{target}!"
    microseconds_after = System.monotonic_time time_unit
    microseconds_duration = microseconds_after - microseconds_before
    IO.puts "Took #{microseconds_duration} microseconds"
  end
end

I intentionally choose the requirements for this application because they have the most extreme combination of easy-to-write yet hard-to-test that I can think of.
As it is in any language, the trick is to isolate the side effects behind an abstraction of some kind.
With Elixir this seems harder to do than normal, but perhaps that is simply because I don’t know the proper language constructs to hide the side effects behind an abstraction.
I don’t see a way to swap out System, IO, and File with stub versions because they are sitting in a global namespace.
I did look at ExUnit.CaptureIO, but rejected it for 2 reasons.
First reason is that as the capture is happening globally, I become vulnerable to one test affecting another.
Second reason is that CaptureIO is not generalizable to File and System.

Ideally I would like each test to validate the implementation in a sandbox independent of other tests with all the side-effecting modules swapped out.

I was able to get 100% test coverage by inverting the dependencies, so it looks like this may indeed be the right answer.
However, since this is my first ever Elixir program, I want to check with people with more Elixir experience than me to make sure I am on the right track.

What do you think of this solution to getting IO, File, and System under 100% test coverage?
Can you guide me to a better solution?

For the implementation, I invert each side-effecting dependency

defmodule HelloAppInverted1 do
  def main(collaborators) do
    system = collaborators.system
    file = collaborators.file
    io = collaborators.io
    time_unit = :microsecond
    microseconds_before = system.monotonic_time.(time_unit)
    target = system.argv.() |> List.first |> file.read!.()
    io.puts.("Hello, #{target}!")
    microseconds_after = system.monotonic_time.(time_unit)
    microseconds_duration = microseconds_after - microseconds_before
    io.puts.("Took #{microseconds_duration} microseconds")
  end
end

For production, I configure to use the real thing:

defmodule HelloAppEntry1 do
  def main() do
    system = %{
      :monotonic_time => &System.monotonic_time/1,
      :argv => &System.argv/0
    }
    file = %{
      :read! => &File.read!/1
    }
    io = %{
      :puts => &IO.puts/1
    }
    collaborators = %{
      :system => system,
      :file => file,
      :io => io
    }
    HelloAppInverted1.main(collaborators)
  end
end

For testing, I configure to use stubs:

defmodule HelloAppInverted1Test do
  use ExUnit.Case
  @moduletag timeout: 1_000

  test "say hello to world" do
    tester = create_tester(
      %{
        :command_line_arguments => ["configuration.txt"],
        :remaining_monotonic_time_values => [1000, 1234],
        :file_contents_by_name => %{
          "configuration.txt" => "world"
        },
        :lines_emitted => []
      }
    )

    tester.run.()

    assert tester.lines_emitted.() == ["Hello, world!", "Took 234 microseconds"]
  end

  def create_tester(initial_state) do
    state_process = create_process(initial_state)

    monotonic_time = fn time_unit ->
      send(state_process, {:get_monotonic_time, time_unit, self()})
      receive do
        x -> x
      end
    end

    argv = fn ->
      send(state_process, {:get_argv, self()})
      receive do
        x -> x
      end
    end

    read! = fn file_name ->
      send(state_process, {:get_file_contents, file_name, self()})
      receive do
        x -> x
      end
    end

    puts = fn output_string ->
      send(state_process, {:puts, output_string})
    end

    lines_emitted = fn ->
      send(state_process, {:get_lines_emitted, self()})
      receive do
        x -> x
      end
    end

    system = %{
      :monotonic_time => monotonic_time,
      :argv => argv
    }
    file = %{
      :read! => read!
    }
    io = %{
      :puts => puts
    }
    collaborators = %{
      :system => system,
      :file => file,
      :io => io
    }

    run = fn ->
      HelloAppInverted1.main(collaborators)
    end

    %{
      :run => run,
      :lines_emitted => lines_emitted
    }
  end

  def create_process(state) do
    spawn_link(fn -> loop(state) end)
  end

  def consume_monotonic_time(state) do
    [time_value | remaining_time_values ] = state.remaining_monotonic_time_values
    new_state = Map.replace(state, :remaining_monotonic_time_values, remaining_time_values)
    {new_state, time_value}
  end

  def append_line(state, line) do
    new_lines_emitted = [line | state.lines_emitted]
    Map.replace(state, :lines_emitted, new_lines_emitted)
  end

  def loop(state)do
    %{
      :command_line_arguments => ["configuration.txt"],
      :remaining_monotonic_time_values => [1000, 1234],
      :file_contents_by_name => %{
        "configuration.txt" => "world"
      },
      :lines_emitted => []
    }
    receive do
      {:get_lines_emitted, caller} ->
        send(caller, Enum.reverse(state.lines_emitted))
        loop(state)
      {:get_monotonic_time, :microsecond, caller} ->
        {new_state, monotonic_time_value} = consume_monotonic_time(state)
        send(caller, monotonic_time_value)
        loop(new_state)
      {:get_argv, caller} ->
        send(caller, state.command_line_arguments)
        loop(state)
      {:get_file_contents, file_name, caller} ->
        file_contents = state.file_contents_by_name[file_name]
        send(caller, file_contents)
        loop(state)
      {:puts, line} ->
        new_state = append_line(state, line)
        loop(new_state)
      x ->
        raise "unmatched pattern #{inspect x}"
    end
  end
end
  • edit, I said production twice, the last snippet was for testing, not production

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hey @SeanShubin welcome!

While true, I think there’s more to say here. As a functional language, the first thing to do wouldn’t be to reach for an abstraction, it would be to reach for extraction (I couldn’t resist). Basically, work to extract the code with side effects from the logic of your program.

Then much of your logic can be tested just with pure inputs and outputs. In particular the values you’re reaching for like the path to read or write a file to should definitely not be produced in the same function that actually does the writing or reading to the file.

sodapopcan

sodapopcan

Hello @SeanShubin and welcome.

Your target audience is a bit unclear here. Your writing style is attracting responses from seasoned Elixir developers and seasoned Elixir developers are well aware they are not working in a purely functional language. Erlang was developed to solve a practical problem and its functional properties emerged as a consequence of that. It’s never been academic. Elixir developers generally favour pure functions for the most part and push all side-effects to a particular boundary which isn’t necessarily the top boundary (depending on how you think about your system).

I’d say go for it if you want. You will generally find people around here encouraging others to code however they see fit. If you look through some prominent open source Elixir projects there are all sorts of different styles.

dimitarvp

dimitarvp

FWIW, what you are describing is more in the lane of integration / end-to-end testing than unit testing. Having pure functions that don’t rely on side effects and testing them with various inputs is what I would do in an imperative / OOP language as well, not only in an FP one (like Elixir). Testing those and/or their modules is what’s unit testing.

If you anticipate wildcard events – like time jumps due to DST (which just happened last night in Europe) – then craft a specific test that mimics the scenario?

As for files, I can partially side with you because it’s not a one-minute job to imitate conditions with no disk space remaining, exhausted file descriptors etc. but with the help of the already mentioned Mox – or Patch – you can find the underlying Erlang code or dig in the docs for the right error responses when these things happen and just return them straight away in a mock.

(Me and many others in the Elixir community use this technique to a crushing success, very often, e.g. when you have an API client contacting 3rd party servers you can just directly return 404, 500 or a validation error in your mock. Or 429 for too many responses etc. You can exercise the unhappy paths very easily. Not super quickly, mind you, it’s still a job where you must apply rigor and cover your expected error conditions exhaustively. But it’s possible and it’s done regularly in commercial projects.)

I don’t think there’s a magic silver bullet here. As programming matures as a profession / discipline, people at large mostly understand that 99% of your code should be pure and the rest 1% should be well-covered with integration tests utilizing mocking.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

The thing is, for all practical purposes, I already have this. For stubbing out side effects involving interaction with APIs Mox basically is what you’re asking for with DI, and for interacting with the DB you can either do the same thing or you can use Ecto sandboxes. Personally I prefer Ecto sandboxes because unless you’re doing very simple things with a DB the stubs would be pretty complicated to code.

GitHub - jjh42/mock: Mocking library for Elixir language uses meck (an erlang library) to do actual code substitution which seems to be what you’re referring to in your last paragraph.

Languages with a strong reliance on DI aren’t my main wheelhouse so I’m sure I could be missing some nuance, but to me Mox seems like it is exactly what you’re looking for.

SeanShubin

SeanShubin

Thanks! I will definitely check it out.

Where Next?

Popular in Questions Top

pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
Kagamiiiii
Student & New to elixir. Nice language. I want to convert a english character, e.g. “a”, which is stored in a variable, to it’s asci...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New

Other popular topics Top

freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New

We're in Beta

About us Mission Statement