Fl4m3Ph03n1x
Test processes using handle_continue
Background
I have a process that does nothing in it’s init function and delegates all the work to a handle_continue. I do this because the work being done in the handle_continue is quite heavy, and since this is a Worker process, I don’t want to slog it’s supervisor (and thus the entire application) with the slow initialization of a Worker (of which there can be hundreds or thousands of).
Problem
The problem here comes when testing. When using ExUnit, it will execute the code assertions right after the init function of my Worker, which I remind you, does next to nothing.
So effectively, ExUnit is sometimes running the test assertions before the Worker has even initialized. I say sometimes, because everything is concurrent, so sometimes I am lucky and the assertions run after the Worker’s handle_continue has run, sometimes they don’t.
Questions
Is there a way to make ExUnit run the assertions without forcing the Worker process to send a message in handle_continue signaling it? (think of it as forcing the Worker to broadcast a message once handle_continue is done running).
I ask this because I frown upon this idea. If I change the Worker to broadcast a message once it’s handle_continue is done, then I am just changing my production code for the sake of testing, which is something I abhor completely.
Marked As Solved
Fl4m3Ph03n1x
I feel there is some clarification needed here. The Worker process is a GenServer, that does, some work.
As all GenServers, it’s code is divided into 3 parts:
- The public API that clients can call
- GenServer calls to the process itself
handle_xcalls which do the real work
In the tests I am performing right now, I am testing the public API. Say I have the following code:
defmodule Clint.Worker do
use GenServer
defmodule State do
defstruct conn_pid: nil,
active_streams: [],
worker_id: nil,
opts: %{},
deps: %{}
end
###############
# Public API #
###############
@spec start_link(map) :: Supervisor.on_start
def start_link(args) do
deps =
args
|> Map.get(:deps, %{})
|> build_deps()
worker_id = Map.fetch!(args, :worker_id)
opts = Map.fetch!(args, :opts)
Logger.debug("Starting worker #{inspect worker_id }")
init_state = %State{worker_id: worker_id, opts: opts, deps: deps}
pname = :bananas
GenServer.start_link(__MODULE__, init_state, name: pname)
end
@spec request(atom, integer, charlist, map) :: {:ok, :received}
def request(group_name, worker_id, url, injected_deps \\ %{}) do
deps = build_deps(injected_deps)
GenServer.cast(:bananas, {:fire, url})
{:ok, :received}
end
@spec build_deps(map) :: map
defp build_deps(injected_deps), do:
Map.merge(@default_deps, injected_deps, fn _, a, b -> Map.merge(a, b) end)
#############
# Callbacks #
#############
@impl GenServer
def init(state) do
Process.flag(:trap_exit, true)
{:ok, state, {:continue, :establish_conn}}
end
@impl GenServer
def handle_continue(:establish_conn, state) do
with {:ok, conn_pid} <- Logic.establish_connection do
new_state = %{state | conn_pid: conn_pid}
{:noreply, new_state}
else
{:error, reason} -> {:stop, reason, state}
end
end
@impl GenServer
def handle_cast({:fire, url}, state) do
stream_ref = state.deps.http.get.(state.conn_pid, url)
new_state = %{state | active_streams: [stream_ref | state.active_streams]}
{:noreply, new_state}
end
end
Here the public API (what clients will call) are the start_link and request functions. I am testing the Public API of the worker, the face it shows to the world.
This fits nicely into the “Test the interface, not the implementation” rule of testing, but it is insufficient. For example, using this methodology I can’t test any handle_info calls, unless I create a worker and then send him the exact messages that trigger handle_info (which according to some community members, I should do. I am now trying out this strategy).
Is this Unit testing? Is this integration?
I argue that the Worker is my unit, so I can argue this is unit testing. Some of you will say “You are mixing your state tests with GenServer OTP and therefore this is integration testing”. I wouldn’t say you are wrong, but I will definitely say this is a very thin line, at least for me.
@chrismcg I have never used trace for testing before, and I fail to see how I could apply it here. I need to invest more time into this idea, as it looks really cool!
I am however not sure why I need to wait for the
:erlang.trace(:new, true, [:call, :return_to])
call.
I usually (try to) make my questions and topics as small and isolated as possible, to make the load on the people reading smaller. Sometimes, this comes at the cost of clarity, which I believe is the case here.
My worker does not depend on any HTTP client. It depends on an HTTP contract, which can be implemented by any client I wish. The boundary is well defined, the way I see it ![]()
Perhaps, the trouble here is in making it clear it is a boundary. Perhaps you believe I am testing too much the details of my worker which leads you to think I have no boundaries defined. You are one of the most well educated people in testing I have seen and I find it curious how your opinions differ from mine in so many areas. All I can say to defend myself is that I am following a traditional London style TDD, while injecting the dependencies directly without creating Mock modules, because I believe functional injection is the best way of passing dependencies.
Test first is not always the solution, true. This is actually the refactor of a project that I made and that I consider is a complete disaster. No better time to fix it than now ![]()
In fact, one of the many problem is that this project has not tests at all ![]()
Oh my God. Thank you so much. I don;t want to sound … ungrateful, but I am not well versed in erlang yet, so I have trouble understanding what this actually accomplishes. I can only hope this comes with great documentation for dummies like me ![]()
@sorentwo So you let the tests fail repeatedly until you hit a timeout that is slow enough? I understand you provide a maximum number of tries for the test to run, correct (50 times) ?
Also Liked
swelham
I have been dealing with this same issue recently of wanting to wait for handle_contiue to complete before I start testing my server.
I found a the simplest solution was to just put in a call to the server since the message won’t be processed until after the continue has completed. I didn’t want to wrote code in a handle_call just for testing so I used the erlang sys module for this since it provides convince debug functions for working with processes. I found calling :sys.get_state(server_pid) often did the trick. This way I don’t have to use some arbitrary sleep duration to try and guess how long it will take.
chrismcg
So I had a play around with this:
A simple server:
defmodule Tracetest.Server do
use GenServer
def start_link(arg) do
GenServer.start_link(__MODULE__, [arg])
end
@impl true
def init([arg]) do
{:ok, :some_state, {:continue, arg}}
end
@impl true
def handle_continue(_, state) do
{:noreply, state}
end
end
The test:
test "can know when handle_continue finished" do
:erlang.trace(:new, true, [:call, :return_to])
:erlang.trace_pattern({Tracetest.Server, :handle_continue, 2}, true, [:local])
{:ok, pid} = Tracetest.Server.start_link(:foo)
assert_receive {:trace, ^pid, :call,
{Tracetest.Server, :handle_continue, [:foo, :some_state]}}
assert_receive {:trace, ^pid, :return_to, {:gen_server, :try_dispatch, 4}}
assert true == true
end
I setup the same tracing calls in the iex console, started the server there, and ran flush to see what trace messages got received. There were only two so it was straightforward to match on them. I am sure your real code is a lot more complicated than this but perhaps just waiting till the method had been called rather than a return would be enough to avoid the race condition.
LostKobrakai
I just want to mention one big caveat with using tracing in that context: It’ll quite severely couple the test to the implementation.
LostKobrakai
I’m not sure this is a really worthwhile intent to have: Never change production for testing. I’d rather phrase it as creating flexible enough production code so you don’t need to add anything extra for testing the code.
In your example the worker could receive a callback, which defines the actual work.
This way you can test the runtime behaviour of the worker without any of your prod code needing to know about the callback it receives from the test, and it’s even more flexible because you can now use the worker to do all kinds of tasks and not just one specific one.
# Worker
def start_link(callback), do: GenServer.start_link(__MODULE__, callback)
def init(callback), do: {:ok, callback, {:continue, :run_callback}}
[…]
# Business Code
Worker.start_link(&fancy_callback_doing_business_logic/1)
# Test
test = self()
callback = fn -> send(test, :msg) end)
Worker.start_link(callback)
assert_receive :msg
Edit:
This also allows you to test fancy_callback_doing_business_logic without a process around it.
peerreynders
Maybe I simply prefer Inside-Out testing in order to control test maintenance costs (giving up some defect localization).
Once one becomes diligent with testing it’s important to find ways to balance the value of testing with the burden it imposes - test obsessed can go too far.
Kent Beck’s (2008) opinion just recently surfaced here again.
There are times where “should I even be testing this”, “should I be testing this particular aspect” or “should I be testing it in this manner” are valid questions.







