kaa.python
Task.await terminates GenServer because of timeout. How to fix?
My Genserver terminates after a little while, after sending a few http requests. I can’t understand the reason:
[error] GenServer MyGenServer terminating
** (stop) exited in: Task.await(%Task{owner: #PID<0.420.0>, pid: #PID<0.1054.0>, ref: #Reference<....>}, 5000)
** (EXIT) time out
(elixir) lib/task.ex:416: Task.await/2
(elixir) lib/enum.ex:966: Enum.flat_map_list/2
(my_app123) lib/my_genserver.ex:260: MyApp.MyGenServer.do_work/1
(my_app123) lib/my_genserver.ex:180: MyApp.MyGenServer.handle_info/2
(stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:683: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: :tick
State: [%{var1: "fdsafdsfd", var2: "43243242"}]
A chunk of the code:
# it's called from handle_info
def do_work(some_data) do
Enum.map(some_data, fn(x) ->
Task.async(fn ->
case HTTPoison.post(.....) do
# ...........
Is “Task.async” causing the timeout? But why? Yes, it can take more than 5 seconds to complete, but why does it cause an exception which then terminates GenServer? How to fix it?
About await:
If the timeout is exceeded, await will exit; however, the task will continue to run. When the calling process exits, its exit signal will terminate the task if it is not trapping exits.
Most Liked
peerreynders
Actually exit/1 can be caught:
iex(1)> try do
...(1)> exit("foo")
...(1)> catch
...(1)> :exit, _ -> :caught
...(1)> end
:caught
however your objection is still intact:
- the timeout doesn’t have anything to do with the
Task, which just keeps running - it is theawait/2process that times out - the termination of the
await/2process emits an exit signal to all linked processes. That exit signal cannot be “caught” - the exit signal will terminate any linked processes which aren’t trapping exits and those which do trap exits will get an:EXITmessage in their mailbox.
.
-
Kernel.exit/1can only be caught inside the process that invokes it - if it is allowed to escape the process it turns into an exit signal - at which point it is too late tocatchit anywhere. -
Process.exit/2is an entirely different animal. Whileexit/1terminates the process that invokes it,exit/2sends an exit signal to the specified process usually with the intent to “tell that process to terminate”. Soexit/2cannot be caught under any circumstances (but it can be trapped - unless the reason is:kill).
.
iex(2)> try do
...(2)> Process.exit(self(),"foo")
...(2)> catch
...(2)> :exit, _ -> :caught
...(2)> end
** (EXIT from #PID<0.87.0>) "foo"
.
iex(1)> Process.flag :trap_exit, true
false
iex(2)> try do
...(2)> Process.exit(self(),"foo")
...(2)> catch
...(2)> :exit, _ -> :caught
...(2)> end
true
iex(3)> flush()
{:EXIT, #PID<0.103.0>, "foo"}
:ok
NobbZ
Task.await/1 calls exit/1 on timeout, so it will end your current process. If you do not wan’t that behaviour you will have to implement the receiving of the result and receiving of :DOWN messages on your own.
benperiton
I’m still getting to grips with Tasks myself, but I think it’s because the Task is linked to the calling process, so if it throws an error and exits, then the calling process will also die. See https://hexdocs.pm/elixir/Task.html#module-async-and-await
You could use Task.start_link/1 if you don’t need the response, I’ve been using a Task.Supervisor for mine, so that if a Task dies, it doesn’t bring the genserver down.
Add a supervisor:
supervisor(Task.Supervisor, [[name: App.MyTaskSupervisor]])
Then can use it like:
def do_work(some_data) do
Enum.map(some_data, fn(x) ->
Task.Supervisor.async_nolink(App.MyTaskSupervisor, fn ->
case HTTPoison.post(.....) do
Because I wnted to know if it was a timeout, I use yield
task = Task.Supervisor.async_nolink(
App.MyTaskSupervisor,
MyModule,
:task_function,
[args]
)
case Task.yield(task) || Task.shutdown(task) do
{:ok, result} ->
ack_message({:ok, %{channel: channel, tag: tag}})
{:error, msg} ->
ack_message({:error, %{error: msg, channel: channel, tag: tag, redelivered: redelivered}})
{:exit, reason} ->
ack_message({:error, %{error: reason, channel: channel, tag: tag, redelivered: redelivered}})
nil ->
ack_message({:error, %{error: "TIMEOUT", channel: channel, tag: tag, redelivered: redelivered}})
end
NobbZ
I’m not sure if I do understand your question correctly here. But outsourcing work from a GenServer into another process as soon as it will take longer than a couple of “cycles” is idiomatic and good practice.
This is done to keep the work the actual GenServer process has to do to a minimum so that it is able to process incomming calls and casts as fast as possible,
But as a rough concept it looks like this:
def handle_call{:do_stuff, from, status} do
spawn(fn ->
answer = do_stuff()
GenServer.reply(from, answer)
end)
{:noreply, state}
end
Doing it this way, will remove the burden to handle any communication with that process from your GenServer. Assuming that HTTPoison does have timeouts, you can even handle them inside that spawned process and reply differently from the default when a timeout happens. Also you can try and catch there. You will still run into trouble when do_stuff or something inside of it does call exit!
peerreynders
Basically ditch Task.await and simply grab the results in handle_info/2. If your tasks have a habit of abnormally exiting, trap exits and process the :EXIT messages accordingly.
defmodule Charlie do
use GenServer
defp new_task do
max_time = 5000
work =
case {(:rand.uniform max_time), (:rand.uniform max_time)} do
{work, panic} when work < panic ->
fn ->
Process.sleep work
work # return the work time as the result
end
{_, panic} ->
fn ->
Process.sleep panic
exit(:panic) # simulate non-normal exit
end
end
Task.async work
end
defp add_task(tasks, task),
do: Map.put tasks, task.pid, task
defp purge_task(tasks, pid) do
case Map.get tasks, pid, :none do
:none ->
{tasks, :none}
task ->
Process.demonitor task.ref, [:flush] # purge related :DOWN messages
{(Map.delete tasks, pid), task}
end
end
defp shutdown_tasks(tasks) do
tasks
|> Map.values()
|> Enum.each(&(Task.shutdown &1, :brutal_kill))
end
## callbacks
def handle_info(:next_task, {tasks, _ref}) do
# time to create another task
new_tasks =
cond do
(length (Map.keys tasks)) < 50 ->
add_task tasks, new_task()
true ->
tasks
end
timer_ref = Process.send_after self(), :next_task, 500
{:noreply, {new_tasks, timer_ref}}
end
def handle_info({ref, result}, state) when is_reference ref do
# handle Task completion
Process.demonitor ref, [:flush] # purge related :DOWN messages ASAP
IO.puts "Task #{inspect ref} completed with result #{result}"
{:noreply, state} # remove task when :EXIT :normal is processed
end
def handle_info({:EXIT, from, reason}, {tasks, timer_ref} = state) do
# Handle exit signal not handled by the behaviour (i.e. signal from parent process)
case purge_task tasks, from do
{_, :none} ->
IO.puts "Unknown :EXIT #{inspect reason}"
case reason do
:normal ->
{:noreply, state}
_ ->
{:stop, reason, state} # Follow protocol: unknown non-normal exit signal - time to terminate
end
{new_tasks, task} ->
case reason do
:normal ->
:ok
_ -> # The task panicked
IO.puts "Task #{inspect task.ref} panicked: #{inspect reason}"
end
{:noreply, {new_tasks, timer_ref}}
end
end
def init(_args) do
Process.flag(:trap_exit, true)
Kernel.send self(), :next_task # start spinning off tasks
{:ok, {%{}, :none}}
end
def terminate(_reason, {tasks, timer_ref}) do
IO.puts "Terminating"
cond do
is_reference timer_ref ->
Process.cancel_timer timer_ref
true ->
0 # no timer, therefore no time left
end
shutdown_tasks tasks
:ok
end
## public interface
def start_link do
GenServer.start_link(__MODULE__, [])
end
def stop(pid) do
GenServer.stop pid
end
# NOTE:
# https://hexdocs.pm/elixir/Task.html#await/2-compatibility-with-otp-behaviours
# It is not recommended to await a long-running task inside an OTP behaviour such as GenServer.
# Instead, you should match on the message coming from a task inside your
# GenServer.handle_info/2 callback.
#
end
.
$iex -S mix
iex(1)> Process.flag :trap_exit, true
false
iex(2)> {:ok,pid} = Charlie.start_link
{:ok, #PID<0.127.0>}
Task #Reference<0.0.6.653> panicked: :panic
Task #Reference<0.0.6.651> completed with result 1306
Task #Reference<0.0.6.647> panicked: :panic
Task #Reference<0.0.6.658> panicked: :panic
Task #Reference<0.0.5.605> panicked: :panic
Task #Reference<0.0.6.649> panicked: :panic
Task #Reference<0.0.6.661> completed with result 464
Task #Reference<0.0.6.668> panicked: :panic
Task #Reference<0.0.6.677> panicked: :panic
Task #Reference<0.0.6.673> panicked: :panic
Task #Reference<0.0.6.682> completed with result 957
Task #Reference<0.0.6.684> completed with result 665
Task #Reference<0.0.6.675> panicked: :panic
iex(3)> Process.exit pid, :whatever
Terminating
true
iex(4)>







