stefpankov

stefpankov

Loggin when Task fails due to raised exception

Hey there,

I’m new to Elixir and OTP and coming from an OO background, I’m still wrapping my head around Processes, Supervisors, Tasks etc.

I’m currently working on a little program that has to go through a bunch of files, read from them, parse the data and do something with it.

I’m trying to speed up this process by using Tasks and I’ve setup a TaskSupervisor which I then use to dynamically start Tasks that parse the data from the file.

It looks something like this

def do_the_thing(dir) do
    for file <- File.ls!(dir) do
      file_name = "#{dir}/#{file}"

      File.read!(file_name)
      |> Parser.parse!()
      |> Enum.each(fn parsed_data ->
        Task.Supervisor.start_child(MyApp.TaskSupervisor, MyTask, :run, [
          parsed_data
        ])
      end)
    end
end

Now this works fine but when an error is raised and MyTask fails, I want to somehow trap the exception and log it in a file with additional metadata so I can see where it failed.

I had a try rescue block inside the MyTask.run function but I have a feeling that’s unnecessary since I’m fine with the Task failing, the ever vigilant Supervisor handles that for me just fine.

I guess my question is, what’s considered best practices when I want to catch exit reasons and log if they interest me. I tried to setup something with Process.monitor but couldn’t really figure out how to match exceptions and not :normal exits.

Marked As Solved

david_ex

david_ex

I’m not sure your approach is the best for what you want to achieve (although I’m no expert): although you’ll get the pid of the child process from start_child (and could therefore monitor it), it the task fails and restarts you won’t be able to “re-monitor” it.

If handling the first failure is good enough for you, the basic steps would be

{:ok, pid} = Task.Supervisor.start_child(MyApp.TaskSupervisor, MyTask, :run, [parsed_data])
ref = Process.monitor(pid)
receive do
  {:DOWN, ^ref, :process, _object, :normal} -> nil # success => do nothing
  {:DOWN, ^ref, :process, _object, reason} -> ... # Log that task for `file_name` failed due to `reason`
end

That said, and depending on what you want to do, you can also use a GenServer and track the state of ongoing tasks. Then, you can use one of the Task.Supervisor.async_nolink to trigger a task and store the ref:

# in e.g. a handle_call
      File.read!(file_name)
      |> Parser.parse!()
      |> Enum.map(fn parsed_data ->
        %Task{ref: ref} = Task.Supervisor.async_nolink(MyApp.TaskSupervisor, MyTask, :run, [
          parsed_data
        ])
        {ref, parsed_data}
      end)
      |> Enum.into(state)

Where state is the GenServer state. To handle task results, use handle_info:

def handle_info({task_ref, task_result}, state) do
  # task was successfully completed, with `task_result`

  # we don't care about the coming `:DOWN` message for this task (which will have reason `:normal`
  # you could also just have a `handle_info` for `:DOWN` with `:normal` reason and do nothing there
  Process.demonitor(task_ref, :flush)
  {:noreply, Map.delete(state, task_ref)}
end

def handle_info({:DOWN, ref, :process, _object, reason}, state) do
  # log the fact that the task processing `Map.get(state, ref)` failed
  {:noreply, state}
end

Also Liked

stefpankov

stefpankov

Thanks for your answer!

Regarding re-monitoring tasks, for now, they’re not contacting any external service so the only fail reason is missing data that is crucial, that’s why I want to get to the exception that caused them to fail and format it nicely with more information so I know exactly what caused them to fail. That means they won’t be restarted after failure and that works just fine, the only thing I need to setup is that monitoring logic.

I like the GenServer approach and read a suggestion about it somewhere else but with no example of how to do it. This is extremely helpful, I’ll try it and post progress here.

lud

lud

Hello,

Not sure if it would fit with your supervision tree, but sometimes I just use this pattern to handle failing tasks:

t1 =
  Task.async(fn ->
    Process.flag(:trap_exit, true)
    parent = self()

    t2 =
      Task.async(fn ->
        raise "failed"
        # do some heavy stuff
        result = :data
        send(parent, result)
      end)

    receive do
      {:EXIT, _, e} ->
        {:error, e}

      result ->
        Task.await(t2)
        {:ok, result}
    end
  end)

t1
|> Task.await()
|> IO.inspect(pretty: true)

Where Next?

Popular in Questions Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

peerreynders
Manning 2016 Halloween weekend sale via Deal of the Day Friday, October 28 - Half off all MEAPs - code WM102816LT Saturday, October 29 ...
326 29600 154
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New

We're in Beta

About us Mission Statement