geofflane

geofflane

Getting Message and Errors from Broadway

Broadway catches exceptions and handles other errors by logging the error and “failing” the message. Is there a way to get a callback called to do something with that error as well? I would need the original message and the error (message, exception, etc).

My use case:

We’re collecting a bunch of data from a bunch of people over time. As the data gets collected it gets put on a queue and consumed by another app to coalesce that data into a bunch of “facts” (for lack of a better term) that we can query and report on. Any errors get sent back to the collection application by publishing a message to another queue with the original payload and the error reason. The sending application listens for these messages and collects them into a report for “quality control” to rectify.

Rabbit dead-letters don’t work because a nack doesn’t allow you change the message as far as I know. So we need to actually push another message. We built all of this ourselves before Broadway, but not quite as sophisticated as Broadway (e.g. termination process). I was looking to convert to Broadway and couldn’t see a way to do that. Any pointers?

Thanks.

Marked As Solved

msaraiva

msaraiva

Broadway Core Team

Hi @geofflane!

We’ve been discussing the possibility of adding a callback (or something similar) for that purpose. For now, one thing you could try would be wrapping the code that might fail inside handle_message/3 or handle_batch/4 in a try/catch block, do your stuff and then re-raise the exception (or set the message as failed using Message.failed/2).

Not optimal, I confess. But it might do the trick until we have a built-in solution.

Cheers.

-marlus

Where Next?

Popular in Questions Top

pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New

We're in Beta

About us Mission Statement