fireproofsocks

fireproofsocks

Best practice for SQS message: JSON encoded objects for the message body? Or message attributes?

This is a general question, but what are people doing for their SQS message formats? In a lot of places I’ve worked, people just JSON encoded an object and sent it as the SQS message and they never used the SQS message attributes. Even the Broadway testing functions don’t put message attributes at the forefront: the envisioned use case is that all test messages in a pipeline will share the same message attributes.

Is the JSON encoding route the “proper” way? Although I can see some advantages (mostly in pattern matching) that leverage using message attributes, it’s usually a chunk more work.

What are people’s thoughts on this?

Most Liked

gregvaughn

gregvaughn

As always with these “best” practice type of questions, the answer is “it depends.” There is no single best way of doing it. it really depends upon what your payload is. If, for example, you were sending pdf files or images, then json is a horrible choice. If you’re sending some structured data payload, then json is a fine choice (though there’s cases where csv or xml or protobuf or other might be the best for that situation).

My personal use involves integration with a 3rd party company, so they dictated the message format. They have a message attribute (I asked the Broadway team to expose those) that is a cryptographic IV (initialization vector) that I have to read and combine with a shared secret to decrypt the json payload.

I would not use message attributes for domain knowledge purposes though. I see them more as infrastructural metadata. However, I tend to be cautious about buying too deeply into one vendor’s implementation. The less you depend upon their special features the easier it will be for you to migrate to some other queueing service. Tradeoffs apply and your situation may not match mine though.

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

Other popular topics Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

We're in Beta

About us Mission Statement