henrikc

henrikc

Use Broadway with ElasticMQ in docker

This is my first attempt at using Broadway, SQS and I am very new to Elixir, so there is hopefully a simple answer. I want to use broadway to consume from a queue in ElasticMQ so that I do not have to use AWS during development. My attempt at the configuration for broadway is,

name: __MODULE__,
producer: [
  module:
    {BroadwaySQS.Producer,
      queue_url: "http://localhost:9324/queue/service-queue",
      config: [
        access_key_id: "x",
        secret_access_key: "x",
        region: "elasticmq"
      ]}
],
processors: [
  default: []
],
batchers: [
  default: [
    batch_size: 10,
    batch_timeout: 2000
  ]
]

The docker-compose to set-up for ElasticMQ looks like,

 sqs:
  image: "softwaremill/elasticmq"
  ports:
   - "9324:9324"
   - "9325:9325"
  volumes:
    - ./elasticmq.conf:/opt/elasticmq.conf

and the elastic.conf file is,

include classpath("application.conf")

queues {
   service-queue {
     defaultVisibilityTimeout = 10 seconds
     delay = 5 seconds
     receiveMessageWait = 0 seconds
   }
}

The problem is that no events are received and after a few seconds I get the error message [error] Unable to fetch events from AWS. Reason: :nxdomain. How should I configure Broadway to make it work with ElasticMQ? Or is that not possible?

I should perhaps note that I tried out the url(since the error indicates a non-existing domain) to the queue using Python and boto3, as described on github, and that works for both receiving and sending messages. I suspect that there is something I missed in the configuration for the producer, any input would be awesome.

Most Liked

henrikc

henrikc

Turns out It was a problem with the producer config. For the next person that might run in to the same error this was the correct config given the settings of elasticmq above:

BroadwaySQS.Producer,
queue_url: "http://localhost:9324/queue/service-queue",
config: [
   access_key_id: "x",
   secret_access_key: "x",
   host: "localhost",
   port: "9324",
   scheme: "http://",
   region: "elasticmq"
]

You need to specify port,scheme and host in the config as well. It was the same as this issue for connecting to localstack with ex_aws.

Where Next?

Popular in Questions Top

lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1140 51847 244
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement