darnahsan
Broadway Kafka only one 1 node consuming messages
I have a phoenix app running in 2 containers both connected to redpanda(zookeeper less kafka) but I only see 1 node consuming messages from the topic, topic has only 1 partition. Below is my setup code. If I understand correctly both should consume messages as they are part of the same (consumer) group
@redpanda_port 9092
@redpanda_host "redpanda"
@redpanda_nodes [{@redpanda_host, @redpanda_port}]
@redpanda_group_id "stream_00"
@redpanda_topic "stream"
@producer_concurrency 1
@processors_concurrency 10
def start_link(_opts) do
Broadway.start_link(__MODULE__,
name: __MODULE__,
producer: [
module:
{BroadwayKafka.Producer,
[
hosts: @redpanda_nodes,
group_id: @redpanda_group_id,
topics: [@redpanda_topic],
client_id_prefix: nodename()
]},
concurrency: @producer_concurrency
],
processors: [
default: [
concurrency: @processors_concurrency
]
]
)
end
Marked As Solved
darnahsan
seems its a kafka limitation when there is 1 partition only 1 consumer is assigned
1
Popular in Questions
After calling mix ecto.create I get this error:
17:00:32.162 [error] GenServer #PID<0.412.0> terminating
** (Postgrex.Error) FATAL...
New
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]
15:22:35.803 [error] gen_event {lager_file_backend...
New
can someone please explain to me how Enum.reduce works with maps
New
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
Hello, I have map which I want to convert it to string like this:
the map:
%{last_name: "tavakkoli", name: "shahryar"}
the string I ne...
New
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: )
Hello all, this is ...
New
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
I have a super simple question about elixir - how would I take a file like this
foo bar baz
and output a new file that enumerates th...
New
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
Other popular topics
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
Hello!
Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
Hi there,
I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine)
This is a plugin that adds support for Elixir to JetBrains IntelliJ...
New
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
I have a User schema with a :from_id field set to type :string:
defmodule TweetBot.Repo.Migrations.CreateUsers do
use Ecto.Migration
...
New
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this:
...
New







