darnahsan
Broadway Kafka only one 1 node consuming messages
I have a phoenix app running in 2 containers both connected to redpanda(zookeeper less kafka) but I only see 1 node consuming messages from the topic, topic has only 1 partition. Below is my setup code. If I understand correctly both should consume messages as they are part of the same (consumer) group
@redpanda_port 9092
@redpanda_host "redpanda"
@redpanda_nodes [{@redpanda_host, @redpanda_port}]
@redpanda_group_id "stream_00"
@redpanda_topic "stream"
@producer_concurrency 1
@processors_concurrency 10
def start_link(_opts) do
Broadway.start_link(__MODULE__,
name: __MODULE__,
producer: [
module:
{BroadwayKafka.Producer,
[
hosts: @redpanda_nodes,
group_id: @redpanda_group_id,
topics: [@redpanda_topic],
client_id_prefix: nodename()
]},
concurrency: @producer_concurrency
],
processors: [
default: [
concurrency: @processors_concurrency
]
]
)
end
Marked As Solved
darnahsan
seems its a kafka limitation when there is 1 partition only 1 consumer is assigned
1
Popular in Questions
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this:
...
New
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database.
Dep...
New
I would like to know what is the best IDE for elixir development?
New
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
Hey,
I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
Hey,
Just curious what are the main benefits of Elixir compared to Clojure?
When is Elixir more useful than Clojure and vice versa?
Th...
New
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size?
Thanks
New
Hi,
I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir
iex(2)...
New
Other popular topics
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this
"1000"
What is the ...
New
The Elixir Typespec docs show the following syntax for keyword lists in typespecs:
# ...
| [key: type] # keyword lis...
New
I have a User schema with a :from_id field set to type :string:
defmodule TweetBot.Repo.Migrations.CreateUsers do
use Ecto.Migration
...
New
Hello, I have map which I want to convert it to string like this:
the map:
%{last_name: "tavakkoli", name: "shahryar"}
the string I ne...
New
Hi guys, i’m new in the Elixir world, and i have to say, that i love it!
i’m having some problem to understand anonymous functions with ...
New
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
I am trying to run a deploy with docker and I successfully runned with this command:
docker build -t romenigld/blog-prod .
but when I t...
New
Hello guys,
I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
Hi everyone!
I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New







