francesco.pessina
Kafka Ex :timeout error on connector
Hello everyone 
I have a strange problem with Kafka Ex (GitHub - kafkaex/kafka_ex: Kafka client library for Elixir). While running locally and connecting to a local Kafka instance, all is fine.
When deploying the application on a Kubernetes AWS cluster, I experience a weird error. The Kafka broker is an AWS managed one, and I’m sure it is reachable because other microservices (Java) can connect.
This is the error I’m experiencing:
07:05:21.853 [error] GenServer #PID<0.4672.0> terminating │
│ ** (MatchError) no match of right hand side value: [async_commit: nil] │
│ (kafka_ex 0.12.1) lib/kafka_ex/gen_consumer.ex:774: KafkaEx.GenConsumer.handle_message_set/2 │
│ (kafka_ex 0.12.1) lib/kafka_ex/gen_consumer.ex:668: KafkaEx.GenConsumer.handle_info/2 │
│ (stdlib 3.12.1) gen_server.erl:637: :gen_server.try_dispatch/4 │
│ (stdlib 3.12.1) gen_server.erl:711: :gen_server.handle_msg/6 │
│ (stdlib 3.12.1) proc_lib.erl:249: :proc_lib.init_p_do_apply/3 │
│ Last message: :timeout
and this is my broker and consumer group configuration:
# Kafka brokers configuration
config :kafka_ex,
brokers: "csv list of the brokers"
config :my_app, :children, [
%{
id: "consumer",
start:
{KafkaEx.ConsumerGroup, :start_link,
[
MyApp.Kafka.Consumer,
System.get_env("KAFKA_CONSUMER_GROUP"),
[<topic1>,<topic2>],
[
# setting for the ConsumerGroup
heartbeat_interval: 1_000,
# this setting will be forwarded to the GenConsumer
commit_interval: 1_000,
auto_offset_reset: :latest
]
]}
}
]
The application is running in a Docker image built from elixir.1.11-alpine with OTP 23.
I don’t understand what’s happening 
First Post!
francesco.pessina
Nevermind, the problem may be some conflicting process… Rollbacking to a previous commit solved the issue 
Popular in Questions
What is the idiomatic way of matching for not nil in Elixir?
E.g.,
First way:
defp halt_if_not_signed_in(conn, signed_in_account) when...
New
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it.
I’m very interested in Elixir,...
New
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database.
Dep...
New
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
Hi,
I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
Hi guys, i’m new in the Elixir world, and i have to say, that i love it!
i’m having some problem to understand anonymous functions with ...
New
Hi all
I want to have a unix time, from the current time plus 1 hour.
DateTime.now + 1 hour
How to get it in elixir?
Thanks
New
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
Other popular topics
What is the idiomatic way of matching for not nil in Elixir?
E.g.,
First way:
defp halt_if_not_signed_in(conn, signed_in_account) when...
New
Hi there,
I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
New
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
I have an umbrella app.
Some of the apps inside depend on other apps in the umbrella, unsurprisingly.
I'm writing a test for one of t...
New
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service.
Currently when I de...
New
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New







