IanVaughan
Consuming Kafka messages from an offset with KafkaEx
I have a working Elixir app consuming messages from my Kafka broker, it was really easy thanks to the excellent KafkaEx package.
The question I have revolves around reading from different offsets on a partition.
My GenConsumer is implemented like here https://hexdocs.pm/kafka_ex/KafkaEx.html#module-consumer-groups always reads the latest unread from the last committed offset, which is fine.
- While its reading the messages, it would be nice to know the offset position it is at, for performance monitoring, but the state is nil as its up to us to set.
- If I wanted to read from a given offset, I guess the
KafkaEx.GenConsumeris no good for this? I would have to implement my ownGenServer/GenStagewithKafkaEx.fetch/KafkaEx.streamin?
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Courses that'll move you from confusion to "Aha, now I get it!"