1stSolo
Can Brod get/set topic retention?
So far I hadn’t been able to figure out how to get or set topic retention values for kafka topics via brod, using versions up to 3.16.3. I can set it when creating a topic by passing name-value pairs when creating topic.
config = [
%{name: "retention.ms", value: "-1"},
%{name: "retention.bytes", value: "-1"}
]
Is this even possible via Brod, or one must install Java and kafka client libraries and invoke kafka-configs.sh or kafka_topics.sh to do it? It seems like a huge limitation. For example, I can’t drop and re-create a topic with same retention because I don’t have all of the metadata.
However, I’m looking at https://hexdocs.pm/brod/3.16.3/brod_client.html#get_metadata/2 and it should return
struct()
struct() = #{field_name() => field_value()} | [{field_name(), field_value()}]
but I’m getting a different struct…
I must be missing something.
Marked As Solved
1stSolo
I solved it using kafka_protocol library, although it was quite a brain tease to figure out how to call Erlang from Elixir, by wrapping :kpro_req_lib.describe_configs, :kpro_brokers.with_connection, :brod_utils.request_sync and :kpro_req_lib.alter_configs APIs. It was very helpful to go through the source code for the unit tests included with kafka_protocol library.
Also Liked
lud
So it does indeed return {ok, #{field_name() => field_value()}} but yeah, sorry, I don’t know another way to get that info. You may look into the kafka_protocol application (kpro modules and friends) but you may have to encode and decode the request yourself.







