1stSolo

1stSolo

Can Brod get/set topic retention?

So far I hadn’t been able to figure out how to get or set topic retention values for kafka topics via brod, using versions up to 3.16.3. I can set it when creating a topic by passing name-value pairs when creating topic.

        config = [
          %{name: "retention.ms", value: "-1"},
          %{name: "retention.bytes", value: "-1"}
        ]

Is this even possible via Brod, or one must install Java and kafka client libraries and invoke kafka-configs.sh or kafka_topics.sh to do it? It seems like a huge limitation. For example, I can’t drop and re-create a topic with same retention because I don’t have all of the metadata.

However, I’m looking at https://hexdocs.pm/brod/3.16.3/brod_client.html#get_metadata/2 and it should return

struct()

struct() = #{field_name() => field_value()} | [{field_name(), field_value()}]

but I’m getting a different struct…

I must be missing something.

Marked As Solved

1stSolo

1stSolo

I solved it using kafka_protocol library, although it was quite a brain tease to figure out how to call Erlang from Elixir, by wrapping :kpro_req_lib.describe_configs, :kpro_brokers.with_connection, :brod_utils.request_sync and :kpro_req_lib.alter_configs APIs. It was very helpful to go through the source code for the unit tests included with kafka_protocol library.

Also Liked

lud

lud

So it does indeed return {ok, #{field_name() => field_value()}} but yeah, sorry, I don’t know another way to get that info. You may look into the kafka_protocol application (kpro modules and friends) but you may have to encode and decode the request yourself.

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New

We're in Beta

About us Mission Statement