polypush135

polypush135

Trying to better understand broadway

Was reading https://blog.appsignal.com/2019/12/12/how-to-use-broadway-in-your-elixir-application.html
and wanted to get a better understanding of how the incoming messages can be triggered and managed.

Right out the box with this tut, it uses a WorldTemp.CityProducer example that I have a hard time understanding. There are two cases of handle_demand, one is for when demand > length(state) the other for when this is not true.

First: What does demand here represent? It starts with a default value of 10. Where does this value come from and what does it represent? What should I read to get a betting understanding of this value?

Second: The city list function is a static list. What if I wanted to do an ecto query and get a list of rows that should be processed. Is this where that request would look up said rows?

I guess so far the producer is the most confusing part to me.

Edit: I think I figured out my disconnect. This module is a GenStage. https://hexdocs.pm/gen_stage/GenStage.html?#c:handle_demand/2 and not directly related to Broadway.
Time to go learn some GenStage I guess.

Most Liked

akoutmos

akoutmos

Author of Build a Weather Station with Elixir and Nerves

Author of the article here and good questions!

The reason for the two different handle_demand functions (one with the guard and one without) is because our GenStage producer effectively creates an infinite list of cities. That first function with the guard ensures that there are enough cities in the list to satisfy the incoming demand by adding more items to the list. The demand in this case represents what the downstream Broadway consumers are requesting from this producer.

While you can use GenStage as a producer, in production you most often would use Broadway with something like RabbitMQ or Kafka. I chose GenStage as a producer for that article mostly to keep the length of the article within reason. I have a more in depth 2-part series tutorial on my personal blog that leverages RabbitMQ, Broadway, and Ecto:


polypush135

polypush135

Thanks for the pointers, that helps.

Where Next?

Popular in Questions Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New

Other popular topics Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New

We're in Beta

About us Mission Statement