minhajuddin
Is there a way to split a stream into two outputs? similar to `tee` on linux
Like the title says, I want to pass a stream to two outputs.
I am looking for something like below:
function_which_returns_stream
|> Enum.tee( mapper_for_func1 |> postgrex_stream1, mapper_for_func2 |> postgrex_stream2)
Is there a way to achieve this in Elixir?
Most Liked
wmnnd
You could simply use Enum.map/2 with an anonymous function that calls your other two functions like this:
fn1 = fn (x) -> IO.puts("1: #{x}") end
fn2 = fn (x) -> IO.puts("2: #{x}") end
1..3
|> Stream.map(&(&1))
|> Enum.map(stream, fn (x) ->
fn1.(x)
fn2.(x)
end)
3
pma
If you’re streaming to external systems (postgres), you can also use GenStage and a Broadcast Dispatcher? This would give you concurrency and back-pressure.
3
benwilson512
Author of Craft GraphQL APIs in Elixir with Absinthe
Can you elaborate a bit on your actual scenario?
3
Qqwy
TypeCheck Core Team
What about something like this?
defmodule Test do
def tee(input_stream, functions) do
Stream.map(functions, fn fun ->
run_async_pipeline(input_stream, fun)
end)
|> Stream.concat
|> Stream.run
end
def run_async_pipeline(stream, fun) do
stream
|> Task.async_stream(fun)
|> Stream.map(fn {:ok, item} -> item end) # error handling could be added here.
|> put_in_appropriate_postgres_stream
end
end
tee([1,2,3,4], [&(&1*&1), &(&1+1), &(&1-1)])
3
OvermindDL1
You should give it a try, then figure out how you would like a more simple interface, then post that example here and/or make a library to handle it. ![]()
3
Popular in Questions
What is the proper way to load a module from a file in to IEX?
In the python world, doing something like this pretty standard:
from ....
New
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]
15:22:35.803 [error] gen_event {lager_file_backend...
New
can someone please explain to me how Enum.reduce works with maps
New
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
Hi all,
Trying to get some more clarity over utc_datetime and naive_datetime for Ecto:
https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database.
Dep...
New
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this
"1000"
What is the ...
New
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
Other popular topics
Update:
How to use the Blogs & Podcasts section
You can post links to your blog posts or podcasts either in one of the Official Blog...
New
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
Credo is smart enough to check for (something like) this:
assert length(the_list) == 0
with this response:
Checking if an enum is empt...
New
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
Hello all!
I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
Hi there,
I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!)
This post collects co...
New
Hey all,
I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
by Lance Halvorsen
Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
New
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New







