laurazard

laurazard

Lazy cartesian product

Hi all,

This is my first post in this forum, and although I did look, please forgive me if it’s a duplicate (or something very simple, I’m quite new at elixir).

Onto the question: I have two streams which I create with File.Stream!. I want to create the Cartesian product of the two streams, but only until I have 10000 items (the full Cartesian product will be over double that). What I would like to really do is do this lazily, but I can’t seem to come up with an easy way to do that.

Currently, what I’m doing is this

for a <- File.stream!("a.txt"),
  b <- File.stream!("b.txt"),
  do: (
    a = String.trim(a)
    b = String.trim(b)
    {a, b}
  )

However, this seems to work eagerly. Of note is that I also need to apply some transformations to the elements (as exemplified by the String.trim).

Any idea as to how I could accomplish this in a lazy way, maybe with better use of Streams, or zipping them, or some such?

Thank you so much!

Marked As Solved

josevalim

josevalim

Creator of Elixir

Welcome @laurazard!

A regular for-comprehension is equivalent to a chain of flat_map with a map at the end. So this comprehension:

iex(2)> for x <- 1..10, y <- 1..10, do: x * y
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 3, 6, 9, 12,
 15, 18, 21, 24, 27, 30, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 5, 10, 15, 20,
 25, 30, 35, 40, 45, 50, ...]

Is equivalent to:

iex(3)> Enum.flat_map(1..10, fn x -> Enum.map(1..10, fn y -> x * y end) end)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 3, 6, 9, 12,
 15, 18, 21, 24, 27, 30, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 5, 10, 15, 20,
 25, 30, 35, 40, 45, 50, ...]

Which means you can make it a stream with:

iex(3)> Stream.flat_map(1..10, fn x -> Stream.map(1..10, fn y -> x * y end) end)

You can use a similar approach in your code.:slight_smile:

13
Post #2

Also Liked

laurazard

laurazard

Hi @josevalim,

Thank you for the comprehensive answer! This is exactly what I need :grinning_face_with_smiling_eyes:

Where Next?

Popular in Questions Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New

Other popular topics Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

We're in Beta

About us Mission Statement