Fl4m3Ph03n1x
Is it possible to have lazy evaluation on list comprehensions?
Background
I have some code that uses list comprehensions in elixir and performs some long operations. I need this code to be lazily evaluated instead of running immediately (eager evaluation).
Code
Imagine I have a list comprehension like the following:
for a <- long_operation(),
b <- longer_operation() do
a + b
end
Now, here a and b will be eagerly evaluated, so as to return a + b. However, both long_operation() and longer_operation() take a very long time to run (as the name implies) and I will only absolutely run them if I have to.
I want to come up with a solution that allows for my code to be lazily evaluated. Say, for example, to have a data structure or some other construct where I can then call run() to actually do the hard work.
Research
My first idea was to just put everything inside of anonymous functions:
for a <- fn -> long_operation() end,
b <- fn -> longer_operation() end do
a + b
end
This worked as well as you would expect, i.e., it didn’t. The main reason being: “You cannot sum two functions”.
And it makes sense.
My next option is then to have a data-structure hold the values of these computations, and have the list comprehension return said data-structure.
I would then call run() or something similar, and then the operation would be executed.
You can probably think of this as the IO construct in languages like Scala or Haskell (pardon for the poor comparison).
Other people have suggested the use of GenServers to achieve this, but this avenue does not sit well with me for two main reasons:
- I am adding a runtime dependency to a code that should not even know
GenServers exist in the first place. - Also I fail to see how this would help in any way.
Problem
The issue here is that I believe this will force me to implement a mini AST with the instructions of what needs to be executed when I call run(). Not only do I lack the knowledge to do this, It also sounds overkill at first glance.
Surely, Elixir has a way to have expressions being lazily evaluated that I am not aware of.
So this brings me to the question:
- What mechanisms does Elixir use to have lazy evaluation?
- Are there any data-structures/libraries out there that do this? (lazy evaluation)
Please let me know!
Most Liked
LostKobrakai
I don’t want to judge too hard here, but this – like a few prev. topics of yours – seems to come from a place of shoehorning features available in other languages into elixir. Doing things lazily is totally possible, but I don’t see why there would be the need to do so in a for comprehention.
What’s the problem with this?
a = fn -> long_operation() end
b = fn -> longer_operation() end
a.() + b.()
Anonymous functions and Stream, where stream is specifically for enumerables (so not single values) and it also often uses anonymous functions.
msimonborg
FWIW, I think the difference is in how one asks the question:
“How do you implement this feature/concept from X language in Elixir”
vs.
“In X language you can solve this problem with Y feature/concept. How do you solve this problem in Elixir?”
The first question is centered on how to use concepts and solutions in a language that maybe doesn’t natively or easily support it. The second question centers the problem itself and leads to answers that go directly to idiomatic solutions to common problems. The second question will still produce answers about how those original concepts can be used in Elixir if they exist. ![]()
Fl4m3Ph03n1x
I am OK with that, in previous topics I did create some datatypes that implement the Enumerable protocol to a good level of success. It’s mostly a matter how getting the reduce part of the protocol to actually reduce what I want, which is easier said than done.
I am not sure how to reply to this.
On one hand, you are not wrong. I am absolutely exploring concepts from places that are alien to Elixir and playing around, testing the waters and seeing what I can use and what I cannot.
On the other hand, I don’t see this as being bad. I see this as being positive. I am also of the opinion that I am not the only person doing these kind of experiments.
When I first started creating a desktop application for Elixir many people suggested I moved away and that “this was not the purpose of Elixir”. And today we have things like elixir-desktop.
You may argue, “Yes, but you would have to be delusional to compare you silly questions to a project like elixir-desktop”, which I would agree, however the point I am trying to make here is that many of the new tools and functions elixir has now, come from alien places. Some of them stick around. Others don’t. I make no claim that anything I do will stick to anyone ![]()
But if it helps clear the air, then please be kind enough to consider this an academic exercise ![]()
This is quite interesting. Do you know where I could contact the developers and ask this question? Perhaps some mailing list?
krstfk
Against my better judgement, I’m gonna have a few remarks.
- for comprehension, in scala is basically syntactic sugar for chains of foreach/map/flatMap/withFilter, applied to whatever object that’s on the right of the arrow. If you’re trying to learn, you must desugar those calls.
- Scala is absolutely not lazy by default.
- Scala is impure. There is no IO monad in the standard library. Given the fact that Martin Odersky (creator of scala), has stated many times that he doesn’t believe monads are the right model for effects, an IO monad in the standard library won’t happen.
- I guess the IO type you’ve shown comes from the
cats-effectlibrary. Its goal is to help with concurrency, not effects. (basically, it’s really close to what lwt would offer in ocaml, in both case you get few guarantees that side-effects will be push outside)
All that being said, it’s hard to say, what your goal is. And since you brought up being pedantic,
{:ok, result} | {:error, details} does not a monad make, you still need return and bind (or in cats scala speak, pure and flatMap).
If learning fp is your goal, I’d recommend starting with Stanford CS 3110, then if you want to know more about lazy by default and pure, check out Haskell from first principles. Then if you want to understand a bit more about category theory in programming watch/read Bartosz Milewski’s work ( eg : videos on ct , or Blog posts, and Fong-Spivak-Milewski), after that you’ll pretty much end up in the realm of research papers.
If you want to apply any of the above to elixir/erlang, you’ll need to have a deep understanding of that + the primitives available in erlang/elixir.
Finally, when you use abstractions, you should consider what they actually bring to the table (and, push side effects outside doesn’t really work).
LostKobrakai
No worries. I do like the exploration, but I personally favor migrating usecases to ideomatic code within another language instead of shoehorning concepts over. That’s a bit difficult to do with just an abstract example though, as there’s not much to make tradeoffs by from “I’d like to have lazy language features”.
As far as I understand lazy evaluation is baked into both scala and haskell upfront. Elixir on the other hand doesn’t do that. Lazyness is introduced for Stream due to practical concerns – enumerating data, which is to big to fit onto the computer or even is infinite – and not as a “language level” concept. Anonymous functions can also enable lazy evaluation, but it’s also clearly differenciated at the syntax level and clearly not a lazily, single evaluated, variable.
You could consider a macro, which embeds lazy evaluation onto elixir. That would be comparable to e.g. what Nx did with defn, which runs elixir syntax with completely different underlying semantics, but that’s quite a bit of work, especially given it needs to include both the code for assigning the lazy variable as well as any code reading it.
You might be able to use two separate macros between marking something as lazy and reading the value, but unless you want to evaluate the code multiple times, you need to also think about retaining/caching the evaluated values somehow.







