orestis

orestis

Python generators equivalent?

So Python has a nice concept of generators - functions that become iterators when called:

def step(x, s):
    while True:
        yield x
        x = x + s

This is called like so:

s = step(1, 2)
s.next() # 1
s.next() # 3
s.next() # 5

And it implements the iterator protocol so you can pass it wherever an iterator is expected.

What’s the idiomatic way in Elixir to do this? We can get partway there with:

s = Stream.unfold(1, fn(x) -> {x, x+2} end)
Enum.take(s, 3) # [1, 3, 5]

But obviously this is not stateful. To get stateful, a process would be needed… But even if we don’t maintain state, I couldn’t find an easy way to do this:

{[1, 3], s} = Stream.split(s, 2)
{[5], s} = Stream.split(s, 1)

Using Enum.split blocks, probably trying to reach the end of the Stream.

Thoughts?

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

The concept of a Generator is not something that is unique to Python. Someone has taken the time in the past to write a version (using message passing) in Erlang (which was also translated to Elixir) on RosettaCode. Interestingly, as the Wikipedia Article mentions, Haskell’s lazily evaluated functions are also generators and as such the Stream that is built-in in Elixir indeed is also a generator.

Of course, in an immutable+pure language, you cannot call the same function with the same value over and over nextVal(mygen); nextVal(mygen); nextVal(mygen); and expect different results (unless you ‘cheat’ by using message passing). Indeed, when you’re working with for instance a Random Number Generator in Haskell, you have two options:

  1. Call a function which takes a RNG, and returns a {randomNumber, newRNG} tuple. This is basically similar to what Enumerable does internally, which @benwilson512 talked about as well.
  2. Specify that you want an infinite stream of random numbers, and take as many numbers from that stream as you need. In this case, the generator is ‘consumed’; you’ve turned in an infinite stream of numbers.

In most functional languages (including Elixir), depending on the specific situation, one of these two ways is used. The third one, ‘cheating’ using message-passing, is often too much overhead for what you want to use the generator for. Exceptions of course do exist: A GenServer that returns guaranteed-unique identifiers, for instance.


I do wonder if we can implement a generator based on fixpoint combinators as well, by the way… :smiley:

orestis

orestis

Can’t we all be friends? :slight_smile:

Every new language you learn impacts your style. Some concepts tend to stick, so naturally I want to explore how they could be replicated or what the alternatives are.

NobbZ

NobbZ

Well, at least for Enum.split/2s strictness there has been a discussion recently:

This also tackles some small bits of your “state” problem.

In elixir I do usually not expect side effects to happen, so next(s) which returns 1 on the first call and 2 on the second without rebinding s would be counterintuitive for me.

If you need a central authority handing out free ids, work items, whatever, I do think that is what GenStage has been developed for.

If you really need something as your python thingy (which involves far too much magic for me, I’d expect it to loop forever) then you need to implement it yourself. Some protocol or behaviour and a handfull of macros, and you are ready to release it on hex.

NobbZ

NobbZ

Of course I do expect a stream that has no end to run forever. Stream.run/1 is documented to evaluate the complete stream.

As well as I do expect while true {} to do nothing forever, and not to do a single nothing only when I do ask for it…


After reformatting your code, I do see a little bit better what you meant. Still I’d expect that Stream to run forever and to block that process it is run in, but of course you can ask for printing “hi” and doing an otherwise useless addition if you do know the PID of the starting process.

orestis

orestis

Thanks everyone for your thoughts.

I have currently this passing test case:

  test "generators" do
    defmodule MyGen do
      use Snakeoil.Generators

      gen "counter", x do
        yield x
        counter(x+1)
      end

      gen "two", {x, y} do
        yield x + y
        two({x+1, y + 1})
      end
    end

    pid = spawn(fn -> MyGen.counter(0) end)
    assert 0 == MyGen.next(pid)
    assert 1 == MyGen.next(pid)
    assert 2 == MyGen.next(pid)

    pid = spawn(fn -> MyGen.two({2, 3}) end)
    assert 5 == MyGen.next(pid)
    assert 7 == MyGen.next(pid)
  end

This is a toy, of course - mostly an exercise in metaprogramming. I have to admit that having optional parentheses makes things like that fun to write - but I’m not sure how fun it would be if someone did use that in their own code :slight_smile: If i’m not mistaken, Elixir also favors explicit over implicit, which is good from me, coming over from Python land.

Where Next?

Popular in Questions Top

vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
Phillipp
Hey, I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New

Other popular topics Top

bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
sergio
I couldn’t find any guides that worked well with Phoenix 1.6.0 and esbuild. I hope this helps people test the waters and eases you into t...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement