igalic

igalic

How to create a custom StreamData Generator

Hi folks,

I’m trying to transform our tests for AuthToken (a wrapper around JOSE) to ExUnitProperties. But I’m kinda failing at the most basic task of transforming a simple function into a generator. What I’ve done is this:

  defp stream_authtoken_keys(_seed, _range) do
    Stream.repeatedly(&AuthToken.generate_key/0)
  end

  defp gen_authtoken_key() do
    %StreamData{generator: &stream_authtoken_keys/2}
  end

However, when trying to use it…

  describe "keys" do
    property "generate_key/0 returns a valid AES128 key" do
      check all authtoken_key <- gen_authtoken_key()  do
       {:ok, key} = authtoken_key
        assert byte_size(key) == 16
      end
    end
  end

I get the following error:

  1) property keys generate_key/0 returns a valid AES128 key (AuthTokenTest)
     test/authtoken_test.exs:23
     ** (MatchError) no match of right hand side value: #Function<51.91433161/2 in Stream.repeatedly/1>
     code: check all authtoken_key <- gen_authtoken_key()  do
     stacktrace:
       (stream_data) lib/stream_data.ex:203: StreamData.bind_filter/5
       (stream_data) lib/stream_data.ex:346: anonymous fn/5 in StreamData.bind_filter/3
       (stream_data) lib/stream_data.ex:203: StreamData.check_all/6
       test/authtoken_test.exs:24: (test)

Marked As Solved

igalic

igalic

After posting this in elixir (on irc & discord) I got a very decent reply that works out pretty fine, so I’m gonna close this topic for now with it!

First off, trying to generate %StreamData{} myself, is kinda bad form, since it’s basically a private struct…
We looked for alternatives, and found StreamData.constant/0. However, on its own, it always generates the same value.

Which makes a certain amount of sense when you think about how it’s called:

iex(1)> x = StreamData.constant(AuthToken.generate_key) # this is now evaluated, we're done here
%StreamData{generator: #Function<16.32994346/2 in StreamData.constant/1>}
iex(2)> x |> Enum.take(2)                              
[
  ok: <<109, 179, 255, 84, 16, 160, 239, 132, 198, 157, 218, 254, 13, 160, 196, 137>>,
  ok: <<109, 179, 255, 84, 16, 160, 239, 132, 198, 157, 218, 254, 13, 160, 196, 137>>
]
iex(3)>

The clue came from @michalmuskala:

You usually construct the opaque type one way or the other, you rarely construct it directly. For example the MapSet struct is opaque, but you can generate it by doing something like map(list(), &MapSet.new/1)

and the code we now have from @ericmj

  defp gen_authtoken_key() do
    StreamData.map(StreamData.list_of(StreamData.constant(:unused_tick)), fn _ ->
      AuthToken.generate_key()
    end)
  end

what we’re doing here is using StreamData.constant/1 as a sort of clock generator, that drives the call of AuthToken.generate_key/0… and it’s nice start, but we’re needlessly generating a list_of which we then map
So let’s update this with all responses below!

  defp gen_authtoken_key() do
    StreamData.unshrinkable(
      StreamData.bind(StreamData.constant(:unused), fn _ ->
        StreamData.constant(AuthToken.generate_key())
      end)
    )
  end

This is much cleaner! Since we use bind/2, which is essentially made for this… better see what’s going on, we can use pipes to extract the flow:

  defp gen_authtoken_key() do
    gen_ticker = fn _ ->
      StreamData.constant(AuthToken.generate_key())
    end

    StreamData.constant(:unused)
    |> StreamData.bind(gen_ticker)
    |> StreamData.unshrinkable()
  end

We feed a stream of :unused into gen_ticker lambda, just so it will be constantly called. Since our call to AuthToken.generate_key/0 is wrapped in said lambda, it will be executed on every call, rather than on ge_ticker’s declaration. Finally, we pass the so generated %StreamData{} into unshrinkable/0 (because it is) before returning it.

Also Liked

whatyouhide

whatyouhide

Elixir Core Team

Seeing this now after https://github.com/whatyouhide/stream_data/pull/104 has been opened. Just wanted to mention here too that I also think that a generic way to convert streams to generators would be nice but there is a big limitation: generators are stateless while streams aren’t. Streams are enumerated in order and there is no way for generators to know at which index in the stream it’s at.

@ericmj also your solution can be even simpler as bind + constant is just map:

def repeatedly(fun) do
  StreamData.map(StreamData.constant(nil), fn _ -> fun.() end)
end
ericmj

ericmj

Elixir Core Team

I found a (maybe) cleaner way of doing it that doesn’t involve generating a list:

defp gen_authtoken_key() do
  StreamData.bind(StreamData.constant(:unused), fn _ ->
    StreamData.constant(AuthToken.generate_key())
  end)
end

I haven’t tested this so I can’t verify that it works.

EDIT: @whatyouhide has a more concise solution below: How to create a custom StreamData Generator

LostKobrakai

LostKobrakai

It might also be good to make the generator unshrinkable, because shrinking doesn’t seem to make sense in that context.

igalic

igalic

I really couldn’t figure this out from the documentation…
maybe we should add something to the docs that helps people figure out how to create custom generators

@michalmuskala suggested to have StreamData.repeatedly/1. I was thinking maybe a better way would be to have a way of converting a Stream to a StreamData.

LostKobrakai

LostKobrakai

I found it quite easy to build more complex generators out of the basic generators already in the streamdata module, but I’ve to say that your usecase really doesn’t seem very well supported by the current functions.

Where Next?

Popular in Questions Top

gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement