tombh

tombh

Weight-based random sampling

I’m new to Elixir, and relatively new to functional programming. I’ve just written this little algorithm to take a weighted sample from a list of tuples;

# items = [apples: 10, oranges: 30, bananas: 60]
defp weight_based_choice(items) do
    max = sum_weights(items)

    state = %{
      current: 0,
      choice: nil,
      random_value: Enum.random(1..max)
    }

    Enum.reduce(items, state, fn item, state ->
      weight = elem(item, 1)
      state = %{state | current: state[:current] + weight}

      choice =
        if(state[:random_value] <= state[:current]) do
          elem(item, 0)
        else
          nil
        end

      if state[:choice] == nil && choice != nil do
        %{state | choice: choice}
      else
        state
      end
    end)[:choice]
  end

  defp sum_weights(items) do
    Enum.reduce(items, 0, fn item, acc ->
      weight = elem(item, 1)
      acc + weight
    end)
  end

Are there more idiomatic ways to do this?

Most Liked

gregvaughn

gregvaughn

If I were to really use this in production, I’d likely wrap it in its own struct module like this:

defmodule WeightedList do
  defstruct [:acc_list, :max]

  def new(kw_list) do
  # Might want to verify it is sorted lowest to highest weight
  acc_list = Enum.scan(kw_list, fn {k, w}, {_, w_sum} -> {k, w + w_sum} end)
  {_, max} = List.last(acc_list)
  %__MODULE__{acc_list: acc_list, max: max}
  end

  def sample(%__MODULE__{acc_list: acc_list, max: max}) do
    rand_value = Enum.random(1..max)
    Enum.find_value(acc_list, fn
      {k, w} when rand_value <= w -> k
      _ -> false
    end)
  end
end

Which could then be used in calling code like

wl = WeightedList.new(apples: 10, oranges: 30, bananas: 60)
# now let's get 10 random samples
samples = for _ <- 1..10, do: WeightedList.sample(wl)

This represents a pattern I’ve found myself favoring lately. Create an optimized data structure for the sorts of functions you will want to use on it. The calling code then has a 2 step process: 1) create the struct, 2) call the function. This may seem unnecessary, but you can unit test step 1 easily since there are no side effects (or randomness).

Plus, having a struct allows you to participate in Protocols. I could imagine WeightedList implementing Inspect and perhaps even Collectable if it’s an important enough part of your system.

Edited to use Enum.find_value instead of Enum.reduce_while which I find clearer for this situation

tombh

tombh

Thank you so much everyone! This is way more than I expected. It’s a lot to digest, I’ve learned so much. I like @gregvaughn’s suggestion the most, not because of the struct idea, but because it seems to me to realise the thought process in my original approach, except without all the noobish bloat. The struct idea is also fascinating as well of course.

A quick question about differentiating functions by arity, like this:

This of course seems to be quite idiomatic, but I’m struggling with reaching for it myself. I just feel that if a function does something different it should have a different name. Perhaps naively I think it’s better to make the condition explicit in say a case statement, and then call out to differently named functions just so that the intent of the code becomes more intuitive. Thoughts?

NobbZ

NobbZ

As @LostKobrakai already pointed out, both do the same thing. They select the item from the list of items.

The multiheaded function you see there, has nothing to do with “differentiating functions by arity”, in fact, nothing is differentiated there, both are the same function, as both have the same name and the same arity.

What though I do use there is “pattern matching”, something very important to learn and understand when writing elixir.

To be honest, those 2 function clauses are semantically equivalent with the following single-clause function:

defp select_item(items, idx) do
  case items do
    [{name, weight}|_] when idx < weight -> name
    [{_, weight} | tail] -> select_item(tail, idx - weight)
  end
end

But it is idiomatic to move such a case which matches on an argument directly and spans the full function body into a multi clause function.

Also in general pattern matching in the functions head or a case is usually prefered of using if and some predicate functions and/or using getter functions.

peerreynders

peerreynders


Pattern matching‡ is the conditional construct - the case expression is just one way of utilizing it.

‡ Not to be confused with destructuring:

NobbZ

NobbZ

Can’t we write select_item/3 roughly like this, making it effectively a select_item/2?

defp select_item([{name, weight} | _], idx) when idx < weight, do: name
defp select_item([{_, weight} | tail], idx), do: select_item(tail, idx - weight)

While also making use of Enum.reduce/3 in sum_weights/2 we can make it a sum_weights/1:

defp sum_weights(items) do
  Enum.reduce(items, 0, fn {_, weight}, sum -> weight + sum end)
end

Then we leave weight_based_choice nearly untouched (only adjusting calls to the functions we changed):

defmodule Demo do
  def weight_based_choice(items) do
    value =
      items
      |> sum_weights()
      |> :rand.uniform()

    select_item(items, value)
  end

  defp select_item([{name, weight} | _], idx) when idx < weight, do: name
  defp select_item([{_, weight} | tail], idx), do: select_item(tail, idx - weight)

  defp sum_weights(items) do
    Enum.reduce(items, 0, fn {_, weight}, sum -> weight + sum end)
  end
end

 1..10 |> Enum.map(fn _ -> Demo.weight_based_choice(items) end) |> IO.inspect
#=> [:bananas, :bananas, :bananas, :bananas, :bananas, :bananas, :bananas, :bananas, :oranges, :bananas]

This is probably one of the most idiomatic approaches. We could even implement select_item/2 in terms of Enum.reduce_while/3, but usually this doesn’t add much in terms of readability, also not many people are used to that function…

defp select_item(items, idx) do
  Enum.reduce_while(items, idx, fn
    {_, weight}, acc when weight < acc -> {:cont, acc - weight}
    {name, weight}, acc -> {:halt, name}
  end
end

PS: Perhaps we need to adjust </<=/>/>= in the guards… I’m writing this mostly freehand…

Where Next?

Popular in Questions Top

New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement