dimitarvp

dimitarvp

Can you improve this? Zipping two lists, the result must be same size as the first list

Heya.
I am inviting you to copy-paste the module below and add your own functions and measure them, or simply give other ideas about how the desired result can be produced best.

I got interested in writing a function that takes two lists where the first is always bigger than the second and make pairs a la Enum.zip but not stop when the smaller list depletes. I want it to continue, while cycling through the second list, until we have a list with the same size as the first one, containing 2-size tuples.

Example:

list0 = [1, 2, 3, 4, 5]
list1 = [:a, :b]

I want the output to be:

[{1, :a}, {2, :b}, {3, :a}, {4, :b}, {5, :a}]

I came up with these two functions (and benchmark code):

defmodule Xyz do
  def pair_with_enum_reduce(list0, list1) do
    list1_tuple = List.to_tuple(list1)
    list1_last_position = tuple_size(list1_tuple) - 1

    Enum.reduce(list0, {[], 0}, fn item, {final_list, position} ->
      mapped_item = {item, elem(list1_tuple, position)}

      position =
        case position do
          ^list1_last_position ->
            0

          _ ->
            position + 1
        end

      {[mapped_item | final_list], position}
    end)
    |> elem(0)
    |> Enum.reverse()
  end

  def pair_with_stream_cycle(list0, list1) do
    stream = Stream.cycle(list1)
    Enum.zip(list0, Enum.take(stream, length(list0)))
  end

  def bench() do
    list0 = Enum.to_list(1..2000)
    list1 = [:worker_0, :worker_1, :worker_2, :worker_3, :worker_4]

    Benchee.run(%{
      "pair_with_enum_reduce" => fn -> pair_with_enum_reduce(list0, list1) end,
      "pair_with_stream_cycle" => fn -> pair_with_stream_cycle(list0, list1) end
    })
  end
end

Benchmark results on my machine:

Name                             ips        average  deviation         median         99th %
pair_with_enum_reduce        19.80 K       50.49 μs    ±28.32%          50 μs          85 μs
pair_with_stream_cycle        7.64 K      130.89 μs    ±40.15%         108 μs      327.12 μs

Comparison:
pair_with_enum_reduce        19.80 K
pair_with_stream_cycle        7.64 K - 2.59x slower +80.39 μs

I know Stream incurs some performance penalty but was rather surprised by how much. I don’t like the size of the pair_with_enum_reduce function, nor the fact that it has to call Enum.reverse at the end but it’s still significantly faster. And I don’t like that the pair_with_stream_cycle relies on calling length on the first list. Both functions I am kind of unhappy with.

Any criticisms? And, do you think you could do better?

(Alternatively, another implementation might not care about which size list is bigger.)

Marked As Solved

LostKobrakai

LostKobrakai

That’s what I created as well, though slightly different. First using Stream.unfold, then using tail recursion. The latter being about twice as fast.

Summary
def pair_with_unfold(list0, list1) do
    Stream.unfold({list0, list1, list1}, fn 
     {[], _, _list1} -> nil
     {[head_l0 | rest_l0], [], [head_l1 | rest_l1] = list1} -> {{head_l0, head_l1}, {rest_l0, rest_l1, list1}}
     {[head_l0 | rest_l0], [head_l1 | rest_l1], list1} -> {{head_l0, head_l1}, {rest_l0, rest_l1, list1}}
    end)
    |> Enum.to_list()
  end

  def pair_with_recursion(list0, list1) do
    pair_with_recursion(list0, list1, list1, [])
  end

  defp pair_with_recursion([], _, _list1, acc) do
    Enum.reverse(acc)
  end

  defp pair_with_recursion([head_l0 | rest_l0], [], [head_l1 | rest_l1] = list1, acc) do
    pair_with_recursion(rest_l0, rest_l1, list1, [{head_l0, head_l1} | acc])
  end

  defp pair_with_recursion([head_l0 | rest_l0], [head_l1 | rest_l1], list1, acc) do
    pair_with_recursion(rest_l0, rest_l1, list1, [{head_l0, head_l1} | acc])
  end

Also Liked

dimitarvp

dimitarvp

Yes, absolutely. Not all threads should be about the 2763th person panicking when iex shows them two characters when trying to print e.g. [10, 13]. :slightly_frowning_face: Or “how do I make this Ecto query” or “how do I do X with Phoenix”. Or “I haven’t read even the basics of Elixir but please write this code for me”… :021:

Threads like this one give people a chance to go deep and work on a small but (hopefully) interesting problem and show their talents. We indeed need more of them and I hope others will not be shy and start such threads.

BartOtten

BartOtten

@dimitarvp This topic is awesome!

I would like to see more of those ‘using the community to find the best solution’-topics from experienced Elixir developers (opposed to newbie questions). Great value and with a good amount of SEO it might be so that inexperienced developers find those topics first :wink:

derek-zhou

derek-zhou

How about this:

  def pair_with_body_recursion(list0, list1) do                                                     
    pair_recursion(list0, list1, list1)                                                             
  end                                                                                               
                                                                                                    
  defp pair_recursion([], _, _), do: []                                                             
  defp pair_recursion(l0, [], l1), do: pair_recursion(l0, l1, l1)                                   
  defp pair_recursion([h0 | t0], [h1 | t1], l1) do                                                  
    [{h0, h1} | pair_recursion(t0, t1, l1)]                                                         
  end                                                                                               

al2o3cr

al2o3cr

Seems like this is encountering a specifically-optimized path in Enum.zip when both arguments are lists:

derek-zhou

derek-zhou

On my machine tail recursion is faster:

  def pair_with_tail_recursion(list0, list1) do
    Enum.reverse(pair_tail_recursion(list0, list1, list1, []))
  end
  
  defp pair_tail_recursion([], _, _, acc), do: acc
  defp pair_tail_recursion(l0, [], l1, acc), do: pair_tail_recursion(l0, l1, l1, acc)
  defp pair_tail_recursion([h0 | t0], [h1 | t1], l1, acc) do
    pair_tail_recursion(t0, t1, l1, [{h0, h1} | acc])
  end

Where Next?

Popular in Questions Top

gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New

Other popular topics Top

JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1140 51847 244
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
William
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

We're in Beta

About us Mission Statement