dimitarvp

dimitarvp

Can you improve this? Zipping two lists, the result must be same size as the first list

Heya.
I am inviting you to copy-paste the module below and add your own functions and measure them, or simply give other ideas about how the desired result can be produced best.

I got interested in writing a function that takes two lists where the first is always bigger than the second and make pairs a la Enum.zip but not stop when the smaller list depletes. I want it to continue, while cycling through the second list, until we have a list with the same size as the first one, containing 2-size tuples.

Example:

list0 = [1, 2, 3, 4, 5]
list1 = [:a, :b]

I want the output to be:

[{1, :a}, {2, :b}, {3, :a}, {4, :b}, {5, :a}]

I came up with these two functions (and benchmark code):

defmodule Xyz do
  def pair_with_enum_reduce(list0, list1) do
    list1_tuple = List.to_tuple(list1)
    list1_last_position = tuple_size(list1_tuple) - 1

    Enum.reduce(list0, {[], 0}, fn item, {final_list, position} ->
      mapped_item = {item, elem(list1_tuple, position)}

      position =
        case position do
          ^list1_last_position ->
            0

          _ ->
            position + 1
        end

      {[mapped_item | final_list], position}
    end)
    |> elem(0)
    |> Enum.reverse()
  end

  def pair_with_stream_cycle(list0, list1) do
    stream = Stream.cycle(list1)
    Enum.zip(list0, Enum.take(stream, length(list0)))
  end

  def bench() do
    list0 = Enum.to_list(1..2000)
    list1 = [:worker_0, :worker_1, :worker_2, :worker_3, :worker_4]

    Benchee.run(%{
      "pair_with_enum_reduce" => fn -> pair_with_enum_reduce(list0, list1) end,
      "pair_with_stream_cycle" => fn -> pair_with_stream_cycle(list0, list1) end
    })
  end
end

Benchmark results on my machine:

Name                             ips        average  deviation         median         99th %
pair_with_enum_reduce        19.80 K       50.49 μs    ±28.32%          50 μs          85 μs
pair_with_stream_cycle        7.64 K      130.89 μs    ±40.15%         108 μs      327.12 μs

Comparison:
pair_with_enum_reduce        19.80 K
pair_with_stream_cycle        7.64 K - 2.59x slower +80.39 μs

I know Stream incurs some performance penalty but was rather surprised by how much. I don’t like the size of the pair_with_enum_reduce function, nor the fact that it has to call Enum.reverse at the end but it’s still significantly faster. And I don’t like that the pair_with_stream_cycle relies on calling length on the first list. Both functions I am kind of unhappy with.

Any criticisms? And, do you think you could do better?

(Alternatively, another implementation might not care about which size list is bigger.)

Marked As Solved

LostKobrakai

LostKobrakai

That’s what I created as well, though slightly different. First using Stream.unfold, then using tail recursion. The latter being about twice as fast.

Summary
def pair_with_unfold(list0, list1) do
    Stream.unfold({list0, list1, list1}, fn 
     {[], _, _list1} -> nil
     {[head_l0 | rest_l0], [], [head_l1 | rest_l1] = list1} -> {{head_l0, head_l1}, {rest_l0, rest_l1, list1}}
     {[head_l0 | rest_l0], [head_l1 | rest_l1], list1} -> {{head_l0, head_l1}, {rest_l0, rest_l1, list1}}
    end)
    |> Enum.to_list()
  end

  def pair_with_recursion(list0, list1) do
    pair_with_recursion(list0, list1, list1, [])
  end

  defp pair_with_recursion([], _, _list1, acc) do
    Enum.reverse(acc)
  end

  defp pair_with_recursion([head_l0 | rest_l0], [], [head_l1 | rest_l1] = list1, acc) do
    pair_with_recursion(rest_l0, rest_l1, list1, [{head_l0, head_l1} | acc])
  end

  defp pair_with_recursion([head_l0 | rest_l0], [head_l1 | rest_l1], list1, acc) do
    pair_with_recursion(rest_l0, rest_l1, list1, [{head_l0, head_l1} | acc])
  end

Also Liked

dimitarvp

dimitarvp

Yes, absolutely. Not all threads should be about the 2763th person panicking when iex shows them two characters when trying to print e.g. [10, 13]. :slightly_frowning_face: Or “how do I make this Ecto query” or “how do I do X with Phoenix”. Or “I haven’t read even the basics of Elixir but please write this code for me”… :021:

Threads like this one give people a chance to go deep and work on a small but (hopefully) interesting problem and show their talents. We indeed need more of them and I hope others will not be shy and start such threads.

BartOtten

BartOtten

@dimitarvp This topic is awesome!

I would like to see more of those ‘using the community to find the best solution’-topics from experienced Elixir developers (opposed to newbie questions). Great value and with a good amount of SEO it might be so that inexperienced developers find those topics first :wink:

derek-zhou

derek-zhou

How about this:

  def pair_with_body_recursion(list0, list1) do                                                     
    pair_recursion(list0, list1, list1)                                                             
  end                                                                                               
                                                                                                    
  defp pair_recursion([], _, _), do: []                                                             
  defp pair_recursion(l0, [], l1), do: pair_recursion(l0, l1, l1)                                   
  defp pair_recursion([h0 | t0], [h1 | t1], l1) do                                                  
    [{h0, h1} | pair_recursion(t0, t1, l1)]                                                         
  end                                                                                               

al2o3cr

al2o3cr

Seems like this is encountering a specifically-optimized path in Enum.zip when both arguments are lists:

derek-zhou

derek-zhou

On my machine tail recursion is faster:

  def pair_with_tail_recursion(list0, list1) do
    Enum.reverse(pair_tail_recursion(list0, list1, list1, []))
  end
  
  defp pair_tail_recursion([], _, _, acc), do: acc
  defp pair_tail_recursion(l0, [], l1, acc), do: pair_tail_recursion(l0, l1, l1, acc)
  defp pair_tail_recursion([h0 | t0], [h1 | t1], l1, acc) do
    pair_tail_recursion(t0, t1, l1, [{h0, h1} | acc])
  end

Where Next?

Popular in Questions Top

lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New

Other popular topics Top

lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

We're in Beta

About us Mission Statement