user20230119

user20230119

Is there a better way to traverse nested maps than this?

Is there a better way to traverse a tree structure and accumulate leaves than this code?

  def traverse_start(value, acc_list) when is_list(value) do
    for v <- value, reduce: acc_list do
      acc -> acc ++ traverse(v, acc_list)
    end
    |> Enum.uniq()
  end

  def traverse_start(value, acc_list) when is_map(value) do
    for {k, v} <- value, reduce: acc_list do
      acc -> acc ++ traverse([k], v, acc_list)
    end
    |> Enum.uniq()
  end

  def traverse(keys, value, acc_list) when is_map(value) do
    # keys |> IO.inspect
    for {k, v} <- value, reduce: acc_list ++ [keys] do
      acc -> acc ++ traverse(keys ++ [k], v, acc)
    end
  end

  def traverse(keys, value, acc_list) when is_list(value) do
    # keys |> IO.inspect
    for v <- value, reduce: acc_list ++ [keys] do
      acc -> (acc ++ traverse(keys, v, acc)) |> Enum.uniq()
    end
  end

  def traverse(keys, _value, _acc_list) do
    # keys |> IO.inspect
    [keys]
  end

This code traverses a map or list and gets a list of unique paths. It’s used to get column names for a CSV file from an array of JSON documents.

    columns = Report.traverse_start(data_list, []) |> Enum.sort()

input

    data_list = [
      %{},
      %{
        "a" => 1,
        "b" => %{"bb" => 2},
        "c" => %{"cc" => %{"ccc" => 3}},
        "n" => nil,
        "u" => %{"uu" => nil},
        "w" => %{"ww" => %{"www" => nil}},
        "r" => [],
        "z" => %{}
      },
      %{
        "a" => 1,
        "b" => %{"bb" => 2},
        "c" => %{"cc" => %{"ccc" => 3}},
        "n" => nil,
        "u" => %{"uu" => nil},
        "w" => %{"ww" => %{"www" => nil}},
        "r" => [
          %{
            "a" => 1,
            "b" => %{"bb" => 2},
            "c" => %{"cc" => %{"ccc" => 3}},
            "n" => nil,
            "u" => %{"uu" => nil},
            "w" => %{"ww" => %{"www" => nil}},
            "r" => [
              %{
                "a" => 1,
                "b" => %{"bb" => 2},
                "c" => %{"cc" => %{"ccc" => 3}},
                "n" => nil,
                "u" => %{"uu" => nil},
                "w" => %{"ww" => %{"www" => nil}},
                "r" => [],
                "z" => %{}
              },
              %{
                "d" => 4
              }
            ],
            "z" => %{}
          },
          %{
            "d" => 4
          }
        ]
      }
    ]

output

   columns = [
      ["a"],
      ["b"],
      ["b", "bb"],
      ["c"],
      ["c", "cc"],
      ["c", "cc", "ccc"],
      ["n"],
      ["r"],
      ["r", "a"],
      ["r", "b"],
      ["r", "b", "bb"],
      ["r", "c"],
      ["r", "c", "cc"],
      ["r", "c", "cc", "ccc"],
      ["r", "d"],
      ["r", "n"],
      ["r", "r"],
      ["r", "r", "a"],
      ["r", "r", "b"],
      ["r", "r", "b", "bb"],
      ["r", "r", "c"],
      ["r", "r", "c", "cc"],
      ["r", "r", "c", "cc", "ccc"],
      ["r", "r", "d"],
      ["r", "r", "n"],
      ["r", "r", "r"],
      ["r", "r", "u"],
      ["r", "r", "u", "uu"],
      ["r", "r", "w"],
      ["r", "r", "w", "ww"],
      ["r", "r", "w", "ww", "www"],
      ["r", "r", "z"],
      ["r", "u"],
      ["r", "u", "uu"],
      ["r", "w"],
      ["r", "w", "ww"],
      ["r", "w", "ww", "www"],
      ["r", "z"],
      ["u"],
      ["u", "uu"],
      ["w"],
      ["w", "ww"],
      ["w", "ww", "www"],
      ["z"]
    ]

Marked As Solved

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

You’ve got a LOT of O(n^2) going on here from appending to lists. Here is a more constant time answer:

defmodule PathFinder do
  def paths(collection) when is_map(collection) or is_list(collection) do
    Enum.flat_map(collection, &paths/1)
  end
  
  def paths({k, v}) do
    case paths(v) do
      [] ->
        [[k]]
      subpaths ->
        Enum.map(subpaths, fn subpath -> [k | subpath] end)
    end
  end
  
  def paths(_leaf) do
    []
  end
end

Tree traversal is often a lot easier to do as a body recursive function, particularly when your output shape is also pretty tree-like.

Notably these do NOT produce identical results, but I think that might be actually what you want. My version produces only full paths to leaf nodes, your answer includes paths like ["b"] that do not point to leaf nodes.

Also Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Gonna leave that as an exercise for the reader :wink: Hint: it’s in how you return the subpaths here:

      subpaths ->
        Enum.map(subpaths, fn subpath -> [k | subpath] end)

You want to make sure to include [k] itself as one of the subpaths.

Deduping is also pretty straight forward. I’d probably just rename paths to do_paths and then have def paths(val), do: val |> do_paths |> Enum.uniq.

user20230119

user20230119

This worked for me.

[[k]] ++ Enum.map(subpaths, fn subpath -> [k | subpath] end)

Your code processed my data more than twice as fast.

user20230119

user20230119

If you have columns for the parent map with its fields user, user.name, user.email, then you can filter by user to see if some rows don’t have a property for user.

ryanwinchester

ryanwinchester

Why not

[[k] | Enum.map(subpaths, fn subpath -> [k | subpath] end)]

(even though the compiler would probably rewrite it to this anyway in this case)

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Yeah I’d consider either fine, the compiler does indeed rewrite it to be identical in both cases.

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New

We're in Beta

About us Mission Statement