zatae

zatae

Most efficient way to retrieve multiple values from nested map

Let’s say I have nested map like this :

data = %{
    gps: %{
        lat: 48.857144,
        lon: 2.340242,
        altitude: 35.0
    },
    metadata: %{
        payload: %{
            content: "1aff4c0002154a080bff4c0010774058308876aa3a29",
            scantime: 1644362438
        }
        macaddress: "B8:C0:78:CD:12:AB",
        receivetime: 1663000169,
        tags: "v1.2,test.app"
    }
}

# There is way more depth and values but I will keep it simple here.

What would be the best way to put lat, lon, scantime & receivetime in a new map ? The idea behind this would be to use this new simplified map with Ecto to add rows in database. I also need to ensure that every needed values is set.

Currently I am using something like this :

with {:ok, gps} <- Map.fetch(data, :gps),
     {:ok, lat} <- Map.fetch(gps, :lat),
     {:ok, lon} <- Map.fetch(gps, :lon),
     {:ok, metadata} <- Map.fetch(data, :metadata),
     {:ok, payload} <- Map.fetch(metadata, :payload),
     {:ok, scantime} <- Map.fetch(payload, :scantime),
     ... do
        %{
            lat: lat,
            lon: lon,
            scantime: scantime,
            ...
        }
     end

I need to extract something like twelve values from even more nested map, this looks really awful. Any idea?

Marked As Solved

Eiji

Eiji

Here you go:

defmodule Example do
  def sample(acc \\ %{}, data, info)

  # in case nested value does not exists
  def sample(acc, nil, _info), do: acc

  # instead of above you may want to use another code
  # as it would place a nil value for each nested info who does not exists in data you passed
  # 
  # def sample(acc, nil, info) when is_atom(info), do: Map.put(acc, info, nil)
  # def sample(acc, nil, info) when is_list(info), do: Enum.reduce(info, acc, &sample(&2, nil, &1))
  # 
  # def sample(acc, nil, info) when is_map(info) do
  #   Enum.reduce(info, acc, &sample(&2, nil, elem(&1, 1)))
  # end

  # when we need to fetch a flat list of fields
  # or said list + some extra nested fields
  def sample(acc, data, info) when is_list(info) do
    # there should be no more than one map in info list
    # the map info stores information for nested fields
    # the remaining items is a list of fields to take from current data
    groups = Enum.group_by(info, &is_map/1)
    [map_info] = groups[true] || [%{}]
    groups[false] |> Enum.reduce(acc, &Map.put(&2, &1, data[&1])) |> sample(data, map_info)
  end

  # here we are reducing info map over our acc
  # which means all nested fields logic goes here
  def sample(acc, data, info) when is_map(info) do
    Enum.reduce(info, acc, fn {info_key, info_value}, acc ->
      sample(acc, data, info_key, info_value)
    end)
  end

  # this clause would match if we want to fetch just one nested field
  defp sample(acc, data, info_key, info_value) when is_atom(info_value) do
    Map.put(acc, info_value, get_in(data, [info_key, info_value]))
  end

  # in any other case we have a map or list which we already support
  # so all we need to do is to call the same logic, but with nested data
  defp sample(acc, data, info_key, info_value) when is_list(info_value) or is_map(info_value) do
    sample(acc, data[info_key], info_value)
  end
end

data = %{
  gps: %{
    lat: 48.857144,
    lon: 2.340242,
    altitude: 35.0
  },
  metadata: %{
    payload: %{
      content: "1aff4c0002154a080bff4c0010774058308876aa3a29",
      scantime: 1_644_362_438
    },
    macaddress: "B8:C0:78:CD:12:AB",
    receivetime: 1_663_000_169,
    tags: "v1.2,test.app"
  }
}

info = %{gps: [:lat, :lon], metadata: [%{payload: :scantime}, :receivetime], a: %{b: :c}}
iex> Example.sample(data, info)

This code would automatically pick data you need by simply passing an info. Also if you want to add something to result you simply can pass it as first argument which means that you can call this function multiple times for different data:

data1
# without passing acc
|> Example.sample(info1)
# with acc (result of above pipe)
|> Example.sample(data2, info2)

Helpful resources:

  1. is_atom/1, is_list/1 and is_map/1 guards
  2. Enum.reduce/3
  3. Map.put/3

Also Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

In this case, pattern match!

%{
    gps: %{
        lat: lat,
        lon: lon,
        altitude: alt
    },
    metadata: %{
        payload: %{
            scantime: scantime
        }
        receivetime: receive_time,
    } = data
}

Pattern matching is super useful here because it provides a way to declaratively extract what you want.

tcoopman

tcoopman

You can also have a look at a library like GitHub - hissssst/pathex: Fastest way to access data in Elixir.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

What should happen if a value is missing? Is that a case you need to worry about, or is the shape of the payload reliable?

Eiji

Eiji

The pattern matching is not useful only for validation, but also for assigning variables. Simplest example:

iex> %{a: a} = %{a: 5}
%{a: 5}
iex> a
5

What @benwilson512 suggested is to do exactly same for nested structures, for example:

iex> %{nested: %{a: a}, b: b} = %{nested: %{a: 5}, b: 10}
%{nested: %{a: 5}, b: 10}
iex> %{a: a, b: b}
%{a: 5, b: 10}

However since you said that you have many cases for this and they could be even more complicated, the pattern matching here for just a single data structure would take lots of lines. For me it looks much worse comparing to usage of a simple info variable in my example.

zatae

zatae

I understand now ! Thank you :slightly_smiling_face:

Where Next?

Popular in Questions Top

gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

Other popular topics Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

We're in Beta

About us Mission Statement