dkuku

dkuku

Reduce list of lists to a map where keys come from the first element (like csv file)

I think its a fairly common problem.
Lets say I got a csv file and I want to create a list of maps:

a,b,c
1,2,3
4,5,6

and I want to get:

[%{"a" => "1", "b" => "2", "c" => "3"}, %{"a" => "4", "b" => "5", "c" => "6"}]

I came with a solution but I’m wondering if there is a simpler and more performant way?

[keys | values] =
    "a,b,c\n1,2,3\n4,5,6" 
    |> String.split("\n") 
    |> Enum.map(&String.split(&1, ","))   

values
    |> Enum.map(&Enum.zip(keys, &1))
    |> Enum.map(&Map.new/1)

Marked As Solved

Eiji

Eiji

Regarding charlist you can write code like:

# split by any horizontal whitespace character (regex: \h)
# do it globally i.e. for all horizontal whitespaces occurrences
# return binary i.e. Elixir string (Erlang string is charlist)
'your charlist' |> :string.trim() |> :re.replace("\\h+", ',', [:global, return: :binary])

and use it as a normal csv.

Regarding ps aux command output you can create a custom parser using NimbleCSV:

defmodule Example do
  # [head | tail] pattern for fetching headers and rows
  def sample([headers | rows) do
    Enum.map(rows, &sample(headers, &1, %{}))
  end

  # again [head | tail] pattern for fetching key-value pairs from 2 lists
  # we are updating accumulator
  defp sample([key | rest_keys], [value | rest_values], acc) do
    sample(rest_keys, rest_values, Map.put(acc, key, value))
  end

  # which is returned when we are at end of headers or row cells
  defp sample([], _, acc), do: acc
  defp sample(_, [], acc), do: acc
  # or just
  # defp sample([], [], acc), do: acc
  # in case we are sure that every row length is equal to headers length
end

# for a different number of spaces between columns
# for me it's 11 columns, so I used 20 as safe value
separator = Enum.map 1..20, &String.duplicate(" ", &1)
# simple custom parser with multiple separators
NimbleCSV.define(MyParser, separator: separator)
# a csv string
csv = 'ps aux' |> :os.cmd() |> List.to_string() |> String.trim()
# parse a csv and transform it to map
result = csv |> MyParser.parse_string(skip_headers: false) |> Example.sample()

Here is how I found my highest separator length:

:os.cmd('ps aux')
|> List.to_string()
# split by everything which is not space character
|> String.split(~r/[^ ]+/)
# get highest length
|> Enum.reduce(1, fn elem, acc ->
  length = String.length(elem)
  if length > acc, do: length, else: acc
end)
# this is faster than
# list |> Enum.max_by(&String.length/1) |> String.length()
# as we do not call String.length/1 twice for separator with highest length

Helpful resources:

  1. :re Erlang module
  2. NimbleCSV.define/2 documentation

Also Liked

ericgray

ericgray

NimbleCSV is good for that.

  alias NimbleCSV.RFC4180, as: CSV
    File.read!("data/myfile.csv")
    |> CSV.parse_string()
    |> Enum.map(fn [a, b, c] -> %{a: a, b: b, c: c} end)
dkuku

dkuku

Thanks, I know I can parse it as csv but my problem really is multiple lists where the first is the header. I’m interested in the design pattern for this problem. I ended up joining the last two Enum.maps: https://dev.to/dkuku/phoenix-live-dashboard-custom-page-4chj

Where Next?

Popular in Questions Top

belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New

Other popular topics Top

belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1140 51847 244
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New

We're in Beta

About us Mission Statement