voughtdq

voughtdq

Could someone review this CSV to data structure code?

As an exercise, I decided to convert a CSV, similar in format to the one located here, to a map. Please notice that the file is subject to some non-free license, just in case you planned to use the data commercially.

The idea is that I convert the three categories (provided as columns in the CSV file) into a map of maps of lists.

%{"Group1" => %{"Classification1" => ["Domain1", "Domain2", "Domain3"]},
              %{"Classification2" => ["Domain4", "Domain5", "Domain6"]},
              %{"Classification3" => []}, 
%{"Group2" => %{"Classification4" => ["Domain7"]}}

You’ll notice that a classification can have a missing domain (in other words, the tuple would be {grouping, classification, ""}.

I am using NimbleCSV to get the file and convert it to the map.

defmodule TaxonomyMap do
  @doc """
  Open the target file, parse it, and create a list of tuples of
  {grouping, classification, specialization} for each row.
  """
  def get_taxonomies(file) do
    file
    |> File.stream!(read_ahead: 1000)
    |> NimbleCSV.RFC4180.parse_stream
    |> Stream.map(fn [_, grouping, classification, specialization, _, _] ->
      {grouping, classification, specialization}
    end)
  end

  defp maybe_blank_list(value) do
    case value do
      "" -> []
      _ -> [value]
    end
  end

  defp classification_map(classification, specialization) do
    %{classification => maybe_blank_list(specialization)}
  end

  defp nil?(acc, key_or_keys) do
    get_in(acc, key_or_keys) |> is_nil
  end

  def run(taxonomies) do
    taxonomies
    |> Enum.reduce(%{}, fn({g, c, s}, acc) ->
      cond do
        nil?(acc, [g]) ->
          # The grouping is not in the map
          # Add the grouping, classification, and specialization for this row
          put_in(acc, [g], classification_map(c, s))
        nil?(acc, [g, c]) ->
          # The classification is not in the grouping
          # Add the classification and specialization to the grouping
          put_in(acc, [g], Map.merge(get_in(acc, [g]), classification_map(c, s)))
        !nil?(acc, [g, c]) ->
          # The classification and grouping both exist
          # Add the specialization to the grouping
          put_in(acc, [g, c], get_in(acc, [g, c]) ++ maybe_blank_list(s))
      end
    end)
  end

The way you’d use this is by running TaxonomyMap.get_taxonomies("taxonomy.csv") |> TaxonomyMap.run.

Is there anything that can or should be improved? I’d love to hear your thoughts on how I can improve for clarity or to make my code more “elixiric”.

Marked As Solved

bbense

bbense

It all looks reasonably good to me. The one place that might be more “elixiry” is the cond switch inside the reduce.

I found that part somewhat hard to reason about without flipping back and forth in the file. You’re basically doing a test and then a transformation. It’s more ‘elixiry’ to simply write the transformations as pattern matching function heads or case statements and let the computer sort out which one to use.

|> Enum.reduce(%{}, fn({g, c, s}, acc) -> inject(acc, [g, c], s ) end 

def inject( acc, [group, class], spec ) do
       case get_in(acc, [group, class] ) do 
           nil   -> inject_class( acc, [group, class], spec)
           found -> put_in(acc, [group, class], found ++ maybe_blank_lists(spec))
       end 
end  

def inject_class(acc, [group, class], spec ) do 
     case get_in(acc, [group] ) do 
           nil   -> put_in(acc, [group], classification_map(class, spec))
           found -> put_in(acc, [group], Map.merge( found, classification_map(class, spec)))
     end 
end 

I’m not entirely happy with that, but hope it shows the idea. I feel like there is probably an additional refactoring involving pulling out the case statements from inject that might make the code even more straightforward.

Also Liked

voughtdq

voughtdq

Noice! Thanks for the pattern. This is exactly what I needed to hear.

I can’t wait to get back to my workstation to refactor this.

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
AstonJ
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New

We're in Beta

About us Mission Statement