hscspring

hscspring

How to read text into a dictionary

I have some data like:



---
title1
line1
line2
---
title2
line1
line2

and i want a structure of :

%{title1: [title1/line1, title1/line2], title2: [title2/line1, title2/line2}

Here is my code, but i cannot get the dict:

defmodule Index do
    @spliter "---"
    def build_custom_index(index_path) do
        {:ok, contents} = File.read(index_path)
        contents \
        |> String.split(@spliter, trim: true)
        |> Stream.map( &(String.split(&1, "\n", trim: true)) )
        |> Stream.filter( &(Enum.count(&1) > 0) )
        |> Enum.each( fn group -> 
            [head | tail] = group
            tail \
            |> Enum.each( fn line -> 
                new_line = head <> "/" <> line
            end)
        end)
    end
end

This is actually my first Elixir program, I have searched several materials but still in trouble.

I have used Python before.

Would anyone help… many thanks.

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hey @hscspring welcome! The main thing to remember with Elixir is that you should always be thinking about returning values. You can never mutate data, so if you want something different than you have now, you need to construct some kind of function that will return the changed data. Here’s an example from your code:

tail \
|> Enum.each( fn line -> 
  new_line = head <> "/" <> line
end)

Here, instead of doing each and trying to create some new variable outside the function, use

new_lines = Enum.map(tail, fn line -> head <> "/" <> line end)

Which takes the tail list and returns a new list where the function has been applied to each item.

Overall you need two do two things: iterate through the groups, and maintain a dictionary. I’m going to show you two different ways to do this.

The first way will be to build a recursive function that walks through the groups “manually”:

defmodule Index do
  @spliter "---"
  
  def index_from_file(index_path) do
    {:ok, contents} = File.read(index_path)
    build_index(contents)
  end

  def build_index(contents) do
    contents
    |> String.split(@spliter, trim: true)
    |> Stream.map(&String.split(&1, "\n", trim: true))
    |> Stream.filter(&(Enum.count(&1) > 0))
    |> index_groups(%{})
  end
  
  defp index_groups([], index) do
    index
  end

  defp index_groups([group | groups], index) do
    [title | items] = group
    items = Enum.map(items, fn item -> head <> "/" <> line)
    index = Map.put(index, title, items)
    index_groups(groups, index)
  end
end

As a simple thing, note that I split the idea of reading from a file from building the index. This is a common pattern in Elixir, where you try to maximize the number of functions that are “pure” and don’t depend on external inputs or outputs.

The main thing though is the recursive build_index function. It takes the list of groups as a first arg, and then the dictionary as the second arg. If there are no groups, then it just returns the index. If there is a group, it uses Map.put to return a new index containing the updated rows, and then recursively passes remaining groups and the update index to itself.

This pattern is so common that there’s the handy https://hexdocs.pm/elixir/Enum.html#reduce/3 function:

  def build_index(contents) do
    contents
    |> String.split(@spliter, trim: true)
    |> Stream.map(&String.split(&1, "\n", trim: true))
    |> Stream.filter(&(Enum.count(&1) > 0))
    |> Enum.reduce(%{}, fn group, index ->
      [title | items] = group
      items = Enum.map(items, fn item -> head <> "/" <> line)
      Map.put(index, title, items)
    end)
  end

There are a bunch of other little changes that could be done to the code, but hopefully this helps introduce the core concepts around iterating through data and returning new data. Enum.each/2 is honestly used pretty rarely, it’s only useful when you want to do some kind of side effect and you don’t care about keeping the result.

hauleth

hauleth

I haven’t tested it, but this should work. Just remember that content need to be list of lines.

def build(content), do: do_build(content, nil, [])

defp do_build([], agg), do: Map.new(agg)
defp do_build(["---", title | rest], agg) do
  {entries, rest} = content(rest)

  do_build(rest, [{title, entries} | agg])
end

defp content(data, agg \\ [])
defp content([], agg), do: {Enum.reverse(agg), []}
defp content(["---" | _] = rest, agg), do: {Enum.reverse(agg), rest}
defp content([entry | rest], agg), do: content(rest, [entry | agg])
hscspring

hscspring

It’s really kind of you, I’ve gotten that. That’s quite interesting, thank you again.

hscspring

hscspring

Thanks a lot. I’m trying.

hscspring

hscspring

I’m sorry, i got an error like this:

** (FunctionClauseError) no function clause matching in Index.index_groups/2

    The following arguments were given to Index.index_groups/2:

        # 1
        #Stream<[enum: ["\n\n", "\ntitle1\nline1\nline2\n", "\ntitle2\nline1\nline2"], funs: [#Function<48.51129937/1 in Stream.map/2>, #Function<40.51129937/1 in Stream.filter/2>]]>

        # 2
        %{}

and i have to change the code to:

defmodule Index do
    @spliter "---"

    def index_from_file(index_path) do
        {:ok, contents} = File.read(index_path)
        build_custom_index(contents)
    end

    def build_custom_index(contents) do
        stream = contents
        |> String.split(@spliter, trim: true)
        |> Stream.map(&String.split(&1, "\n", trim: true))
        |> Stream.filter(&(Enum.count(&1) > 0))
        # |> index_groups(%{})
        # add this line
        index_groups(Enum.to_list(stream), %{})
    end

    defp index_groups([], index) do
        index
    end

    defp index_groups([group | groups], index) do
        [title | items] = group
        items = Enum.map(items, fn item -> title <> "/" <> item end)
        index = Map.put(index, title, items)
        index_groups(groups, index)
    end
end

then the code is ok. But i still do not understand the error .
i’ve found this: Elixir: (FunctionClauseError) no function clause matching - Stack Overflow, but i don’t think this is my problem.

i guess the problem may come from the Stream, but i can’t find out that.

Oh, by the way, the reduce approach worked well.

Where Next?

Popular in Questions Top

pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

We're in Beta

About us Mission Statement