John-Goff

John-Goff

Tagging data with NimbleParsec based on a lookahead

Hey everyone,

I’m trying to write a parser for todo.txt using NimbleParsec, it’s my first time writing a parser as well as first time being exposed to the idea of parser combinators, so I’m not really sure how to express what I want. At the moment, I have a pretty basic parser which correctly does done state as well as priority of a task. I’m trying to parse dates next.

The todo.txt specification says that two dates can be provided, a start date and an end date. A line item with both a start and end date might look like this

x 2021-01-02 2021-01-01 Make a new years resolution

If two dates are provided, the end date must come first followed by the start date. If only one date is given, it should be the start date. Right now, I have my dates defined like this

  date =
    integer(4)
    |> ignore(string("-"))
    |> integer(2)
    |> ignore(string("-"))
    |> integer(2)

  start_date =
    date
    |> post_traverse({TodoTex.ParserHelper, :map_date, [:end]})
    |> lookahead_not(date)

  end_date =
    date
    |> post_traverse({TodoTex.ParserHelper, :map_date, [:end]})

the function ParserHelper.map_date turns the three integers into a %Date{} struct, as well as adding :start or :end respectively. It looks like this:

  def map_date(_rest, results, context, _line, _offset, tag) do
    {[{:date, tag, apply(Date, :new!, Enum.reverse(results))}], context}
  end

and finally my parser is created like so:

  defparsec(
    :todo,
    optional(done)
    |> optional(ignore(string(" ")))
    |> optional(priority)
    |> optional(ignore(string(" ")))
    |> optional(choice([start_date, end_date]))
    |> optional(ignore(string(" ")))
    |> optional(end_date)
    |> optional(ignore(string(" "))),
    debug: true
  )

When I run this parser with the string with two dates, both of them are tagged with :end. If I run the parser on a string with only one date it’s tagged with :end as well. How can I get the date to be correctly tagged as start if it’s either the only date or the last date given? Thanks.

Marked As Solved

kip

kip

ex_cldr Core Team

Looks like you’re tagging both of these with :end which might be part of the issue.

In a more general case, this kind of challenge is better suited to an approach of parsing the longest match first, then shorter matches. This way you avoid having to do lookahead and you leverage the parser combinators automatic backtracking. Making everything optional also makes it harder to understand the intent and harder to debug. Using your example, I would think about something like this:

  import ParserHelpers

  defparsec(
    :todo,
    optional(done())
    |> ignore_whitespace() 
    |> choice([
      end_date_and_start_date(),
      start_date()
    ])
  )

  # In parser_helpers.ex
  defmodule ParserHelpers do
    def end_date_and_start_date do
      date()
      |> ignore_whitespace()
      |> date()
    end

    def start_date do
      date()
    end

    def ignore_whitespace do
      ascii_char([?\s, ?\t])
      |> ignore()
      |> repeat()
    end
  end

Also Liked

kip

kip

ex_cldr Core Team

The reason is that in your line optional(choice([start_date, end_date])), the start_date will match and be tagged with :start

Then later on you have optional(end_date) which will be tagged with :end.

Overall your code is expected the dates to be in the order of start_date end_date and that sounds like what you are seeing.

Where Next?

Popular in Questions Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New

Other popular topics Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New

We're in Beta

About us Mission Statement