GavinJoyce
Trying to write a simple nimble_parsec parser
Hi,
I’m trying to write a parser using NimbleParsec v1.1.0 — Documentation which parses arbitrary text wrapped in a character like an underscore.
Here are some examples of what I’m hoping the parser will do:
"hello there" => ["hello there"]
"hello _Alex_!" => ["hello ", "_Alex_", "!"]
"Ũnicode is _allŏwed everywhere_ _ŏk?_" => ["Ũnicode is ", "_allŏwed everywhere_", "_ŏk?_"]
Here’s my attempt:
defmodule Txtparser do
import NimbleParsec
defcombinatorp :plain_string,
utf8_string([], min: 1)
|> lookahead_not(parsec(:wrapped_string))
defcombinatorp :wrapped_string,
string("_")
|> parsec(:plain_string)
|> string("_")
defparsec(:parse,
repeat(
choice(
[parsec(:wrapped_string), parsec(:plain_string)]
)
)
)
end
But I think I’m misundertanding how lookahead_not works, as the following is happening
"hi _there_" => ["hi _there_"]
instead of this desired result:
"hi _there_" => ["hi ", "_there_"]
Most Liked
ityonemo
a few NimbleParsec tips:
- try not to use defcombinatorp
- don’t be afraid to break up into smaller testable chunks
- try to limit circular references.
- test test test, I like having the
Mix.env == :testdefparsec block pattern - don’t be afraid to set variables in you “module-building-script”
- use post_traverse
here is what I got:
defmodule Np do
import NimbleParsec
basic_chars =
utf8_string([not: ?_], min: 1)
wrapped_string =
string("_")
|> concat(basic_chars)
|> string("_")
plain_string =
times(
utf8_char([])
|> lookahead_not(wrapped_string),
min: 1
)
|> choice([eos(), utf8_char([])])
|> post_traverse(:post_plain)
parser =
repeat(
choice([
wrapped_string
|> post_traverse(:post_wrapped),
plain_string
])
)
defparsec(:parse, parser)
defp post_wrapped(_rest, parsed, context, _line, _offset) do
{[parsed |> Enum.reverse |> List.to_string], context}
end
defp post_plain(_rest, parsed, context, _line, _offset) do
{[parsed |> Enum.reverse |> List.to_string], context}
end
if Mix.env() == :test do
defparsec(:basic_chars, basic_chars)
defparsec(:wrapped_string, wrapped_string)
defparsec(:plain_string, plain_string)
end
end
np_test.exs
defmodule NpTest do
use ExUnit.Case
defmacrop assert_parses(result, what) do
quote bind_quoted: [result: result, what: what] do
assert {:ok, ^result, _, _, _, _} = what
end
end
defmacrop assert_leaves(result, leftover, what) do
quote bind_quoted: [result: result, leftover: leftover, what: what] do
assert {:ok, ^result, ^leftover, _, _, _} = what
end
end
defmacrop assert_fails(what) do
quote bind_quoted: [what: what] do
assert {:error, _, _, _, _, _} = what
end
end
test "basic_chars" do
assert_parses ["foo"], Np.basic_chars("foo")
assert_leaves ["foo "], "_foo_", Np.basic_chars("foo _foo_")
assert_fails Np.basic_chars("_foo_")
end
test "wrapped_string" do
assert_parses ["_", "foo", "_"], Np.wrapped_string("_foo_")
assert_leaves ["_", "foo", "_"], " bar", Np.wrapped_string("_foo_ bar")
end
test "plain string" do
assert_parses ["foo"], Np.plain_string("foo")
assert_leaves ["foo "], "_bar_", Np.plain_string("foo _bar_")
end
end
8
GavinJoyce
Amazing, thanks so much!. I’ll dig into this and play around with it now - thank you!
1
Popular in Questions
What is the proper way to load a module from a file in to IEX?
In the python world, doing something like this pretty standard:
from ....
New
I have followed this StackOverflow post to install the specific version of Erlang.
And When I am running mix ecto.setup then getting fol...
New
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
Hi,
I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
I will often find my self writing things similar to:
case some_value do
nil -> something()
"" -> something()
_ -> someth...
New
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size?
Thanks
New
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217
Let’s say I have a map with required and optional k...
New
Other popular topics
Hi! May someone helps me, please!
I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
What is the proper way to load a module from a file in to IEX?
In the python world, doing something like this pretty standard:
from ....
New
Credo is smart enough to check for (something like) this:
assert length(the_list) == 0
with this response:
Checking if an enum is empt...
New
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
Good day to you all.
I have been struggling to get a query involving like and ilike to work.
Can anyone assist me on this, please?
pro...
New
Hi folks,
Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
New
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service.
Currently when I de...
New
I have a User schema with a :from_id field set to type :string:
defmodule TweetBot.Repo.Migrations.CreateUsers do
use Ecto.Migration
...
New
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New







