It’s hard to tell exactly what’s going on since we’re missing a lot of context, but I recommend a few “best practices” with nimbleparsec.

here is my example:

1. initialize your parser context as a struct (https://github.com/ityonemo/zigler/blob/master/lib/zig/parser.ex#L68)
2. avoid using anything besides structs in your context
3. each combinator should be a series of textual matching, only followed by single (optional post_traverse), that analyzes all text that you have kept around. The only other place I use post_traverse is for internal validations, but I make sure that these traversers don’t alter context state, they only raise on syntax error.
4. I make a conscious decision to either throw away all of the text always or, only do textual transformations. You can put arbitrary terms in your outputs, but you probably shouldn’t. It’s better to stash them in the context and pull them out at the end. Thus in most cases I avoid label, for example. Here is an example of clear helper function I wrote to make sure that I’m in a sane state after any block. https://github.com/ityonemo/zigler/blob/master/lib/zig/parser.ex#L410.
5. create tests for your more complex intermediate combinators: https://github.com/ityonemo/zigler/blob/master/lib/zig/parser.ex#L151 that you can verify in unit testing. You don’t have to keep them around in dev/prod.
6. don’t be afraid to raise. I typically use SyntaxError. If you’re new to elixir, when you’re testing, assert_raise SyntaxError, fn <erroring code> end is a thing.

A few years ago I was working on a SQL query parser, and did some tweaking to improve error handling. At the time nimble_parsec didn’t exist, we were using combine, but I think that the same approach would hold, with a major difference that we were able to make it work with combine without changing the library. In contrast I think you could only do this with nimble if you modify the library (I might be wrong though).

I found that combinators such as choice and repeat are not very good at error reporting. For example let’s say we want to parse an arithmetic expression such as 1 + (2 + 3) + 4. Somewhere, we’ll have a parser named term which is defined choice([integer, parenthesized_expression]). Now, let’s say we parse the expression 1 + (2 + 3. We want to report that closing parenthesis is missing. However, choice is too generic, so we can only report something like expecting a valid term.

We solved this by adding a custom parser we called switch, which would branch based on a lookahead. If we see that the next character is (, we commit to parenthesized_expression. If the next character is a digit, we commit to integer. Otherwise, we return a custom error. Since we commit to the choice, early, we can return the more precise error from the chosen branch. We used the same approach to improve error reporting with the optional parser.

Another issue is repeat. For example, let’s say you want to parse a list of arguments, (arg1, arg2, ...). You’ll have something like repeat(arg_parser). However, since repeat always succeeds, we can’t return a precise error that occurred in some argument. For example, if there is an error in arg3, repeat will succeed, consuming the first two args, and then the best we can do is return something like unexpected , character.

This can be handled in a similar way as choice. We first do a switch lookahead to decide if we’ll parse the first argument (next char is a valid start of the argument) or not (next char is a closing parenthesis). For every subsequent argument, we do a lookahead switch to check if there’s a next argument (next char is ,) or not (next char is a closing parenthesis). This allows us to precisely report an error in a repeated element.

The changes above will uglify the parser code. In general, it’s my experience that precise error reporting complicates the parser implementation. In our SQL parser we were able to control readability by doing two-phase parsing. We’d first convert the string intu a list of tokens such as {:constant, 123}, {:identifier, "foobar"}, … (aka tokenization), and then in the second pass we’d do parse the list of tokens according to the SQL grammar. This approach made the code of the parser significantly easier to follow and change.

It was possible to do this with combine, but I don’t think you can make it work with nimble, because nimble always expects a binary input, and if you’re doing two passes, your input in the 2nd pass is the list of terms, not a string of chars. Therefore, my impression is that nimble is better suited for the cases where precise error reporting is not required, or the cases where the grammar is not too complicated. Otherwise, I’d either roll my own combinator (I did a talk on this, you can find it here), or hand-write a recursive-descent parser. The latter will give you maximum flexibility and speed, but the code will be significantly noisier.

If precise error handling is required, I personally wouldn’t use Nimble, except maybe for tokenization.

Hand-rolling a parser for complex languages is definitely feasible. For example see this PR in Gleam where switching to a hand-rolled parser improved error reporting and parsing speed. Also, I recall reading that at some point GCC switched to a hand-rolled recursive-descent parser to improve error reporting.

In cases where the grammar is more involved, I’d probably start with a hand-rolled parser combinator. Being in control of the combinator should provide maximum flexibility, while the parser code should remain readable.

If maximum performance is required, parts of the parser could be converted to manual recursive descent. For example of this technique, take a look at this parser which parses regex-like inputs (input example: "^ENWWW(NEEE|SSE(EE|N))$", full problem description is here).

To better understand this, I advise trying these techniques on a small toy grammar. An arithmetic expression parser which supports parentheses and operator precedence is IMO a great example. You could develop it incrementally, e.g.:

1. Support flat expression with just one operator (e.g. 1+2+34).
2. Add support for ignoring whitespaces
3. Introduce the * operator, and handle operator precedence (e.g. parsing 1 * 2 + 3 * 4 should return something like {+, [{*, [1, 2]}, {*, [3, 4]}}).
4. Add support for parentheses
5. Improve quality of reported errors

Start by implementing this with the recursive descent technique. Then see how could it be done with the combinators. This should help you understand the differences between these two approaches.

As a bonus exercise, try implementing two-pass compiling. This can be done by using e.g. Nimble to convert the original input into a list of tokens (e.g. convert 1 * (2 + 3) into [{:integer, 1}, {:operator, "*"}, {:operator, "("}, ...]), and then converting such list into the final ast. IMO, when grammar is more complex, explicit tokenization pass can do wonders for code readability.

Finally, you can also consider using yecc. IIRC, Elixir parser is based on yecc. Also, absinthe does two-phase parsing, using nimble for the first pass, and yecc for the second.

I understand this may all seem overwhelming, and I barely scratched the surface of the complex topic of parsing So as a final parting advice, I think that starting with a hand-rolled combinator is probably a sensible choice which will give you a lot of control and flexibility, while the code should still be readable. As soon as the grammar becomes more complex, consider doing two-phase parsing, with the first pass powered by e.g. nimble.

Error reporting is not the best part of nimble_parsec for sure. In your case, I suspect you are seeing the result of this comment in the source where the error is being attributed to the start of the scene combinator.

Do you also label the attribute combinator? If not, does adding a label to it make the error return more relevant?

There are definitely occasions where I have thrown an exception to capture a specific error in parsing to ensure I return an error to the user that is more understandable. I’m not happy with it but it works - especially when the parsing is a compile time action as it seems yours may be.

Sorry, I was speaking from memory and it seems I misspoke.

Please try this interpretation: the offset in {line, offset} is the offset the current line starts. And the other offset is either the offset to the current line or the offset in comparison to the whole binary. If the latter, you can get the line offset by doing binary_offset - line_offset.