Hello everyone,

I am having some trouble understanding leex behaviour. To give some context, I would like to create a parser for pgn files.

Here is some code to explain my problem.

I created the lexer file pgn_lexer.xrl with

% -- Definitions.

Definitions.

TAG        = (\[[^\]]*\])+
COMMENT    = ({[^}]*})+
WHITESPACE = [\s\t\n\r]

Rules.

{TAG}         : {token, {tag, TokenLine, TokenChars}}.
{COMMENT}     : {token, {comment, TokenLine, TokenChars}}.
{WHITESPACE}+ : skip_token.

Erlang code.

The important rule is for TAG

( \[ [^\]] * \] )+

I cannot understand why

iex> :pgn_lexer.string '[Tag1 "Value1"] [Tag2 "Value2"]{comment}'                                                        
{:ok,
 [
   {:tag, 1, '[Tag1 "Value1"]'},
   {:tag, 2, '[Tag2 "Value2"]'},
   {:comment, 2, '{comment}'}
 ], 2}
# This works has expected, it detects 2 tags

iex> :pgn_lexer.string '[Tag1 "Value1"][Tag2 "Value2"]{comment}'  
{:ok, [{:tag, 1, '[Tag1 "Value1"][Tag2 "Value2"]'}, {:comment, 1, '{comment}'}],
 1}
# This does not work, tag1 and tag2 are merged

Why do the tags need to be separated by a space?

Thanks for enlightments.