[antlr-interest] Reuse of tokens and rules
Emond Papegaaij
e.papegaaij at student.utwente.nl
Mon Jul 3 08:26:45 PDT 2006
On Friday 30 June 2006 16:48, Thomas Brandon wrote:
> On 7/1/06, Emond Papegaaij <e.papegaaij at student.utwente.nl> wrote:
> <cut need token reuse>
> > Is it possible to write small specifications for these constructs and
> > import the one needed? These rules are only a small portion of all the
> > rules. I don't like the solution of having to maintain the same parser and
> > lexer for multiple target languages.
> >
> > PS. I'm using ANTLR v3.
>
> Check out the island-grammar example in the v3 examples pack. Should give
> you the basis for everything you need.
I'm running into a problem with the island-grammar solution. In that example
the boundaries of the tokens are easily matched in the outer language. For
example javadoc starts with /** and ends with */. The embedded Simple
statements are surrounded by '{' and '}'. In my language I most of
the 'islands' are not that clear. For example 'identifier' is taken from the
java specification.
In the island-grammar example the nested grammar is started with the beginning
of the token is matched. However in my case there is no way to know when to
switch grammars. It seems I need some way to try to parse a token (and thus a
rule in the nested grammar) and continue when it is not a match.
Another problem I'm facing is terminating the nested grammar. Most rules don't
end with a specific token. This makes it impossible to end the lexer. Isn't
it possible to have the lexer produce tokens on demand, rather than at the
start? That way I can simply return from the parser rule and continue with
the outer grammar.
Best regards,
Emond
More information about the antlr-interest
mailing list