[antlr-interest] Newbie conundrums

Thomas Brandon tbrandonau at gmail.com
Wed Aug 22 20:00:15 PDT 2007


On 8/23/07, cgorski at cgorski.org <cgorski at cgorski.org> wrote:
> On Wed, 22 Aug 2007 21:22:43 +0200
>   "Jan-Willem van den Broek" <monstermanual at gmail.com>
> wrote:
> > But what would I do if I wanted those three individual
> >values at a
> > higher level? Now they're just a single token. As a
> >workaround I tried
> > putting them in return values, but that doesn't seem to
> >be supported
> > in the lexer...
> >
> > JW
>
> Everything that you can do in the parser, you can do in
> the lexer.  What I would do is use return values and place
> them in some sort of data structure.  If isn't working for
> you, it's you, not ANTLR.
>
> Look here:
>
> http://www.cs.bris.ac.uk/Teaching/Resources/General/antlr/doc/lexer.html
>
> and search for "A Surprising Use" on the page.  It has an
> example of the lexer returning values.  Keep in mind that
> the example uses an older version of ANTLR, but the
> general idea is the same.
ANTLR 3 does not support return values in the lexer.
By overriding emit and nextToken you can have ANTLR allow multiple
tokens to be emitted from a single rule which would be one way to
solve this issue. The basic idea is to replace the token variable with
a list of tokens.

Tom.
>
> -Chris
>
>


More information about the antlr-interest mailing list