[antlr-interest] Newbie conundrums
tbrandonau at gmail.com
Wed Aug 22 20:00:15 PDT 2007
On 8/23/07, cgorski at cgorski.org <cgorski at cgorski.org> wrote:
> On Wed, 22 Aug 2007 21:22:43 +0200
> "Jan-Willem van den Broek" <monstermanual at gmail.com>
> > But what would I do if I wanted those three individual
> >values at a
> > higher level? Now they're just a single token. As a
> >workaround I tried
> > putting them in return values, but that doesn't seem to
> >be supported
> > in the lexer...
> > JW
> Everything that you can do in the parser, you can do in
> the lexer. What I would do is use return values and place
> them in some sort of data structure. If isn't working for
> you, it's you, not ANTLR.
> Look here:
> and search for "A Surprising Use" on the page. It has an
> example of the lexer returning values. Keep in mind that
> the example uses an older version of ANTLR, but the
> general idea is the same.
ANTLR 3 does not support return values in the lexer.
By overriding emit and nextToken you can have ANTLR allow multiple
tokens to be emitted from a single rule which would be one way to
solve this issue. The basic idea is to replace the token variable with
a list of tokens.
More information about the antlr-interest