[antlr-interest] Global scopes for lexers

Alex Shneyderman a.shneyderman at gmail.com
Tue Jul 31 08:51:48 PDT 2007


> It's unsupported.  May i ask why you want them?
> Ter

I have a language where new lines can be significant or not depending
on the type of curlies they are in. So, if say we are in {} new lines
are significant but if we are in () or [] they are ignored.

Currently, v2 grammar keeps a stack in lexer to keep track of this.
Every time there is { the levelNumber is pushed onto the stack and
levelNumber member var of lexer is set to 0. If } is read we pop the
stack and assign the value popped to the levelNumber instance member
of the lexer. If we see [ or ( we increment levelNumber; seeing ) or ]
we decrement the levelNumber.

Now if we see a new line and levelNumber != 0 then we skip the new
line otherwise we emit a NL (as it is significant) token.

Comparing the code generated for parser's global scope it is
practically identical to what the hackish stack solution looks like
with v2 grammar.

I thought it would be neat to have the same mechanism for lexer as
exists in parser. But of course you probably have a better idea of how
to keep track of
these levels. I am all ears :-)

Thanks,
Alex.


More information about the antlr-interest mailing list