[antlr-interest] Languages where keywords can be used
as identifiers
Thomas Krebs
Thomas_Krebs at gmx.de
Wed Feb 8 00:11:40 PST 2006
John Green wrote:
>> Are there any hooks (from the parser) into the lexer, to tell it to
>> switch off testLiterals, or (due to lookahead) is it already too late
>> once the parser is parsing a rule?
>
>
> Due to the lookahead, it's already too late. Having the parser trigger
> state switches in the lexer just leads to a world of hurt.
>
> I agree about the maintenance issue with regard to keeping a list of
> unreserved keywords. The grammar I maintain (I wrote a parser for a
> language called ABL from Progress Software) has over 1000 token types
> now, most of those are unreserved keywords. I had hoped that your
> situation was different.
>
> I don't know of any way around the fact that all of the unreserved
> keywords need to be listed as a rule in your grammar. The parser that
> Antlr generates needs that rule so that it can deal with lookahead issues.
>
> John at joanju dot com
>
Facing the same problem at the same time (see my post: "literal and
string distinction", I got an idea:
Wouldn't it make sense (if possible?) to switch between different lexers
with one dealing with keywords and the other not?
Is switching of lexers possible/supported or does lookahead render this
impossible?
Thomas
More information about the antlr-interest
mailing list