[antlr-interest] newbie lookahead question

John B. Brodie jbb at acm.org
Sat Apr 22 06:26:56 PDT 2006


>John: I'm not sure if your previous remark that the members of the 
>tokens sections are rules is correct.
>
>> And, oh by the way, that stuff between the "s in the tokens{...} 
>> section *IS*
>> a lexer rule --- it means:
>>
>>         'match this explicit string literal when testLiterals is true' 
>
>I think more precisely it says 'when you match a token with a lexer rule 
>and if testLiterals is true, then check the token section to see if it 
>matches a string and output that token if it does.'
>
>So I think it is more of modifier of  the output of the lexer rules 
>(those that have the testLiteral option turned on)  than a lexer rule in 
>its own right.
>
>Do you agree with that?

Yes.


More information about the antlr-interest mailing list