[antlr-interest] skipping whitespaces in code and avoiding it in comments

Gavin Lambert antlr at mirality.co.nz
Sun Mar 8 13:04:45 PDT 2009


At 08:44 9/03/2009, Maciej Gawinecki wrote:
 >Lexer can /skip/ or send to /hidden channel/ the tokens that are 

 >whitespaces.
 >
 >However I would like it not to skip them when the parser 
recognizes
 >a comment fragment, because I want to buffer comments including
 >their whitespaces.

Provided that there are some lexically unique markers for comments 
(like there appears to be in your language, bounded by // and 
newline), then by far the easiest/best thing to do is to match the 
entire comment in a single lexer rule.  That way it'll 
automatically include all the whitespace, without any need to mess 
with the normal WS rule.



More information about the antlr-interest mailing list