[antlr-interest] Separating text from comment in a source file

[Johannes Luber]

Mark Venbrux wrote:
> Hi,
> 
> I would like to separate text from comment in a source file. In the end
> I would like to keep the source code as is, and to process the comments.
> I tried the following grammar, but it doesn't work as expected.
> Warning: [17:01:59] warning(201): d:\temp\antlr.g:26:40: The following
> alternatives are unreachable: 1
> This is about the TEXT rule.
> 
> Comments are picked up, but text is skipped. What is wrong here?
> 
> Cheers,
> Mark

Why is filter=true? With your grammar you don't actually skip anything
but match all text into some tokens. If this isn't enough, then you need
to change TEXT into something like that:

TEXT
   :  (options {greedy=false;} : ~COMMENT_STARTER )+
   ;

COMMENT_STARTER : '/*';

In this case greedy=false may not be needed after all, too, but test it
with this option first.

Best regards,
Johannes Luber

> grammar antlr;
> options {filter=true;}
> @header {
> package test;
> }
> 
> @lexer::header {package test;}
> 
> @members {
> }
> 
> file:  (
> 
>                COMMENT  {System.out.println("CO: "+$COMMENT.text);}
>        |       TEXT     {System.out.println("TE: "+$TEXT.text);}
>        )*
>        ;
> 
> 
> 
> COMMENT
>     :   '/*' (options {greedy=false;} : . )* '*/'
>     ;
> 
> TEXT
>        :  (options {greedy=false;} : . )+
>        ;
>