[antlr-interest] Error with lexer unable to find comment tokens

[Jared Bunting]

I was able to get it to work by changing your Multiline rule:

Multiline    :    '/*' ('*'~'/'|~'*')* '*/' {$channel=HIDDEN;} ;

-Jared

Sidharth Kuruvila wrote:
> I get the following result when iterating through the tokens from my 
> lexer.
>
> [@-1,0:4='<?php',<6>,1:0]
> line 1:18 no viable alternative at character ''
> line 1:6 no viable alternative at character '*'
> line 1:14 no viable alternative at character '*'
> line 1:16 no viable alternative at character '?'
> [@-1,7:13='comment',<4>,1:7]
> [@-1,16:17='?>',<6>,1:16]
>
> This is for the input <?php/*comment*/?>. The lexer appears to be 
> unable to match comments.  The comment matching works fine when I 
> remove the start condition from BodyString.
>
> This might be an issue with the DFA, it matches the comment, but 
> continues to try and match the charector after the comment.
>
> Test.g
> --------
>
> grammar Test;
>
> options {
>     backtrack = true;
>     memoize = true;
> }
>
> @lexer::members{
>     boolean start=true;
> }
>
>
> test: Identifier;
>
>
> BodyString :
>     ({start}?=> ( options {greedy=false;} : . )* '<?php' {start = 
> false;})|
>     '?>' ((BodyStringTest)=>( options {greedy=false;} : . )* '<?php' | 
> (~'<' | '<' ~'?')*)
>     ;
>
>
> fragment
> BodyStringTest :
>     ( options {greedy=false;} : . )* '<?php';
>
>
> Multiline    :    '/*' ( options {greedy=false;} : . )* '*/' 
> {$channel=HIDDEN;}
>     ;
>
> Identifier
>     : '$'*('a'..'z'|'A'..'Z'|'_')('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
>     ;
>
> WhiteSpace
> @init{
>     $channel=HIDDEN;
> }
>     :    ' '| '\t'| '\n'|'\r'
>     ;
>
> Test.Java
> -------------
> import org.antlr.runtime.*;
> //import arg.antlr.runtime.ANTLRStringStream;
> //import org.antlr.runtime.CommonTokenStream;
>
> public class Test{
>     public static void main(String[] args)throws Exception{
>         ANTLRStringStream ass = new 
> ANTLRStringStream("<?php/*comment*/?>");
>         TestLexer tpl = new TestLexer(ass);
>         System.out.println(tpl.nextToken());
>         System.out.println(tpl.nextToken());
>         System.out.println(tpl.nextToken());
>     }
> }
>
> -- 
> I am but a man.
> ------------------------------------------------------------------------
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>