[antlr-interest] Error with lexer unable to find comment tokens

[Sidharth Kuruvila]

I get the following result when iterating through the tokens from my lexer.

[@-1,0:4='<?php',<6>,1:0]
line 1:18 no viable alternative at character ''
line 1:6 no viable alternative at character '*'
line 1:14 no viable alternative at character '*'
line 1:16 no viable alternative at character '?'
[@-1,7:13='comment',<4>,1:7]
[@-1,16:17='?>',<6>,1:16]

This is for the input <?php/*comment*/?>. The lexer appears to be unable to
match comments.  The comment matching works fine when I remove the start
condition from BodyString.

This might be an issue with the DFA, it matches the comment, but continues
to try and match the charector after the comment.

Test.g
--------

grammar Test;

options {
    backtrack = true;
    memoize = true;
}

@lexer::members{
    boolean start=true;
}


test: Identifier;


BodyString :
    ({start}?=> ( options {greedy=false;} : . )* '<?php' {start = false;})|
    '?>' ((BodyStringTest)=>( options {greedy=false;} : . )* '<?php' | (~'<'
| '<' ~'?')*)
    ;


fragment
BodyStringTest :
    ( options {greedy=false;} : . )* '<?php';


Multiline    :    '/*' ( options {greedy=false;} : . )* '*/'
{$channel=HIDDEN;}
    ;

Identifier
    : '$'*('a'..'z'|'A'..'Z'|'_')('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
    ;

WhiteSpace
@init{
    $channel=HIDDEN;
}
    :    ' '| '\t'| '\n'|'\r'
    ;

Test.Java
-------------
import org.antlr.runtime.*;
//import arg.antlr.runtime.ANTLRStringStream;
//import org.antlr.runtime.CommonTokenStream;

public class Test{
    public static void main(String[] args)throws Exception{
        ANTLRStringStream ass = new ANTLRStringStream("<?php/*comment*/?>");
        TestLexer tpl = new TestLexer(ass);
        System.out.println(tpl.nextToken());
        System.out.println(tpl.nextToken());
        System.out.println(tpl.nextToken());
    }
}

-- 
I am but a man.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20081220/67c489ee/attachment.html