[antlr-interest] Error with lexer unable to find comment tokens
Sidharth Kuruvila
sidharth.kuruvila at gmail.com
Sat Dec 20 08:57:14 PST 2008
I get the following result when iterating through the tokens from my lexer.
[@-1,0:4='<?php',<6>,1:0]
line 1:18 no viable alternative at character ''
line 1:6 no viable alternative at character '*'
line 1:14 no viable alternative at character '*'
line 1:16 no viable alternative at character '?'
[@-1,7:13='comment',<4>,1:7]
[@-1,16:17='?>',<6>,1:16]
This is for the input <?php/*comment*/?>. The lexer appears to be unable to
match comments. The comment matching works fine when I remove the start
condition from BodyString.
This might be an issue with the DFA, it matches the comment, but continues
to try and match the charector after the comment.
Test.g
--------
grammar Test;
options {
backtrack = true;
memoize = true;
}
@lexer::members{
boolean start=true;
}
test: Identifier;
BodyString :
({start}?=> ( options {greedy=false;} : . )* '<?php' {start = false;})|
'?>' ((BodyStringTest)=>( options {greedy=false;} : . )* '<?php' | (~'<'
| '<' ~'?')*)
;
fragment
BodyStringTest :
( options {greedy=false;} : . )* '<?php';
Multiline : '/*' ( options {greedy=false;} : . )* '*/'
{$channel=HIDDEN;}
;
Identifier
: '$'*('a'..'z'|'A'..'Z'|'_')('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
;
WhiteSpace
@init{
$channel=HIDDEN;
}
: ' '| '\t'| '\n'|'\r'
;
Test.Java
-------------
import org.antlr.runtime.*;
//import arg.antlr.runtime.ANTLRStringStream;
//import org.antlr.runtime.CommonTokenStream;
public class Test{
public static void main(String[] args)throws Exception{
ANTLRStringStream ass = new ANTLRStringStream("<?php/*comment*/?>");
TestLexer tpl = new TestLexer(ass);
System.out.println(tpl.nextToken());
System.out.println(tpl.nextToken());
System.out.println(tpl.nextToken());
}
}
--
I am but a man.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20081220/67c489ee/attachment.html
More information about the antlr-interest
mailing list