[antlr-interest] Looping Lexer
Terence Parr
parrt at cs.usfca.edu
Mon Jun 11 17:03:15 PDT 2007
On Jun 11, 2007, at 4:57 PM, Stephen Gargan wrote:
> The question remains though, what is wrong with my grammar that
> would cause it to loop like this? It compiles fine, though that is
> not to say it is valid ;), I'm new to all of this and would not be
> surprised if it were my fault (indeed I'm hoping this is the case).
> Is there anything obviously wrong in my grammar that would cause it
> to loop like this? If so is there anything I should look for as
> tell-tale to avoid this problem? Might there be a way for the lexer
> to know it is stuck in a loop and complain?
No empty lexer rules. ;) * and ? are bad in the lexer if they are
entire production. I need to make a warning for this.
Ter
>
> grammar simple;
>
> paths : (path)*;
> path : VOLUME | NAME NEWLINE;
> VOLUME : 'volume' NUMBER;
> NAME : (WORD? WS?)*;
> NUMBER : ('0'..'9')+ ;
> WORD : ('a'..'z'|'A'..'Z'|'_')*;
> WS : ( '\t' | ' ')+ { $channel = HIDDEN; } ;
> NEWLINE : '\r'? '\n' ;
>
> The following snippet of code runs using the files generated by
> antlrworks and gets stuck in the lexer. The test input is
>
> volume1
> this is a test volume2
>
> public class LexerTest {
>
> public static void main(String[] args) throws Exception{
> simpleLexer lex = new simpleLexer(new ANTLRFileStream("/tmp/
> antlrworks/__Test___input.txt"));
> System.out.println("Created Lexer");
>
> Token t = null;
> do{
> t = lex.nextToken();
> System.out.println(t.toString());
> }
> while(t != null);
>
> System.out.println("done"); // never gets here
>
> }
> }
>
>
> I am working on reproducing the grammar that caused the repeated
> emission of the same token. I'll post again when I have failing
> this way again. Anyone know what might be wrong?
>
> Thanks in advance,
>
> Stephen.
>
>
>
More information about the antlr-interest
mailing list