[antlr-interest] Understanding priorities in lexing (newbie)[Suggestion]

Daniel Brosseau daniel at lba.ca
Sun Jul 15 19:40:12 PDT 2007


I tried this in ANTLRWorks:

grammar lex;
fragment KEYWORD  :  'a' 'b' 'c';
fragment OTHER : 'a'|'b'|'c'|'d';
TOKEN : (KEYWORD)=> KEYWORD { $type = KEYWORD; }
               | OTHER { $type = OTHER; };
token : TOKEN;
program : token*;

With input: "abd", in the interpreter I get:
grammar lex.g
    program
        token
          .....ab
          .....d

With input "abc", I get:
grammar lex
    program
        token
           .....ab
           .....c

With input "abc", in the debugger I get
root
program
org.antlr.runtime.EarlyExitException

and the input "abc" is in a red box in the Input window. The Output window 
had:
line 1:0 required (...)+ loop did not match anything at input 'abc'


None of this seems right. What am I missing?

Daniel

----- Original Message ----- 
From: "Terence Parr" <parrt at cs.usfca.edu>
To: "antlr-interest Interest" <antlr-interest at antlr.org>
Sent: Friday, July 13, 2007 1:28 PM
Subject: Re: [antlr-interest] Understanding priorities in lexing 
(newbie)[Suggestion]


>
> On Jul 13, 2007, at 10:18 AM, Susan Jolly wrote:
>
>> I really appreciate people who take the time to write the simplest 
>> possible
>> grammar that illustrates the problem they are having rather than 
>> embedding
>> huge grammars in their posts.
>
> Hi Susan!  Agreed.
>
>> Here is one way to get v3 to generate the code so the lexer looks  ahead 
>> for
>> the original example.  Is there any reason not to do this?
>>
>> fragment KEYWORD : 'a' 'b' 'c' ;
>> fragment OTHER : 'a' | 'b' | 'c';
>> DUMMY : (  (KEYWORD) => KEYWORD {$type = KEYWORD;}
>>         |  OTHER  {$type = OTHER;}
>>         );
>
> That is a correct grammar.  Note that the following is not really  going 
> to work:
>
> DUMMY
> options {backtrack=true;}
> : (KEYWORD {$type = KEYWORD;}  | OTHER{$type = OTHER;})
> ;
>
> it will not execute the backtracking in this case as it thinks it  knows 
> what it's doing. :(
>
> Ter
> 



More information about the antlr-interest mailing list