[antlr-interest] new FAQ entry concerning nondeterministic lexer rules

[Terence Parr]

Folks,

Monty suggested I make a faq entry on this subject...hope this helps.

http://www.jguru.com/faq/view.jsp?EID=746286

Ter

On Monday, February 4, 2002, at 07:38  AM, mzukowski at bco.com wrote:

> I think this is covered in the lexer docs but I don't have the reference
> handy, so here's how you combine the two rules which are ambiguous 
> because
> they have the same prefix.  This is a problem for lexers only because 
> of the
> synthesized nextToken() rule:
>
>  END_TOKEN: ( options {greedy=true;} "." (GRAPHIC_CHAR
> {$setType(GRAPHIC_TOKEN);})* );
>
> Monty
>
>> -----Original Message-----
>> From: Michael Labhard [mailto:ince at pacifier.com]
>> Sent: Friday, February 01, 2002 1:05 AM
>> To: antlr-interest at yahoogroups.com
>> Subject: [antlr-interest] distinquishing single from multiple
>> characters
>>
>>
>> Question on Lexer.  The grammar (prolog) has two types of
>> tokens both of
>> which contain the same characters.  Specifically the "." may
>> appear alone as
>> the end token or it may appear together with other characters
>> such as "$&%",
>> etc. as a graphic token.  However, the rules:
>>
>> GRAPHIC_TOKEN: ( options {greedy=true;} "." (GRAPHIC_CHAR)+ );
>>
>> END_TOKEN: ".";
>>
>> are flagged as ambiguous, even though in the graphic token
>> case there must be
>> at least 2 characters.  What is the best way to resolve such
>> an ambiguity?
>> Thanks.
>>
>> -- Michael
>>
>>
>>
>> Your use of Yahoo! Groups is subject to
>> http://docs.yahoo.com/info/terms/
>>
>>
>>
>
>
>
> Your use of Yahoo! Groups is subject to 
> http://docs.yahoo.com/info/terms/
>
>
--
Chief Scientist & Co-founder, http://www.jguru.com
Creator, ANTLR Parser Generator: http://www.antlr.org


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/