[antlr-interest] Fwd: Common Prefix question

[Eric]

---------- Forwarded message ----------
From: Eric <researcher0x00 at gmail.com>
Date: Thu, Nov 10, 2011 at 9:55 PM
Subject: Re: [antlr-interest] Common Prefix question
To: Alexander Kaiser <alexander.g.kaiser at gmail.com>




On Mon, Nov 7, 2011 at 3:56 PM, Alexander Kaiser <
alexander.g.kaiser at gmail.com> wrote:

> Hi,
>
> I am new to antlr (antlr3) and struggling with lexing common prefixes.
> My simple grammar:
>
> grammar simple;
>
>
> R : 'R';
> E : 'e';
> D : '1';
> RESULT : 'Result';
>
>
> expr : ( RESULT | R E D )  EOF;
>
>
> On input 'Result' I expect token RESULT, and on input Re1 I expect tokens R
> E D but input 'Re1' only throws NoViableAltException.
> I searched for info 'antlr common prefixes' on the web and tried increasing
> k, lookahead predicates ..., things like
>
> RESULT options{filter=true;} : 'Result';
>
>
>
> , but to no avail. I would be very much obliged if anyone could give me a
> hint.
>

Hi Alexander,

Usually you want to convert yoru input into an AST and make the lexer and
parser as simple as possible. A common mistake of beginners is to try and
modify the lexer or parser into doing work that should be done with the
AST. In this case, the goal is to end of with one of two tokens, either RED
or RESULT. While it may be possible to contrive the lexer and parser into
doing this, it is much easier to do with AST rewrites.

Here is an example.

grammar mail_004;
options
{
 output=AST;
}
tokens
{
 RESULT;
 RED;
}
s : prefix suffix
 EOF;
prefix  : R E
 ;

suffix : D  -> RED
 | SULT  -> RESULT
 ;
R : 'R';
E : 'e';
D : 'd';
SULT : 'sult';

If AST and rewrites are new concepts to you, then you should get a copy of
"The Definitive ANTLR Reference" and read it. You can find some info on the
website, but there is no substitute for the book.


Eric





>
>
>
> --
> Alexander
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>