[antlr-interest] Inserting missing nodes

Jim Idle jimi at temporal-wave.com
Wed May 4 10:05:47 PDT 2011


Don't use the interpreter, use the debugger.

Jim

> -----Original Message-----
> From: Jean-Sebastien Vachon [mailto:jean-
> sebastien.vachon at wantedtech.com]
> Sent: Wednesday, May 04, 2011 9:33 AM
> To: Jim Idle; antlr-interest at antlr.org
> Subject: RE: [antlr-interest] Inserting missing nodes
>
> Ok I've changed my lexer and parser as you suggested but it didn't
> help. However, I found why the interpreter Is not able to generate the
> tree in Eclipse. I found the cause but not the explanation...
>
> It has something to do with the definition of the and_expr rule
> and_expr
>   : (u1=or_expr (AND^ u2=or_expr)*) {System.out.println("  *and_expr: "
> + $u1.text + ", " + $u2.text);}
>   | (expr expr+) => default_op
>   ;
>
> If I remove the second alternative then the interpreter is able to
> create the tree for my expression but I'm losing the operators that
> were inserted by the second alternative. I don't understand why it is
> complaining about a viable alternative not being found for a simple
> input such as 'abc AND def'. It should match the first alternative
> since both 'abc' and 'def' match the or_expr rule (through the expr
> rule). [ I tried changing the order of the two altervatives but it
> didn't help]
>
> Any idea?
>
>
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Jim Idle
> Sent: May-04-11 11:22 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Inserting missing nodes
>
> You need to fix your lexer first:
>
>
> WORDLIST    : ((PREFIX | WORD | SENTENCE)(','(WS)* (PREFIX | WORD |
> > SENTENCE))+);
>
> is ambiguous with:
>
> > PREFIX	    : WORDCHAR+(STAR);
> > WORD        : WORDCHAR+(('-'|'+')WORDCHAR*)*;
>
> You need to contstruct the lists in the parser not the lexer and should
> probably left factor the common roots in the lexer anyway.
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Jean-Sebastien Vachon
> > Sent: Wednesday, May 04, 2011 7:51 AM
> > To: Bart Kiers; antlr-interest at antlr.org
> > Subject: Re: [antlr-interest] Inserting missing nodes
> >
> > Thanks for your input. So here is the whole thing with two use cases
> > that are not giving me the expected results...
> > (Sorry for the long post)
> >
> > INPUT = abc def zyx toto
> > RESULT = (DEFAULT_OP abc def) (DEFAULT_OP zyx toto) EXPECTED =
> > (DEFAULT_OP (DEFAULT_OP abc def) (DEFAULT_OP zyx toto))
> >
> > INPUT = software engineer OR java programmer RESULT = (DEFAULT_OP
> > software (OR engineer java)) programmer EXPECTED =  (DEFAULT_OP
> > (DEFAULT_OP software (OR engineer java)) programmer)
> >
> > I'm also having some trouble using the Interpreter within Eclipse.
> > The same expressions are not working in the interpreter. It fails to
> > generate the tree with a "NoViableAltException at input 'abc' " (for
> > the first case).
> > I don't think this is related to my other problem since I can't get
> it
> > to generate any tree.
> >
> > Thanks again for your time and comments
> >
> > ---------------------------------------------------------------------
> -
> > -
> > -----------------------------------
> > Grammar (validation by building a tree and trying to insert missing
> > operators)
> > ---------------------------------------------------------------------
> -
> > -
> > -----------------------------------
> > grammar MyGrammar;
> >
> > options {
> >   language = Java;
> >   output = AST;
> >   ASTLabelType = CommonTree;
> > }
> >
> > // Rules to build the tree representation of our expression...
> >
> > query
> >   : and_expr+ EOF!
> >   ;
> >
> > // Each AND expression can contain OR expressions...
> > and_expr
> >   : (expr expr+) => default_op
> >   | (u1=or_expr (AND^ u2=or_expr)*)
> >   ;
> >
> > // A OR expression contains one or more expression or_expr
> >   : u1=expr (OR^ u2=expr)*
> >   ;
> >
> > default_op
> >   : (e1=or_expr e2=or_expr) -> ^(DEFAULT_OP $e1 $e2)
> >   ;
> >
> > expr
> >   : (NOT^)? (operand)
> >   ;
> >
> > // The leafs of the tree.. Words, sentence and so on...
> > // Note that an expression such as '-word' is rewritten in its 'NOT
> > word' form operand
> >   : (f=FIELD^)(o=operand)
> >   | PREFIX
> >   | WORD
> >   | SENTENCE
> >   | WORDLIST
> >   | NEGATIVE(w=PREFIX|w=WORD|w=SENTENCE|w=WORDLIST) -> ^(NOT $w)
> >   | MUST
> >   | LPAREN! and_expr RPAREN!
> >   ;
> >
> > // Lexer ...
> > NEGATIVE    : '-';
> > LPAREN      : '(' ;
> > RPAREN      : ')' ;
> > DOUBLEQUOTE : '"';
> > STAR	      : '*';
> > AND         : 'AND' | '+';
> > OR          : 'OR';
> > NOT         : 'NOT';
> > DEFAULT_OP  : 'DEF_OP';
> > FIELD       : ('title'|'TITLE'|'Title')(FIELDSEPARATOR);
> > WS          : (WSCHAR)+ { $channel=HIDDEN; };
> > PREFIX	    : WORDCHAR+(STAR);
> > WORD        : WORDCHAR+(('-'|'+')WORDCHAR*)*;
> > SENTENCE    : ((DOUBLEQUOTE)(~(DOUBLEQUOTE))*(DOUBLEQUOTE));
> > WORDLIST    : ((PREFIX | WORD | SENTENCE)(','(WS)* (PREFIX | WORD |
> > SENTENCE))+);
> > MUST	      : '+'(PREFIX|WORD|SENTENCE|WORDLIST);
> > fragment WORDCHAR       : (~( WSCHAR | LPAREN | RPAREN | '-' |':' |
> '+'
> > | ',' | STAR | DOUBLEQUOTE) );
> > fragment FIELDSEPARATOR : ':';
> > fragment WSCHAR         : ( ' ' | '\t' | '\r' | '\n');
> >
> >
> >
> > ================================= END OF GRAMMAR
> > ==========================
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Bart Kiers
> > Sent: May-04-11 10:21 AM
> > To: antlr-interest at antlr.org
> > Subject: Re: [antlr-interest] Inserting missing nodes
> >
> > On Wed, May 4, 2011 at 4:12 PM, Jean-Sebastien Vachon < jean-
> > sebastien.vachon at wantedtech.com> wrote:
> >
> > > No one can help me with this? :S
> > > Let me know if something is not clear. I need to fix this issue as
> > > soon as I can.
> > >
> > > Thanks
> >
> >
> > The fact that you didn't provide the lexer rules (although they might
> > be straight-forward as you mentioned), and you didn't mention what
> > input you're specifically having problems with parsing (the following
> > is a bit
> > vague: *"... but I can't get it to parse everything I'm throwing at
> it
> > ..."*), might be some reasons why you haven't been answered.
> >
> > Regards,
> >
> > Bart.
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-
> interest/your-
> > email-address
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-
> interest/your-
> > email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address


More information about the antlr-interest mailing list