[antlr-interest] Inserting missing nodes

[Jim Idle]

You need to fix your lexer first:


WORDLIST    : ((PREFIX | WORD | SENTENCE)(','(WS)* (PREFIX | WORD |
> SENTENCE))+);

is ambiguous with:

> PREFIX	    : WORDCHAR+(STAR);
> WORD        : WORDCHAR+(('-'|'+')WORDCHAR*)*;

You need to contstruct the lists in the parser not the lexer and should
probably left factor the common roots in the lexer anyway.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Jean-Sebastien Vachon
> Sent: Wednesday, May 04, 2011 7:51 AM
> To: Bart Kiers; antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Inserting missing nodes
>
> Thanks for your input. So here is the whole thing with two use cases
> that are not giving me the expected results...
> (Sorry for the long post)
>
> INPUT = abc def zyx toto
> RESULT = (DEFAULT_OP abc def) (DEFAULT_OP zyx toto) EXPECTED =
> (DEFAULT_OP (DEFAULT_OP abc def) (DEFAULT_OP zyx toto))
>
> INPUT = software engineer OR java programmer RESULT = (DEFAULT_OP
> software (OR engineer java)) programmer EXPECTED =  (DEFAULT_OP
> (DEFAULT_OP software (OR engineer java)) programmer)
>
> I'm also having some trouble using the Interpreter within Eclipse.
> The same expressions are not working in the interpreter. It fails to
> generate the tree with a "NoViableAltException at input 'abc' " (for
> the first case).
> I don't think this is related to my other problem since I can't get it
> to generate any tree.
>
> Thanks again for your time and comments
>
> -----------------------------------------------------------------------
> -----------------------------------
> Grammar (validation by building a tree and trying to insert missing
> operators)
> -----------------------------------------------------------------------
> -----------------------------------
> grammar MyGrammar;
>
> options {
>   language = Java;
>   output = AST;
>   ASTLabelType = CommonTree;
> }
>
> // Rules to build the tree representation of our expression...
>
> query
>   : and_expr+ EOF!
>   ;
>
> // Each AND expression can contain OR expressions...
> and_expr
>   : (expr expr+) => default_op
>   | (u1=or_expr (AND^ u2=or_expr)*)
>   ;
>
> // A OR expression contains one or more expression
> or_expr
>   : u1=expr (OR^ u2=expr)*
>   ;
>
> default_op
>   : (e1=or_expr e2=or_expr) -> ^(DEFAULT_OP $e1 $e2)
>   ;
>
> expr
>   : (NOT^)? (operand)
>   ;
>
> // The leafs of the tree.. Words, sentence and so on...
> // Note that an expression such as '-word' is rewritten in its 'NOT
> word' form operand
>   : (f=FIELD^)(o=operand)
>   | PREFIX
>   | WORD
>   | SENTENCE
>   | WORDLIST
>   | NEGATIVE(w=PREFIX|w=WORD|w=SENTENCE|w=WORDLIST) -> ^(NOT $w)
>   | MUST
>   | LPAREN! and_expr RPAREN!
>   ;
>
> // Lexer ...
> NEGATIVE    : '-';
> LPAREN      : '(' ;
> RPAREN      : ')' ;
> DOUBLEQUOTE : '"';
> STAR	      : '*';
> AND         : 'AND' | '+';
> OR          : 'OR';
> NOT         : 'NOT';
> DEFAULT_OP  : 'DEF_OP';
> FIELD       : ('title'|'TITLE'|'Title')(FIELDSEPARATOR);
> WS          : (WSCHAR)+ { $channel=HIDDEN; };
> PREFIX	    : WORDCHAR+(STAR);
> WORD        : WORDCHAR+(('-'|'+')WORDCHAR*)*;
> SENTENCE    : ((DOUBLEQUOTE)(~(DOUBLEQUOTE))*(DOUBLEQUOTE));
> WORDLIST    : ((PREFIX | WORD | SENTENCE)(','(WS)* (PREFIX | WORD |
> SENTENCE))+);
> MUST	      : '+'(PREFIX|WORD|SENTENCE|WORDLIST);
> fragment WORDCHAR       : (~( WSCHAR | LPAREN | RPAREN | '-' |':' | '+'
> | ',' | STAR | DOUBLEQUOTE) );
> fragment FIELDSEPARATOR : ':';
> fragment WSCHAR         : ( ' ' | '\t' | '\r' | '\n');
>
>
>
> ================================= END OF GRAMMAR
> ==========================
>
>
>
>
>
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Bart Kiers
> Sent: May-04-11 10:21 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Inserting missing nodes
>
> On Wed, May 4, 2011 at 4:12 PM, Jean-Sebastien Vachon < jean-
> sebastien.vachon at wantedtech.com> wrote:
>
> > No one can help me with this? :S
> > Let me know if something is not clear. I need to fix this issue as
> > soon as I can.
> >
> > Thanks
>
>
> The fact that you didn't provide the lexer rules (although they might
> be straight-forward as you mentioned), and you didn't mention what
> input you're specifically having problems with parsing (the following
> is a bit
> vague: *"... but I can't get it to parse everything I'm throwing at it
> ..."*), might be some reasons why you haven't been answered.
>
> Regards,
>
> Bart.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address