[antlr-interest] Re: modifying expression grammar ; unexpected token null

[Monty Zukowski]

Try building with -traceParser or -traceLexer or -traceTreeParser, 
depending on what you want to see detail for.  That should help you 
step through the code and see why it is giving you that error.

Monty

ANTLR & Java Consultant -- http://www.codetransform.com
ANSI C/GCC transformation toolkit -- 
http://www.codetransform.com/gcc.html
Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html

On Aug 4, 2004, at 9:18 PM, tjs wrote:

> I still get a null token error
> I tried to redefine the grammar like this, using only sumExpr as the
> root, but it still seems to try to read more input than it is given:
>
> class ExpressionParser extends Parser;
> options { buildAST=true; }
>
> //expr     : (LPAREN^ sumExpr RPAREN!) ;
> sumExpr  : prodExpr ((PLUS^|MINUS^) prodExpr)* ;
> prodExpr : powExpr ((MUL^|DIV^|MOD^) powExpr)* ;
> powExpr  : atom (POW^ atom)? ;
> atom     : (LPAREN^ sumExpr RPAREN!) | INT;
>
> class ExpressionLexer extends Lexer;
>
> LPAREN : '(';
> RPAREN : ')';
> PLUS  : '+' ;
> MINUS : '-' ;
> MUL   : '*' ;
> DIV   : '/' ;
> MOD   : '%' ;
> POW   : '^' ;
> SEMI  : ';' ;
> protected DIGIT : '0'..'9' ;
> INT   : (DIGIT)+ ;
>
>
> {import java.lang.Math;}
> class ExpressionTreeWalker extends TreeParser;
>
> sumExpr returns [double r]
>   { double a,b; r=0; }
>
>   : #(PLUS  a=sumExpr b=sumExpr)  { r=a+b; }
>   | #(MINUS a=sumExpr b=sumExpr)  { r=a-b; }
>   | #(MUL   a=sumExpr b=sumExpr)  { r=a*b; }
>   | #(DIV   a=sumExpr b=sumExpr)  { r=a/b; }
>   | #(MOD   a=sumExpr b=sumExpr)  { r=a%b; }
>   | #(POW   a=sumExpr b=sumExpr)  { r=Math.pow(a,b); }
>   | #(LPAREN a=sumExpr) { r=a;}
>   | i:INT { r=(double)Integer.parseInt(i.getText()); }
>   ;
>
>
>
>
>
> --- In antlr-interest at yahoogroups.com, "tjs" <tjs_tjs4 at y...> wrote:
>> I am trying to modify the Expression grammar given in the
>> JS Mill's tutorial (as of this time down;
>> http://supportweb.cs.bham.ac.uk/documentation/tuto
>> rials/docsystem/build/tutorials/antlr/antlrhome.html)
>>
>> I have 1 main goal here:
>> To make it so you don't need parantheses around the top expression
>> I did this by recrusively defining an expression as the same as in the
>> tutorial (fairly straightforward), and instead of
>>
>> | expr
>>
>> on the atom rule, I used
>>
>> | (LPAREN^ expr RPAREN!)
>>
>> Which is the way the Java grammars I've look at handle expressions.
>> All this grammar does is parse math expressions and enforce
>> parantheses (for example, 2+2 outputs 4.0, and (1-3)*4 outputs -8, or
>> it SHOULD anyway). Again, I modified the ORIGINAL grammar.
>>
>> My PROBLEM is that when I feed it input (in the form of a text file
>> that is only 3 characters in length, it gives me the following error:
>>
>> line 1:4 unexpected token: null
>>  ( + 2 )
>> (AST): unexpected end of subtree
>>
>> My interpretation of this is that it is trying to read the 4th
>> character of the input (which doesn't exist, hence null) and then the
>> AST doesn't understand why it can't find the end of the subtree, it
>> apparently also thinks it needs another token when it shouldn't. I
>> can't figure out why it wants a 4th character or any other problem.
>>
>> The following are 1. the input text 2. the grammar 3. the ORIGINAL
>> grammar ( which does work properly, but you need to do (2+2) instead
>> of 2+2, because it makes you use parantheses around everything ), 4.
>> the way in which I load the text file into the lexer
>>
>> finally, before you think i am a "Newbie" who doesn't deserve any
>> help, i have read 50+ pages of ANTLR documentation, searched
>> everything in /docs and this forum, experimented with this grammar for
>> over 4 hours, looked at Java grammars to see how they handled
>> expressions, and wrote a very crude expression parser that was very
>> messy before to graph functions (its messiness is part of the reason i
>> am using ANTLR to redo it). i just can't figure out my dumb problem,
>> that's all. also, if you help me i'll give you credit in my program
>> for correcting my dumb mistakes!
>>
>> ------- 1. input text
>> 2+2
>>
>> (this is only 3 characters, no newlines, or anything else)
>>
>> ------- 2. my expression grammar
>>
>> class ExpressionParser extends Parser;
>> options { buildAST=true; }
>>
>> expr     :  sumExpr ;
>> sumExpr  : prodExpr ((PLUS^|MINUS^) prodExpr)* ;
>> prodExpr : powExpr ((MUL^|DIV^|MOD^) powExpr)* ;
>> powExpr  : atom (POW^ atom)? ;
>> atom     : INT | (LPAREN^ expr RPAREN!) ;
>>
>> class ExpressionLexer extends Lexer;
>>
>> LPAREN : '(';
>> FUNCSIN : 's';
>> RPAREN : ')';
>> PLUS  : '+' ;
>> MINUS : '-' ;
>> MUL   : '*' ;
>> DIV   : '/' ;
>> MOD   : '%' ;
>> POW   : '^' ;
>> SEMI  : ';' ;
>> protected DIGIT : '0'..'9' ;
>> INT   : (DIGIT)+ ;
>>
>> {import java.lang.Math;}
>> class ExpressionTreeWalker extends TreeParser;
>>
>> expr returns [double r]
>>   { double a,b; r=0; }
>>
>>   : #(PLUS  a=expr b=expr)  { r=a+b; }
>>   | #(MINUS a=expr b=expr)  { r=a-b; }
>>   | #(MUL   a=expr b=expr)  { r=a*b; }
>>   | #(DIV   a=expr b=expr)  { r=a/b; }
>>   | #(MOD   a=expr b=expr)  { r=a%b; }
>>   | #(POW   a=expr b=expr)  { r=Math.pow(a,b); }
>>   | #(LPAREN a=expr) { r=a;}
>>   | i:INT { r=(double)Integer.parseInt(i.getText()); }
>>   ;
>>
>> ------ 3. the original grammar from Mill's tutorial (working, but noly
>> when the input is (2+2)  not 2+2 )
>>
>> class ExpressionParser extends Parser;
>> options { buildAST=true; }
>>
>> expr     : (LPAREN^ sumExpr RPAREN!) ;
>> sumExpr  : prodExpr ((PLUS^|MINUS^) prodExpr)* ;
>> prodExpr : powExpr ((MUL^|DIV^|MOD^) powExpr)* ;
>> powExpr  : atom (POW^ atom)? ;
>> atom     : INT | expr ;
>>
>> class ExpressionLexer extends Lexer;
>>
>> LPAREN : '(';
>> RPAREN : ')';
>> PLUS  : '+' ;
>> MINUS : '-' ;
>> MUL   : '*' ;
>> DIV   : '/' ;
>> MOD   : '%' ;
>> POW   : '^' ;
>> SEMI  : ';' ;
>> protected DIGIT : '0'..'9' ;
>> INT   : (DIGIT)+ ;
>>
>>
>> {import java.lang.Math;}
>> class ExpressionTreeWalker extends TreeParser;
>>
>> expr returns [double r]
>>   { double a,b; r=0; }
>>
>>   : #(PLUS  a=expr b=expr)  { r=a+b; }
>>   | #(MINUS a=expr b=expr)  { r=a-b; }
>>   | #(MUL   a=expr b=expr)  { r=a*b; }
>>   | #(DIV   a=expr b=expr)  { r=a/b; }
>>   | #(MOD   a=expr b=expr)  { r=a%b; }
>>   | #(POW   a=expr b=expr)  { r=Math.pow(a,b); }
>>   | #(LPAREN a=expr) { r=a;}
>>   | i:INT { r=(double)Integer.parseInt(i.getText()); }
>>   ;
>>
>> ---- 4. excerpt from my test program
>>
>> ExpressionLexer lexer = new ExpressionLexer(new DataInputStream(
>> new FileInputStream("C:\\antlr\\EXPRINPUT.txt")));
>> ExpressionParser parser = new ExpressionParser(lexer);
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/