[antlr-interest] One weird, one (hopefully) simple problem ... from a newbie

Robert Colquhoun rjc at trump.net.au
Fri Jan 24 14:30:31 PST 2003


Hello Anthony,

Since this looks like a strangely mutated version of something i wrote.... ;-)

In your lexer you are manually defining the WHILE token, in your parser you 
are trying to match the WLE token which doesn't appear to be defined 
anywhere.  Maybe a typo?

A better way for simple cases is in your parser define "WHILE", "UNTIL" etc 
literally in your parser rules and then in your lexer import the parser 
vocabulary and define "testLiterals" define in lexer IDENTIFIER rule.

ie
Lexer:

IDENT
         options {
                 testLiterals=true;
         }
     : (('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|'_'|'0'..'9'|'.'|'$'|'%')*)

Parser:

while: "WHILE"
until: "UNTIL"

...and automatically there will be LITERAL_WHILE and LITERAL_UNTIL created 
by the lexer for use in parsers and tree walkers.

Because of evilness within the language(ARev/Pick BASIC) ie statements and 
variables clashed, both could be called "PRINT", "LOOP" etc the above 
didn't work, i had to do something more complicated in the lexer, lexer 
states to get the thing working.

PS For anyone interested the original grammar is 
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/maverick/maverick/ 
src/org/maverickdbms/tools/BASIC.g?rev=HEAD&content-type=text/plain

  - Robert

At 04:06 PM 24/01/2003 +0000, Anthony W. Youngman wrote:

>I've got a minimal grammar attached, and am having two problems...
>
>When I feed the code (at the bottom) into the grammar (below) it works 
>fine. Prints out loads of trace stuff right the way down. But if I change 
>UNTIL to WHILE, it only appears to get as far as the LOOP. Having hit this 
>sort of thing earlier, is my "while" code clashing with a reserved word in 
>java/antlr? I don't really know either well enough. And I'm not getting 
>any error messages - it's just that my trace appears to work over the 
>entire file with "until", but stop as it hits "while". That's my weird problem.
>
>And as for the simple problem? How do I get the parser to print 
>"MULTIPLY", "DIVIDE" or whatever - by the way, it's got to be postfix.
>
>Bear in mind I'm an experienced programmer, but I've never used OO stuff 
>like Java or C++ in anger (or even hardly at all...).
>
>Cheers,
>Wol
>
>-----------------------------------------------------------
>
>class BASICLexer extends Lexer;
>
>options {
>         exportVocab=BASIC;
>         k=2;
>}
>
>tokens {
>         LABEL;
>         LOOP; WHILE; UNTIL; REPEAT;
>}
>
>{
>         // Are we in a statement or expression?
>         private final int STATEMENT = 0;
>         private final int EXPRESSION = 1;
>         int state = STATEMENT;
>}
>
>WS                      : ( ' '|'\t') { $setType(Token.SKIP); };
>EOL             : ( "\r\n"|'\r'|'\n' ) { newline(); state = STATEMENT; };
>
>protected ALPHA : ('a'..'z'|'A'..'Z') ;
>protected NUMERIC : ('0'..'9') ;
>IDENT
>         : ( ALPHA ( ALPHA|NUMERIC|'.'|'$'|'%')* )
>                 {
>                         if (state == STATEMENT) {
>             if (LA(1) == ':' ) {
>                                         int len=text.length();
>                                         consume();
>                                         text.setLength(len);
>                                         _ttype = LABEL;
>                                 } else {
>                                         // more stuff here
>                                         String txt = $getText.toUpperCase();
>                                         // cant use switch here - it 
> takes int. bummer
>                                         if (txt.equals("LOOP")) { _ttype 
> = LOOP;}
>                                         else if (txt.equals("WHILE")) { 
> _ttype = WHILE;}
>                                         else if (txt.equals("UNTIL")) { 
> _ttype = UNTIL;}
>                                         else if (txt.equals("REPEAT")) { 
> _ttype = REPEAT;};
>                                 }
>                         }
>                 }
>         ;
>
>INT              : (NUMERIC)+ ;
>DECIMAL : ('.') ;
>STRING_LITERAL   : '"' ('A'..'Z'|'a'..'z'|'0'..'9')* '"' ;
>
>PLUS    : ('+');
>MINUS   : ('-');
>DIVIDE  : ('/');
>MULTIPLY        : ('*') {System.out.println("MULTIPLY ");};
>ASSIGN  : ('=');
>
>class BASICParser extends Parser;
>
>options {
>         exportVocab=BASIC;
>         buildAST=true;
>         k=2;
>}
>
>ig    : ( i:INT {System.out.println("Integer "+i);});
>rg              : ( r:REAL {System.out.println("Real "+r);});
>sl    : ( slit:STRING_LITERAL {System.out.println("String "+slit);});
>id    : ( iden:IDENT {System.out.println("Ident "+iden);});
>nl    : ( EOL {System.out.println("New line");});
>lb              : (lbl:LABEL {System.out.println("Label "+lbl);});
>dc              : (dcm:DECIMAL {System.out.println("Decimal "+dcm);});
>lp              : (loop:LOOP {System.out.println("LOOP "+loop);});
>wl              : (wle:WLE {System.out.println("WHILE "+wle);});
>un              : (until:UNTIL {System.out.println("UNTIL "+until);});
>rp              : (repeat:REPEAT {System.out.println("REPEAT "+repeat);});
>
>atom    : ( i:INT {System.out.println("Integer "+i);});
>prodExpr        : atom ((MULTIPLY^|DIVIDE^) atom)* ;
>sumExpr : prodExpr ( (PLUS^|MINUS^) prodExpr)* ;
>expr    : sumExpr ;
>assign  : ( id:IDENT ASSIGN^ expr ) {System.out.println("ASSIGN 
>");System.out.println("INDENT "+id);} ;
>
>entry   :       (sl|ig|id|nl|rg|lb|dc|lp|wl|un|rp|assign)+;
>
>-----------------------------------------------------------------
>
>XYZ:
>B = 4 + 5
>C = 6 * 7
>LOOP UNTIL X = 4
>    A = 3
>REPEAT
>1234
>567.89
>ABCD:
>"abdcddf"
>RETURN
>END


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list