[antlr-interest] strings and vocab?

ronald.petty at milliman.com ronald.petty at milliman.com
Mon Apr 12 15:07:29 PDT 2004


Alright, I give up :(.  What is the secret to Antlr, jk.  I am still 
having some trouble getting started with Antlr, and I believe most of my 
confusion comes from how strings/tokens/vocab is done.

I was reading the java.g grammar and was wonding, in the parser there is 
the rule

builtInType
        :       "void"
        |       "boolean"
        |       "byte"
        ..
        ;

Then in the Lexer there is

IDENT
options { testLiterals=true; }
        : ('a'..'z'|'A'..'Z'|'_'|'$')('a'..'z'|'A'..'Z'|'_'|'0'..'9'|'$')*
        ;

NUM_INT
{boolean isDecimal=false; Token t=null;}
        :       '.' {_ttype=DOT;}
                (       ('0'..'9')+ (EXPONENT)? (f1:FLOAT_SUFFIX {t=f1;})?
                        {
                                ......

protected 
FLOAT_SUFFIX
        :       'f'|'F'|'d'|'D'
        ;


When the parser says, give me next token (nextToken), the Lexer will eat 
the next token based on the Lexer rules.  Now if the string "void" comes 
in, the Lexer says, let me check if there is a literal yet for this token. 
 However I do not see what is going on here.  The word "void" in the 
parser may not have been seen yet (calling builtinType).  I have read teh 
vocab document, but still don't think I understand.  I have tried using 
tokens {} and don't understand why that works.  Could someone explain 
these simple concepts?  I know I am missing something very simple here.  I 
can follow along the grammars just fine, but I don't understand real 
workings on these issues, espically how or where you check Identifiers vs. 
Keywords (I have read a dozen things, and none of them seem to explain it 
in a way I can follow).

Also does protected mean that the Lexer will never call FLOAT_SUFFIX 
directly,if it is trying to get the nextToken, it will only try to get it 
from the FLOAT_SUFFIX call in NUM_INT.  Correct?  Is this to keep similiar 
issues like (IDENT vs Keywords) from happening?

Thanks Ron

ps.  When I get this all figured out, I will write another tutorial 
hopefully documenting the same issues I have, maybe help someone one day 
:)

**************************************************************************************
This communication is intended solely for the addressee and is
confidential. If you are not the intended recipient, any disclosure, 
copying, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Unless indicated
to the contrary: it does not constitute professional advice or 
opinions upon which reliance may be made by the addressee or any
other party, and it should be considered to be a work in progress.
**************************************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20040412/aecc2fa6/attachment.html


More information about the antlr-interest mailing list