[antlr-interest] Accessing parts of token from parser

[FranklinChen at cmu.edu]

I have a language that consists of complicated tokens with internal
substructure.  I would like to access the substructure in Perl-like or
better fashion, and am wondering what the most elegant way to do it is
in ANTLR.

For the sake of concreteness, let me consider a toy example of what I
mean.  Suppose my language had floating point tokens, e.g., "24.43".

It is natural to write a lexer by defining

FLOAT: INT '.' INT
    ;

protected
INT: ('0'..'9')+
    ;

but then in the parser, if I want to get the integer part and the
fractional part of the float, it would be stupid to do

stuff: junk1 f:FLOAT junk2 { // reparse f to get parts, use them
   }
    ;


Instead, a better solution is to rewrite the lexer to do

FLOAT: x:INT '.' y:INT  {
   // squirrel away x and y into lexer variables lx, ly }
    ;

and have the parser do

stuff: junk1 FLOAT { 
  // squirrel away lx, ly into rule variables rx, ry
  } junk2
  { // use rx, ry
  }
    ;

This is kind of involved.  Is there a better way I am missing?

-- 
Franklin


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/