[antlr-interest] Re: Fixed Length Tokens

[Howard Lin]

Keep in mind that I am a newbie. ;) I think the trick is that if you 
mark something "protected", then lexer won't automatically return it 
as a leave/token.  So, make some public rules 
FIRSTNAME/LASTNAME/ADDRESS...etc, which calls your own private rule 
to consume the char stream up to specified length.  You might have to 
handle whitespaces on your own though...  Heh, I don't know of an 
easy way to specify a length in expression though, so see silly 
solution below.  Another way might be to use the getColumn() in your 
rule (search for getColumn on page 
http://www.antlr.org/doc/lexer.html)  Another idea may be put the 
entire line into the text field of your ID token, build a tree 
walker, and reconstruct the tree with the new nodes.

Good luck,
Howard

protected
DUMMY_CHAR
: ( 'a'..'z' | 'A'..'Z' | '0'..'9' );

// 3 char for bandwidth sake
FIRSTNAME
: t1:DUMMY_CHAR t2:DUMMY_CHAR t3:DUMMY_CHAR
{
  $setText( t1.getText() + t2.getText() + t3.getText() );
}
;

// 3 char for bandwidth sake
LASTNAME
: t1:DUMMY_CHAR t2:DUMMY_CHAR t3:DUMMY_CHAR
{
  $setText( t1.getText() + t2.getText() + t3.getText() );
}
;

RECORDLINE
: "ID" FIRSTNAME LASTNAME EOL
;

--- In antlr-interest at yahoogroups.com, "brb0522" <brb0522 at y...> wrote:
> Any suggestions on how to handle a file of fixed length fields.
> Something like:
> IDfirstname   lastname    address    city   <EOL>
> 
> I'm trying to get that into a tree structure
> ID
> |--firstname
> |--lastname
> |--address
> etc...
> 
> I found an example out here that will parse CHAR by CHAR for some 
> number of characters, but it seems that each character becomes a 
leaf.
> 
> test[int N]
>    {int n=N;}
>    :
>    { n > 1 }? CHAR test[n-1]
>    |
>    { n == 0 }?
>    ;
> CHAR is defined as 'a'..'z' | 'A'..'Z' | '0'..'9'
> 
> Any help or examples on how to handle fixed length fields will be 
> greatly appreciated.
> 
> Thanks
> -Brian


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/