[antlr-interest] Re: keywords appended to identifiers, sugges tion s?

mzukowski at yci.com mzukowski at yci.com
Thu Jun 12 14:12:05 PDT 2003


That looks good to me.  What language are you parsing, anyway?

Monty

-----Original Message-----
From: gdave [mailto:dave.green at valley.net] 
Sent: Thursday, June 12, 2003 1:34 PM
To: antlr-interest at yahoogroups.com
Subject: [antlr-interest] Re: keywords appended to identifiers, sugges tion
s?


> Personally I like the tokenstream approach.  There's no reason why
you can't
> expose the keywords table through a lexer method and use that in
your
> stream.  Then you're not maintaining two tables.

If I read you right, you suggestion is a table/array in a piece of 
java code (which I can embed in the .g file and thus into the lexer) 
which contains my KEYWORD entries.  Then match both keywords and 
names with a single lexer rule GENERIC_WORD which has an action 
attached which sets the token type to NAME or WORD depending on 
whether the text is contained in the table.

just interested in seeing how other people manage these things.  The 
sub-class of the lexer would be ok.  The proxy TokenStream was also 
not desireable becuase one had to first construct a lexer, then wrap 
it in the proxy.  the least room for error the better.  keeping most 
of the data that needs to be in sync in the same file or better yet 
as one copy in the same file seems to be better as well.

by the way, I did stumble my way to a solution via a syntactic 
predicate a few minutes ago...  here it is:

{ 
	//
	// keyPred will test if there is a key NOT followed by a 
character leading forward
	//
    boolean keyPred() {
		boolean result = false;
		int myMark = mark();
		inputState.guessing++;
		try {
			mKEYWORD(false);
			try {
				mCHAR(false);			
	
				result = false;
			}
			catch (Exception pe) {
				result = true;		
			}			
		}
		catch (Exception pe) {
			result = false;			
		}
		rewind(myMark);
		inputState.guessing--;			
		
		return result; 
    }
}

protected
KEYWORD : "FOO" | "BAR";

protected
CHAR : 'a'..'z'| 'A'..'Z';

protected
NAME:   ( CHAR 
          ({!keyPred()}? CHAR)* 
        )
    ;
  
KEY_OR_NAME :
 {keyPred()}? KEYWORD {$setType(KEYWORD);}
 | NAME {$setType(NAME);}
 ;



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list