[antlr-interest] Re: keywords appended to identifiers, sugges tion s?

[gdave]

> Personally I like the tokenstream approach.  There's no reason why 
you can't
> expose the keywords table through a lexer method and use that in 
your
> stream.  Then you're not maintaining two tables.

If I read you right, you suggestion is a table/array in a piece of 
java code (which I can embed in the .g file and thus into the lexer) 
which contains my KEYWORD entries.  Then match both keywords and 
names with a single lexer rule GENERIC_WORD which has an action 
attached which sets the token type to NAME or WORD depending on 
whether the text is contained in the table.

just interested in seeing how other people manage these things.  The 
sub-class of the lexer would be ok.  The proxy TokenStream was also 
not desireable becuase one had to first construct a lexer, then wrap 
it in the proxy.  the least room for error the better.  keeping most 
of the data that needs to be in sync in the same file or better yet 
as one copy in the same file seems to be better as well.

by the way, I did stumble my way to a solution via a syntactic 
predicate a few minutes ago...  here it is:

{ 
	//
	// keyPred will test if there is a key NOT followed by a 
character leading forward
	//
    boolean keyPred() {
		boolean result = false;
		int myMark = mark();
		inputState.guessing++;
		try {
			mKEYWORD(false);
			try {
				mCHAR(false);			
	
				result = false;
			}
			catch (Exception pe) {
				result = true;		
			}			
		}
		catch (Exception pe) {
			result = false;			
		}
		rewind(myMark);
		inputState.guessing--;			
		
		return result; 
    }
}

protected
KEYWORD : "FOO" | "BAR";

protected
CHAR : 'a'..'z'| 'A'..'Z';

protected
NAME:   ( CHAR 
          ({!keyPred()}? CHAR)* 
        )
    ;
  
KEY_OR_NAME :
 {keyPred()}? KEYWORD {$setType(KEYWORD);}
 | NAME {$setType(NAME);}
 ;



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/