[antlr-interest] Re: check tokens for whitespace?

[mazypath]

Right.  I mentioned this earlier in the thread.  I also want to
specify an AST for my tokens.  I should be able to do this in the
tokens block as so:
 tokens {
  FUNC_SIN="sin"<AST=my.ast.SinClass>;
  FUNC_SIN="cos"<AST=my.ast.CosClass>;
  FUNC_SIN="tan"<AST=my.ast.TanClass>;
 }

As I mentioned before, there seems to be a bug.  ANTLR generates java
code that looks like this:
  
  protected void buildTokenTypeASTClassMap(){
    tokenTypeToASTClassMap.put(new Integer("sin"), my.ast.SinClass);
    tokenTypeToASTClassMap.put(new Integer("cos"), my.ast.CosClass);
    tokenTypeToASTClassMap.put(new Integer("tan"), my.ast.TanClass);
  }

Which of course causes an exception when java attempts to make an
integer out of the strings "sin," "cos," and "tan."

Am I doing something wronge or is ANTLR messed up?

In the meantime your suggestion works well enough.  


--- In antlr-interest at yahoogroups.com, "Bryan Ewbank" <ewbank at s...> wrote:
> If I understand what you want to do, you can use tokens instead of some
> external
> command; look for information on the tokens section, and the
testLiterals
> option.
> 
> A quick sketch looks like this:
> 
> 	class scanner extends Lexer;
> 	options {
> 		// do not, by default, attempt to detect matches against literals.
> 		testLiterals = false;
> 	}
> 	// define tokens for every keyword or function call...
> 	tokens {
> 		FUNC_SIN = "sin";
> 		FUNC_TAN = "tan";
> 		FUNC_COS = "cos";
> 	}
> 	...
> 
> 	VAR
> 	options {
> 		// do test against the literals table here.  this means that if any of
> 		// the literals shown in the tokens table (or anywhere else), that
will
> 		// be returned as the type, rather than reporting VAR.
> 		testLiterals = true;
> 	}
> 		: ('a'..'z') ('a'..'z'|'0'..'9')*
> 
> There's no semantic action required - the "testLiterals" option does
it for
> you rather than you needing to do it yourself.
> 
> > -----Original Message-----
> > I think the best way would be if I could use literals (with an AST
> > option) in the token definition of the lexer but this a great work
around.





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/