[antlr-interest] Re: check tokens for whitespace?

[Bryan Ewbank]

If I understand what you want to do, you can use tokens instead of some
external
command; look for information on the tokens section, and the testLiterals
option.

A quick sketch looks like this:

	class scanner extends Lexer;
	options {
		// do not, by default, attempt to detect matches against literals.
		testLiterals = false;
	}
	// define tokens for every keyword or function call...
	tokens {
		FUNC_SIN = "sin";
		FUNC_TAN = "tan";
		FUNC_COS = "cos";
	}
	...

	VAR
	options {
		// do test against the literals table here.  this means that if any of
		// the literals shown in the tokens table (or anywhere else), that will
		// be returned as the type, rather than reporting VAR.
		testLiterals = true;
	}
		: ('a'..'z') ('a'..'z'|'0'..'9')*

There's no semantic action required - the "testLiterals" option does it for
you rather than you needing to do it yourself.

> -----Original Message-----
> I think the best way would be if I could use literals (with an AST
> option) in the token definition of the lexer but this a great work around.



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/