[antlr-interest] ANTLR Lexer Contexts

[Keith Whittingham]

Hi all,

I posted a problem earlier thanks to those that responded.

I find it a little bit difficult to use ANTLR when the lexer needs (lots  
of) different contexts. Here's a prototype of what I think is a readable  
solution. The language is required to recognise "a[[1..2]". Without the  
context of 'brackets' it gets confused by "1", ".." amd "2" as it  
recognises them wrongly as names.

Hope you find it useful...

-------------
grammar Test;

tokens {
	NAME;
	POSINT;
	RANGE;
	OSB;
	CSB;
}

@lexer::members {
	// set up the different contexts
	private final static int NORMAL		= 0;
	private final static int BRACKETS	= 1;
	private int context = NORMAL;
	private int tokenType;
}

tuple	:	NAME OSB POSINT RANGE POSINT CSB;

TOKEN_SET_SELECTOR
	:	{ context == NORMAL }? => NORMAL_TOKEN_SET { $type = tokenType; }
	|	{ context == BRACKETS }? => BRACKETS_TOKEN_SET { $type = tokenType; }
	;

fragment
NORMAL_TOKEN_SET
	:	('a'..'z'|'A'..'Z'|'.'|'_') ('0'..'9'|'A'..'Z'|'a'..'z'|'.'|'_')*  
{ tokenType = NAME; }
	|	'[' { tokenType = OSB; context = BRACKETS; }
	;

fragment
BRACKETS_TOKEN_SET
	:	'0' | ('1'..'9')('0'..'9')*  { tokenType = POSINT; }
	|	'..'  { tokenType = RANGE; }
	|	']'  { tokenType = CSB; context = NORMAL; }
	;
-----------