[antlr-interest] question on simple grammar for tick formatt o decimal conversion

Micheal J open.zone at virgin.net
Wed Nov 30 18:47:59 PST 2005


> Thanks Micheal ! that fixed my problem.

You're wecome.

> Another question. In antlr is it possible to turn off a 
> particular literal while parsing the token stream ? For 
> example - if I have a string "43242-3465678", and I define 
> "NUM" as (DIGIT)+, till I hit the "-" sign I would like to 
> get the first part, i.e "43242" as NUM (single token), but 
> after I hit "-", I want to the parser to start treating the 
> right side of the string as single DIGITs and not a complete 
> number i.e I don't want to get token of type NUM anymore. Is 
> it possible to do this ?

Yes. But since ANTLR doesn't (can't?) provide support for that behaviour
directly, it is neither easy nor pretty.

Easiest option: Deal with this in the parser. The lexer returns NUM MINUS
NUM as usual and in you parser rule you can do whatever you need to do on a
digit-by-digit basis by getting the text of the token and processing it
char-by-char:

------ in parser.g ------
numMinusNumRule
   :  NUM 
	(MINUS d:NUM
	{
		String digits = #d.getText();
		// do your digit-by-digit processing here
	}
	)? EOF
-------------------------

Fancy option: Use a smart, filtering TokenStream. It implements TokenStream
so your parser can use it as input. It reads from another TokenStream (your
lexer). When it detects the NUM MINUS NUM sequence, it breaks the last NUM
into a series of DIGIT tokens (one for each char in the NUM). Your parser
can now be written to expect 

	NUM (MINUS (DIGIT)+ )?

Cheers,

Micheal



More information about the antlr-interest mailing list