[antlr-interest] more newbie help required

Martin Probst mail at martin-probst.com
Wed Feb 15 07:51:42 PST 2006


> Or is it becasue all my grammar is coupled to each other, that the 
> first two tokens in my lexer will consume the whole text by calling 
> other grammar? 

Yes, that's the reason. The 'normal' idea is that you have several lexer
rules and each of them matches a part of the file. Then nextToken() will
return single parts of the file. If you use a parser, the parser would
then depending on the token type descend into different subrules etc.

> Should this be done in the parser for below code to return  
> the composite part tokens of the root expression?
> 
> 	public static void test(String text) throws Exception {
> 		System.out.println("Testing " + text);
> 		MyLexer lexer = new MyLexer(new StringReader(text));
> 		while (true) {
> 			Token token = lexer.nextToken();
> 			System.out.println(token.getType() + "\t" + token.getText());			
> 			if (token.getType() == MyLexerTokenTypes.EOF) {
> 				break;
> 			}
> 		}
> 	}

This is correct.

Martin



More information about the antlr-interest mailing list