[antlr-interest] Re: Token Stream.

micheal_jor <open.zone at virgin.net> open.zone at virgin.net
Thu Jan 30 06:55:02 PST 2003


--- In antlr-interest at yahoogroups.com, "queengiraffe 
<cow_jumped_moon at h...>" <cow_jumped_moon at h...> wrote:
> > Use a loop similar to:
> > 
> > for(;;)
> > {
> >     RefToken tok = lexer.nextToken();
> > 
> >     if ( tok->getType() == Token::EOF_TYPE )
> >         break;
> > 
> >     cout << "Token: " << tok->toString() << endl;
> > }
> > 
> > You should really wrap this in a try...catch. Also see the some 
of 
> > the projects in the examples/cpp directory.
> > 
> > Micheal
> 
> Thank you very much for your help. I'm afraid I'm still a little 
> stuck, the loop you supplied does give me the type for the tokens, 
> however it seems to get stuck right afterwards and will not move 
onto 
> the parser part.

The loop simply sucks out all the tokens that the lexer finds in it's 
input in the order they were seen. You wanted to see the stream of 
tokens for testing purposes if I remember correctly.

If you want the parser to do it's job, you have to pass the lexer to 
it and let it - the parser - suck the tokens from the lexer. The 
parser drives the process by sucking tokens from a supplied 
Lexer/TokenStream as needed. If you have already done that with the 
for-loop, there would be nothing for the parser to recognize!.

> I am trying to apdapt the calc program supplied in examples for 
cpp. 

Calc is a good example to adapt. To adapt Calc's Main.cpp for your 
own Lexer/Parser:
1. Change all occurences of CalcLexer to your lexer's classname.

2. Change all occurences of CalcParser to your parser's classname.

3. Change the starting rule - expr() in this case - on the following 
line to the starting rule for your parser:

		// Parse the input expression
		parser.expr();

4. If you don't have a a TreeParser to test, remove or comment out 
all references to CalcTreeWalker. That would be the following in the 
body of the "if ( t )" statement:

			CalcTreeWalker walker;

			// Traverse the tree created by the parser
			float r = walker.expr(t);
			cout << "value is " << r << endl;

When you compile and run the resulting app, it will take as input 
either stream of chars fed to it's standard input

In the code you posted, the for-loop should be there. Best to start 
with Calc's Main.cpp and follow the steps above.

Micheal



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list