[antlr-interest] Passing information into the parser

Jim Idle jimi at temporal-wave.com
Mon Dec 29 05:19:15 PST 2008


Gokulakannan Somasundaram wrote:
> Hi,
>          I have a requirement, where in before starting the parsing, i 
> need to tell the parser whether the first token is T_FUNCTION / 
> T_TRIGGER(say callType). (i.e) this token is not available as part of 
> the text to be parsed. In Bison, it is just declared with %token 
> T_FUNCTION and %token T_TRIGGER. The scanner makes sure that it emits 
> the callType token before it emits the tokens to be parsed. Although, 
> i am able to come up with some ways to do this, they are too ugly.
>             Can someone provide me a clean way of passing information 
> into the parser and make the scanner emit that as the first token, 
> even before the actual tokens are emitted? I am trying to do this in 
> C-Language target.
I think that it is unlikely that you need to do this as some of the 
reasons that you would do this in bison are not limitations in ANTLR. 
For parsing you only need a token type and then access to the text that 
this represents, then you may or may not require a custom token type for 
an AST node for your tree.

So, you can change token types in the lexer by looking ahead in the 
character stream and making decisions on what you see (which is 
sometimes required), or more likely, just return something generic like 
ID, and let the parser work out what this represents.

You can emit() multiple tokens if you really need to, but the chances 
are fairly slim that you really need to I think. Gavin pointed out where 
you start, but the code needed to implement this is fairly involved and 
I bet if you just rethink your lexer/parser in an 'ANTLR' way, you will 
find you don't need to do it.

Jim


More information about the antlr-interest mailing list