[antlr-interest] Creating fake tokens

[Monty Zukowski]

See the recent thread "Re: [antlr-interest] Embedded token stream 
technique".  The basic idea is to manually subclass the lexer and 
override nextToken() to consult a queue of tokens which your lexer 
maintaines.

Another option is to use a parser filter.  See my article "Using a 
Parser as a TokenFilter" 
http://www.codetransform.com/filterexample.html.

Monty

On May 4, 2004, at 11:38 PM, Brad Schick wrote:

> Is there a clean way to create fake tokens from the lexer? I am
> working on a language that has an "elseif" keyword, and a nice
> solution would be to replace the ELSEIF token with tokens ELSE and
> IF. I looked at the lexer docs and only saw how to replace the
> default token with another.
>
> If not from the lexer, is it possible to push fake tokens back into
> the token stream from the parser? Scanning some antlr code seemed to
> indicate this isn't possible.
>
> Thanks,
> Brad
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
Monty Zukowski

ANTLR & Java Consultant -- http://www.codetransform.com
ANSI C/GCC transformation toolkit -- 
http://www.codetransform.com/gcc.html
Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/