AW: [antlr-interest] Multiplexing question - no lexer token to switch, context necessary

[Maximilian Störzer]

Hi Monty,

Thanks for your hints.

A parser rule won't do the job unfortunately  - never seen so many
nondeterminism warnings, I think the output was longer than the grammar then
;) And you are right - whitespaces are discarded (which might be a reason
for the nondeterminism warnings). But I am not sure if I want to have WS
tokens in the parser.

So I'll try the filter aproach you suggested below.

Thanks again, regards
	Max

-----Ursprüngliche Nachricht-----
Von: Monty Zukowski [mailto:monty at codetransform.com]
Gesendet: Dienstag, 7. September 2004 17:34
An: antlr-interest at yahoogroups.com
Betreff: Re: [antlr-interest] Multiplexing question - no lexer token to
switch, context necessary


On Sep 7, 2004, at 5:15 AM, Maximilian Störzer wrote:

> Hi everybody!
>
> I am stuck with a multiplexing problem. I understand that in general
> it is
> no good idea to switch the lexer from the parser.
>
> However I have the problem that I would need a different lexer
> dependent on
> the context. There is *no* token (like e.g. "/**" for JavaDoc) to allow
> switching the lexer from within the lexer. Only the parser knows about
> this
> necessary context.
>
> The problem is that - depending on context - foo*bar should be parsed
> as
> IDENT STAR IDENT or as WILDCARD_IDENT, but, again, there is no single
> token
> to indicate this state change, only the context (i.e. the current
> parser
> rule the parser sits in).

So do it with a parser rule--
wildcard_ident: IDENT STAR IDENT;

You probably don't like that because then you could recognize foo * bar
as well, assuming you are skipping whitespace like most parsers.

Alternatively see if you can use a filter.  Read my article
http://www.codetransform.com/filterexample.html.  I really have to edit
it to be more to the point, but the gist is to have a filter that is
actually a parser and can start parsing a particular rule if it can,
otherwise pass the token through.  So you would not skip whitespace in
the lexer, instead you would have a filter to do that.  The filter
would also have the rule that required WILDCARD_IDENT so as to change
IDENT STAR IDENT to WILDCARD_IDENT.

You may also be able to have enough context introduced into your lexer
to recognize it.  But without any more specifics it is hard to help.

Monty

ANTLR & Java Consultant -- http://www.codetransform.com
ANSI C/GCC transformation toolkit --
http://www.codetransform.com/gcc.html
Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html




Yahoo! Groups Links








 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/