[antlr-interest] Multipass parsing

FranklinChen at cmu.edu FranklinChen at cmu.edu
Sun Mar 28 17:41:31 PST 2004


> > - As my toy example tries to indicate, "-" can mean three different
> > things, depending on the context
> 
> That's what parsers and tree grammars are good at.
>
> > The whitespace first pass is important, because I want to pass
> > grape[fruit] completely differently from grape [fruit]
> 
> FWIW, that's a *really* bad idea in a language.

It's also not under my control.  I am given a language, I need to
parse it, so I'm interested in ideas on how to do it with ANTLR, if
possible.  I didn't design this language.
 
> > I want to be able to deal with unexpected input at fine granularity and
> > recovery, to minimize parse failures, e.g., if I see 1+2* is 7 I want to
> > not go below the "word" level in the first "token" "1+2*" because I don't
> > want to try to parse an expression and fail.
> 
> As I noted, you can write a very permissive lexer and a permissive parser
> and then do whatever specific checks via multiple tree grammars.
> 
> Have fun,
> 	John

Context-sensitivity, including the whitespace issue, makes this
difficult.  For example:
       grape[fruit]
should be differently parsed from
       grape [C { int i; }]
and differently parsed from
       grape [Perl { my $i = 3; }]

I need multiple lexers, not just one permissive lexer.

Now, I do know how to multiplex lexers dynamically, and have done so
in the past in a different task in constrained circumstances (such as
the Java/JavaDoc example), but the task here seems to require more
than that.

I think that where I am stuck is that ANTLR allows me to take a text
stream to a token stream by means of a lexer, then a token stream to a
tree by means of a parser, and a tree to something else by means of a
tree parser, but what I want is to turn a token i for language A
(permissive) into a token stream for language B_i, and then have all
the B_i conconcatened into a new token stream for language C.

-- 
Franklin


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list