[antlr-interest] Multipass parsing
FranklinChen at cmu.edu
FranklinChen at cmu.edu
Sun Mar 28 17:41:31 PST 2004
> > - As my toy example tries to indicate, "-" can mean three different
> > things, depending on the context
>
> That's what parsers and tree grammars are good at.
>
> > The whitespace first pass is important, because I want to pass
> > grape[fruit] completely differently from grape [fruit]
>
> FWIW, that's a *really* bad idea in a language.
It's also not under my control. I am given a language, I need to
parse it, so I'm interested in ideas on how to do it with ANTLR, if
possible. I didn't design this language.
> > I want to be able to deal with unexpected input at fine granularity and
> > recovery, to minimize parse failures, e.g., if I see 1+2* is 7 I want to
> > not go below the "word" level in the first "token" "1+2*" because I don't
> > want to try to parse an expression and fail.
>
> As I noted, you can write a very permissive lexer and a permissive parser
> and then do whatever specific checks via multiple tree grammars.
>
> Have fun,
> John
Context-sensitivity, including the whitespace issue, makes this
difficult. For example:
grape[fruit]
should be differently parsed from
grape [C { int i; }]
and differently parsed from
grape [Perl { my $i = 3; }]
I need multiple lexers, not just one permissive lexer.
Now, I do know how to multiplex lexers dynamically, and have done so
in the past in a different task in constrained circumstances (such as
the Java/JavaDoc example), but the task here seems to require more
than that.
I think that where I am stuck is that ANTLR allows me to take a text
stream to a token stream by means of a lexer, then a token stream to a
tree by means of a parser, and a tree to something else by means of a
tree parser, but what I want is to turn a token i for language A
(permissive) into a token stream for language B_i, and then have all
the B_i conconcatened into a new token stream for language C.
--
Franklin
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list