[antlr-interest] "everything else" clause

[Ric Klaren]

On Mon, Apr 26, 2004 at 07:24:02PM +0200, Mike Lischke wrote:
> Since the "everything else" part in the parser consists of all tokens not
> mentioned I had to import the lexer vocabulary. This is no big deal as I
> need it anyway for my main parser and now the files are parsed correctly so
> far. The next thing to todo is to make the TokenRewriteEngine work. It
> looks like I have to rewrite the grammar a bit because I cannot address a
> single token in the else_part from, say, the ifdef rule, hence I cannot
> strip out the proper part.

I recall Monty mentioning a little while ago that he did some nifty
overriding on a few parser methods (probably some tricks around match(..)
is my guess) and got it to behave like a tokenfilter/stream. If you go that
way you can just chain the parsers and you won't have to use the
TokenRewriteEngine which is not the most efficient for large input files
(whole file gets stored in memory) I'm not sure if he posted specifics
though.

> Indeed. It seems a bit overkill but requires the least work. The lexer will
> produce the inital token stream and the rewrite engine will spit out the
> modified stream to the main parser. So essentially, the tokenizing is only
> done once (as before). For me it sounds like an effective solution.

Sounds very workable :) (unless the input files are very big)

> Oh, I do. I learned a lot already from the generated code and the examples.
> Coming from a yacc/lex background I'm now addicted to the recursive descent
> parser design. It is so wonderfully debuggable and errors can be found
> quite easily.

Yeah I do like recursive descent as well :)

> I only wonder why there are so few existing grammars. I'd expect many of
> them in such a successful package.

Dunno, there are actually quite a lot I think, although it might be bad
compared to others (not really aware of a big lex/yacc repository with
grammars, also not for some of the other competition (then again I did not
really look)).

Cheers,

Ric
--
-----+++++*****************************************************+++++++++-------
    ---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893755  ----
-----+++++*****************************************************+++++++++-------
 Why don't we just invite them to dinner and massacre them all when they're
  drunk? You heard the man. There's seven hundred thousand of them. Ah? ..
           So it'd have to be something simple with pasta, then.
                 From: Interesting Times by Terry Pratchet



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/