[antlr-interest] Wrapping my head around ANTLR v3

[Johannes Luber]

Matt Holmes wrote:
> First off, let me say hello. This is my first message to the list. I
> recently discovered ANTLR as I was looking for tools to help my
> implement a scripting language in a C#/.NET project I am working on. I
> started writing my lexer and parser with v2, when I realized that v3 was
> available at seemed stable enough to use.

The C# language target for ANTLR 3 is not working yet, so switching over
should be postponed.

> The first thing I noticed is that the lexer and parser in all of the v3
> examples seemed to be combined in a single grammar file now. Is this a
> required situation, or can I split them across multiple grammar files to
> keep the grammar clean and easy to read? Is there any particular reason
> to keep them combined? Perhaps I am too "attached to the old school",
> the ole' yacc/lex days when you not only put them in separate files, you
> used separate tools, hehe. Perhaps I am over analyzing this and keeping
> them in a single grammar file is not that big an issue, I am just
> curious if it's still possible.

As I don't know ANTLR 2 I can't answer this question. It is possible to
create multipass grammars, which reside in separate files, but those may
not be the same thing.

> Also, I noticed that in many grammars, tokens are no longer defined
> (such as tokens { FOR="for"; WHILE="while"; } ), but instead single
> parser rules are used to define those tokens (e.g. FOR: 'for'; ). Is
> there a technical reason behind this? Is it simply a conscious choice by
> the developer, or is something gained by using this method?

The tokens { ... } is still working and useful in situations where one
wants to define the keywords of the programming language. It isn't
required to define everything at this place because you may not need it
otherwise or it is easier to use and to read a short string or character
in the rule.

> Apologies if this are extremely novice questions, I am just trying to
> understand some of the things that seem "odd" to me as a past user of
> older lexical tools.

If you want to use ANTLR 3 in a more intense way than now I suggest to
buy the Definitive Guide at
http://www.pragmaticprogrammer.com/titles/tpantlr/index.html
. Definitely worth the price and able to answer most of your questions. :)

Best regards,
Johannes Luber