[antlr-interest] Code Generation/Optimization.

[John D. Mitchell]

>>>>> "Bharath" == Bharath S <payasam79 at yahoo.com> writes:
[...]

> One other question: If I had actions associated with tokens in my lexer
> rules, does it have any side effects when compared to having actions in
> parser rules and only tokens in lexer rules? My understanding is that
> semantic actions along with tokens is used to resolve non-determinism.

Basically, the actions in a lexer should be very strictly limited to only
what you need to do to create a token from the input stream (including
whatever meta-information that you need such as location information).

One view of translators that I have is that each phase is sort of like a
net.  The early phases are very coarse and each successive phase is tighter
and tighter.  So, a lexer's job is to take the input stream and make a
sequence of tokens for the parser whose job is to make a tree.


In terms of deciding in which phase to make various (kinds of) decisions, a
great place to start is determining the nature of the (kind of) decision.
I.e., what is the scope of the information that you need to be able to make
the decision.  

For example, look at the lexer rules for dealing with floats and integers
-- that's a little crusty but doable completely in the lexer (for a
language like e.g., Java).  Another example is the distinguishment of
language literals from "ID"s in languages such as Java being doable in the
lexer with the use of the literals lookup table but NOT doable in the
lexers for languages that allow the use of literals to depend upon context.

The same goes for parsers.  Some folks put their grammars through wild
gyrations to (try to) deal with all e.g., ambiguities in the parser when
it's much simpler to just build a reasonable clean and useful tree and
create a tree grammar or three to clean things up.

Hope this helps,
		John


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/