[antlr-interest] greedy vs nongreedy lexer rules

[Cliff Hudson]

Ahh, that's what I was getting at.  In my system, the actions would not be
embedded directly in the matching code as they are now.  Instead, all of the
actions which were part of an alternative would be executed together against
the global state.  Consider the following (assuming the C# language target)
based on the example you gave above.  The implementation of the Lexer
subclass would look as follows:

public class MyLexer : Lexer
{
  int i = 0; // from @members

  ... DFA stuff, token decls, etc...
  public void ID()
  {
     // Implemented as normal, except that where {i++} would be, there would
be
     // code which instead saves the action-visible state of the parser into
a State block.
     // This visible state means only the values of those variables which
are generated by the
     // running the the DFA, not those which might be updated by the
actions.  This
     // snapshot is then enqueued.

  }

public void ID_alt1(Queue<State> states)
{
    State currentState = states.Dequeue();
    i++; // While state information was enqueued, it turns out we didn't use
it here.
           // A language target would have translated any references to
action-visible
           // token information such as $text into

}


On Sun, Apr 18, 2010 at 4:53 PM, Terence Parr <parrt at cs.usfca.edu> wrote:

>
> On Apr 18, 2010, at 4:50 PM, Cliff Hudson wrote:
>
> > You wrote:
> >
> > Well, imagine that you are modifying "global" state as you match
> characters in identifier; this is something done in actions that ANTLR can
> analyze. There is no way to "roll this back".
> >
> > The 'global' state you are talking about is state the DFA is modifying
> but which an action in a rule could examine?  Is the amount of state visible
> to actions too much to store while alternatives are being evaulated?
>
> It's state you set up and modify then ref:
>
> @members {
> int i = 0;
> }
>
> ...
>
> ID : ('a'..'z' {i++;})+ {System.out.println(i);} ;
>
> ANTLR can't save i for you each char iteration.
>
> Ter
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>