[antlr-interest] greedy vs nongreedy lexer rules

[Terence Parr]

Hi Cliff, thanks for the input.  I think it might be hard to record complete "state of the lexer" for each input position efficiently.  Users could, for example, update a large global data structure as they lexed.

Hmm...yeah, i was trying this idea earlier but we sort of need to formalize arguments to parser rules to handle predicates that get generated outside of the defining function (when I need to gen cyclic DFA).  THis happens for Java as it has no goto. might as well do locals too.

T

On Apr 18, 2010, at 4:04 PM, Cliff Hudson wrote:

> With respect to local variables and actions in ambiguous sets of rule, it seems to me that the entire rule alternative is the scope for all actions which appear in it, so having an action which declares a variable and then another action later in the alternative which executes some code is really all one method.  What would need to be dealt with is that the language target generator would need to be able to take the state pulled from the DFA and insert that information into the alternative's action sequence so that each action had access to the logical state at the time it executes.
> 
> For instance, in the rule:
> 
> FOO: { int n=4; } 'a'* { n += $text.Length; } 'bcd' { System.WriteLine("{0}: {1}", n, $text); } ;
> 
> the alternative's action function would look like:
> 
> foo_alt1(State[] states)
> {
>     int n=4;
>     n += states[0].Text.Length;
>     System.WriteLine("{0}: {1}", n, states[1].Text);
> }
> 
> The State[] is an output from the DFA.  Ambiguity then doesn't have any effect on your ability to execute actions, but language targets would need to be rewritten.