[antlr-interest] Additional tokens not being passed to Create(IToken token) ?

Gavin Lambert antlr at mirality.co.nz
Thu Oct 23 00:16:00 PDT 2008


At 09:47 22/10/2008, Dejas Ninethousand wrote:
>I see the problem now.  I put a breakpoint in the parser and that 
>production was never hit.  The reason is that in my root 
>production namespace_decl is preceded by non-terminal filler:
>
>cs_file    :    filler    namespace_decl LBRACE ((comment)* 
>class_def)* RBRACE;
>
>
>filler    :    ( options {greedy=false;} : . )*;
>
>removing filler causes my case statement breakpoint to be hit.  I 
>must have misunderstood the nature of "greedy=false".  I 
>interpreted that to mean "consume all characters that could not 
>constitute other non terminals".

That's a parser rule, so replace "characters" in your sentence 
with "tokens".  Parser rules don't get to have anything to do with 
characters.

Additionally, "greedy=false" cannot see outside the rule it is 
mentioned in (which makes sense if you think about it, because it 
might be called from multiple rules), so in the snippet above it 
will have no choice but to consume every single token remaining in 
the input stream.

(It does have another option if you enable backtracking, but even 
there I'm not convinced it'd work the way you want it to, and it'd 
definitely have worse performance than doing it properly.)

If you really want to skip everything before the namespace_decl, 
then first of all you could try inlining it (replacing the 
reference to the filler rule with the contents of the filler 
rule).  If that doesn't work, you could try being more specific, 
eg. (~NAMESPACE)*.


Also: you realise that C# files can contain zero or more namespace 
blocks, each of which can contain zero or more class definitions 
(and other things), right?



More information about the antlr-interest mailing list