[antlr-interest] Debugging doesn't work with grammar

[Thomas Brandon]

On 7/7/07, Johannes Luber <jaluber at gmx.de> wrote:
> Thomas Brandon wrote:
> > On 7/6/07, Johannes Luber <jaluber at gmx.de> wrote:
> >>
> >> I didn't get any errors in the console. While looking again to be sure
> >> that I haven't overlooked them, I somehow managed to start debugging of
> >> the Java version. Not sure why it didn't work yesterday. But the
> >> attached grammar isn't still recognized correctly. I end with "root ->
> >> action -> MismatchedTokenException" and an entirely red input pane. What
> >> goes wrong here?
> > I was not able to replicate this error using your previously attached
> > grammar. I found one grammar error and a few tree building errors but
> > after fixing them it parsed the attached grammar without error.
> > The grammar error I found was in the 'rule' rule where I needed to
> > change 'ruleAction+' to 'ruleAction*'. The tree building errors were
> > in ruleScopeSpec where it didn't like the "'scope'" literal reference
> > in the rewrite, changing to "SCOPE" (the token name) fixed that. I
> > also had to change the 'id+' in the rewrite to 'id*' as the 'id+' is
> > optional. Finally I was getting errors in the '-> ^(atom ebnfSuffix?)'
> > rewrite in element as atom was returning empty trees, making the empty
> > rewrites in atom instead return a PLACEHOLDER node fixed those. Then
> > the attached grammar parsed ok. So not sure what's going on at your
> > end. That was with ANTLRWorks 1.0.2, so ANTLR 3.0, so if you are using
> > a newer build you may want to test in the 3.0 release.
>
> I've tested the attached patched grammar (in case I didn't change it
> correctly), but with the same results. I'm using ANTLRworks 1.0.2 and
> ANTLR 3.0, so I don't know what could be wrong on my machine.
Sorry, should have posted changed rules rather than describing changes.
I changed ruleScopeSpec to:
ruleScopeSpec
       :       ( 'scope' ACTION )?
               ( 'scope' id+ ';' )*
               -> ^(SCOPE ACTION? id*)
       ;
i.e. replacing the SCOPE in rewrite not in the matching. This made
ANTLR create a synthetic SCOPE node rather than trying to use one of
the optional 'scope' tokens. Your version:
ruleScopeSpec
       :       ( SCOPE ACTION )?
               ( SCOPE id+ ';' )*
               -> ^(SCOPE ACTION? id*)
       ;
means ANTLR again tries to re-use the SCOPE tokens and so errors when
they aren't present. Might be best to rewrite it so it doesn't rely on
ANTLR's not equating SCOPE and 'scope' but the first version seems to
work.
And it was the empty rewrites in atom not element that were causing
issues. I left element alone. So I had:
element // Simplified AST
       :       id ('='|'+=') rulepart=(atom|block) ebnfSuffix? ->
^($rulepart ebnfSuffix?)
       |   atom ebnfSuffix? -> ^(atom ebnfSuffix?)
   |   ebnf
       |   ACTION ->
       |   SEMPRED '=>'? ->
       |   treeSpec -> // No tree grammars are supported, as those
don't match the actual input 1:1
       ;

// Simplified AST
atom:   range ('^'|'!')? -> PLACEHOLDER
   |   terminal
   |   notSet ('^'|'!')? -> PLACEHOLDER
   |   rr=RULE_REF ARG_ACTION? ('^'|'!')? -> $rr
   ;
With those changes, the most recently attached grammar parses the
input given previously without error.

I was able to fix the lexer issue with the grammar you sent me
off-list by replacing ACTION_STRING_LITERAL with:

fragment
ACTION_STRING_LITERAL
	:	'"' ((ACTION_ESC)=>ACTION_ESC|~'"')* '"'
	;

Note that this does change the matched language, allowing single
character string literals which the previous version did not. However
I think this is correct as single character string literals should be
allowed in actions.

This allowed me to lex fine but I got a RewriteEmptyStreamException on
token rulepart in the element rule.

> >> Regarding remote debugging: I tried it with the C# version wihtout much
> >> success, but not with the Java version, as I don't have parser for that.
> >> I've tested Java.g with the same result as with the attached grammar.
> >> But using the debug option caused it to be truncated. My non-attached
> >> grammar was also truncated, but resulted in a different parse:
> >>
> >> "               -> MismatchedTokenException
> >>  root -> action -> actionScopeName -> parser
> >>                 -> MismatchedTokenException"
> >>
> >> Looking at this I'm not sure if the culprit is the syntactic predicate
> >> code, as you suggested.
> > Sorry, not sure I'm following you here. You mean parsing Java.g from
> > the ANTLR distribution with your ANTLR3ToRelaxNG grammar resulted in
> > the same error as parsing BackslashBugTest below?
>
> Correct.
>
> > And to test that you
> > used ANTLRWorks debug option which had to truncate the input?
>
> Correct again.
Not sure, but this might have been due to the truncation of input
leading to an invalid grammar file.
>
> Best regards,
> Johannes Luber
>

Tom.