[antlr-interest] Simple Grammar breaks ANTLRWorks Interpreter & Debugger?

John B. Brodie jbb at acm.org
Sun Aug 16 19:00:16 PDT 2009


Greetings!

On Sun, 2009-08-16 at 18:25 -0600, consiliens at gmail.com wrote:
> I looked at ANTLRStringStreams consume() method and noticed that it 
> checks for a '\n'. In the current grammar I'm skip()ing newlines, 
> instead of placing them on a hidden channel.
> 
> I made the following modification to the grammar in ANTLRWorks which 
> promptly broke the compiler.
> fragment NEWLINE : '\n' '\r'? {$channel=HIDDEN;};
> fragment WS      : ('\n'|'\r'|'\t'|' ')+ {$channel=HIDDEN;};
> 
> The error message is "cannot find symbol : variable _channel". I edited 
> QuizLexer.java and added
> public static int _channel; Now when I run the test rig, I can see the 
> tokens returned from lexer.nextToken().getText(), although tokens.size() 
> still reports zero. This change also enables the debugger to correctly 
> parse the input.
> 

$channel is only available - as far as i know - on non-fragment Lexer
rules ... e.g. $channel is an attribute of a Token and fragment Lexer
rules do not produce a Token.

Is there a reason for your NEWLINE and WS rules to be a fragment?

Maybe remove the fragment keyword from these two lexer rules.

BUT

do other Lexer rules reference NEWLINE and/or WS? If so, you probably
need to keep the fragment and want to create a trampoline-style Lexer
rule that creates a Token but then hides it. so perhape something like

fragment NEWLINE : '\n' '\r'? ; // for use by other Lexer rules
fragment WS      : ('\n'|'\r'|'\t'|' ')+ ; // for use by other Lexer rules

HIDDEN_NEWLINE : NEWLINE { $channel=HIDDEN; } ; // hide from Parser
HIDDEN_WS : WS { $channel=HIDDEN; } ; // hide from Parser

> Is $channel=HIDDEN not working in ANTLRWorks a known issue? 

I believe that ANTLRWorks already handles $channel=HIDDEN correctly,
when that construct is applied correctly. altho, in general, it is my
experience that it is not really reasonable to expect any of the ANTLR
tools to deal with any stuff that appears inside braces {} because that
stuff inside the braces is Target-Language specific and the ANTLR tool
suite tries to be Target agnostic...

> The 404 Not 
> Found error for its source code repository does not inspire confidence 
> http://fisheye2.cenqua.com/browse/antlrworks.
> 

a completely different issue which I am unable to address --- other than
the fisheye host has historically been unreliable (in my experience).

Hope this helps....
   -jbb





More information about the antlr-interest mailing list