[antlr-interest] (v3) Token aggregation

Sean Walton swalton at cs.utah.edu
Mon Aug 7 12:34:50 PDT 2006


Ter-
Have you had any time to look at this?  I'm at the point where I need 
this functionality.
Thanks.
-Sean

Sean Walton wrote:
> Terence Parr wrote:
>> actually it's the id+=foo that has been fixed already in my version.
>>
>> Note the error says idx which means it has nothing to do with the 
>> rule you provide.
>>
>> Ter
> oops.  Bad snapshot.
> The version you are talking about... is it released yet?  Because here 
> is a test case that demonstrates the problem:
>
>     grammar Test;
>     options {
>         output=AST;
>         memoize=true;
>         backtrack=true;
>     }
>
>     tokens {
>         PATH;
>     }
>
>     program options{ k=1; backtracking=false; }
>         :    id+=IDENTIFIER ('.' id+=IDENTIFIER) -> ^(PATH $id+)
>         ;
>        
>     IDENTIFIER
>         :    ('a'..'z'|'A'..'Z')+
>         ;
>
> Generates:
>
>     >java -classpath
>     "antlr-3.0b3.jar;stringtemplate-2.3b9.jar;antlr-2.7.6.jar"
>     org.antlr.Tool t.g
>     ANTLR Parser Generator   Early Access Version 3.0b3 (July 21,
>     2006)  1989-2006
>     0:0: syntax error: assign.types: <AST>:13:50: unexpected AST node: id
>     0:0: syntax error: assign.types: <AST>:0:0: expecting EOB, found
>     '<end-of-alt>'
>     0:0: syntax error: assign.types: <AST>:0:0: unexpected end of subtree
>     0:0: syntax error: define: <AST>:13:50: unexpected AST node: id
>     0:0: syntax error: define: <AST>:0:0: expecting EOB, found
>     '<end-of-alt>'
>     0:0: syntax error: define: <AST>:0:0: unexpected end of subtree
>     0:0: syntax error: buildnfa: <AST>:13:50: unexpected AST node: id
>     internal error: t.g : java.lang.NullPointerException
>     org.antlr.tool.NFAFactory.build_AlternativeBlock(NFAFactory.java:373)
>     org.antlr.tool.TreeToNFAConverter.block(TreeToNFAConverter.java:779)
>     org.antlr.tool.TreeToNFAConverter.rule(TreeToNFAConverter.java:548)
>     org.antlr.tool.TreeToNFAConverter.rules(TreeToNFAConverter.java:374)
>     org.antlr.tool.TreeToNFAConverter.grammarSpec(TreeToNFAConverter.java:328)
>     org.antlr.tool.TreeToNFAConverter.grammar(TreeToNFAConverter.java:188)
>     org.antlr.tool.Grammar.createNFAs(Grammar.java:677)
>     org.antlr.tool.Grammar.checkAllRulesForLeftRecursion(Grammar.java:1288)
>     org.antlr.codegen.CodeGenerator.genRecognizer(CodeGenerator.java:276)
>     org.antlr.Tool.processGrammar(Tool.java:320)
>     org.antlr.Tool.process(Tool.java:251)
>     org.antlr.Tool.main(Tool.java:70)
>
> Deleting the "backtracking=true;" I get:
>
>     >java -classpath
>     "antlr-3.0b3.jar;stringtemplate-2.3b9.jar;antlr-2.7.6.jar"
>     org.antlr.Tool t.g
>     ANTLR Parser Generator   Early Access Version 3.0b3 (July 21,
>     2006)  1989-2006
>
>     >
>
> Putting it back in and removing the '+' in the rewrite rule, I get:
>
>     >java -classpath
>     "antlr-3.0b3.jar;stringtemplate-2.3b9.jar;antlr-2.7.6.jar"
>     org.antlr.Tool t.g
>     ANTLR Parser Generator   Early Access Version 3.0b3 (July 21,
>     2006)  1989-2006
>
>     >
>
> Any ideas?
> -Sean

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20060807/308a43ad/attachment.html


More information about the antlr-interest mailing list