[antlr-interest] Additional tokens not being passed to Create(IToken token) ?

Terence Parr parrt at cs.usfca.edu
Tue Oct 21 11:33:43 PDT 2008


Hi. I believe you need some dup methods too in the adaptor.
Ter
On Oct 21, 2008, at 11:18 AM, Dejas Ninethousand wrote:

> Consider the following grammar file snippet:
>
> grammar CSharpDoc;
>
> options {
>     language='CSharp';
>     output=AST;
>     backtrack=true;
> }
>
> tokens {
> NAMESPACE_COMPLETE; // I am a imaginary token, I have no definition  
> elsewhere in the file.
> }
>
> ...
>
> namespace_decl
>     :    NAMESPACE IDENT -> NAMESPACE IDENT NAMESPACE_COMPLETE;
>
> NAMESPACE
>     :     'namespace';
>
> IDENT
>     :   ID_START (ID_PART)*
>     ;
>
> ...
>
>
> I am using a custom TreeAdaptor and have overloaded the  
> Create(IToken token) method to create my own SyntaxTree internally  
> "on the side" within my custom tree adapter as tokens flow through  
> Create.  I have used this technique before with other grammars  
> without a problem.  But for some reason in this implementation I  
> never see any NAMESPACE_COMPLETE tokens passed to Create.  So, for  
> example:
>
>   public override object Create(IToken token)
>         {
>             if (token != null)
>             {
>                 switch (token.Type)
>                 {
>                     case(CSharpDocLexer.NAMESPACE_COMPLETE):
>                     {
>                             // program never reaches this case
>                     }
>                       ... // do custom tree construction in the rest  
> of the function
>                 }
>             }
>             return base.Create(token);
>        }
>
> I can't for the life of me figurure out why, in this instance, i'm  
> not seeing that extra token.
>
> So for some statement like "namespace cat" I would expect to see the  
> following tokens pass trough the Create method in this order:
>
> NAMESPACE
> IDENT
> NAMESPACE_COMPLETE
>
> but what I am seeing is:
>
> NAMESPACE
> IDENT
>
> What am I missing?
>
> Thanks,
> Matt
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>



More information about the antlr-interest mailing list