[antlr-interest] Additional tokens not being passed to Create(IToken token) ?
Terence Parr
parrt at cs.usfca.edu
Tue Oct 21 11:33:43 PDT 2008
Hi. I believe you need some dup methods too in the adaptor.
Ter
On Oct 21, 2008, at 11:18 AM, Dejas Ninethousand wrote:
> Consider the following grammar file snippet:
>
> grammar CSharpDoc;
>
> options {
> language='CSharp';
> output=AST;
> backtrack=true;
> }
>
> tokens {
> NAMESPACE_COMPLETE; // I am a imaginary token, I have no definition
> elsewhere in the file.
> }
>
> ...
>
> namespace_decl
> : NAMESPACE IDENT -> NAMESPACE IDENT NAMESPACE_COMPLETE;
>
> NAMESPACE
> : 'namespace';
>
> IDENT
> : ID_START (ID_PART)*
> ;
>
> ...
>
>
> I am using a custom TreeAdaptor and have overloaded the
> Create(IToken token) method to create my own SyntaxTree internally
> "on the side" within my custom tree adapter as tokens flow through
> Create. I have used this technique before with other grammars
> without a problem. But for some reason in this implementation I
> never see any NAMESPACE_COMPLETE tokens passed to Create. So, for
> example:
>
> public override object Create(IToken token)
> {
> if (token != null)
> {
> switch (token.Type)
> {
> case(CSharpDocLexer.NAMESPACE_COMPLETE):
> {
> // program never reaches this case
> }
> ... // do custom tree construction in the rest
> of the function
> }
> }
> return base.Create(token);
> }
>
> I can't for the life of me figurure out why, in this instance, i'm
> not seeing that extra token.
>
> So for some statement like "namespace cat" I would expect to see the
> following tokens pass trough the Create method in this order:
>
> NAMESPACE
> IDENT
> NAMESPACE_COMPLETE
>
> but what I am seeing is:
>
> NAMESPACE
> IDENT
>
> What am I missing?
>
> Thanks,
> Matt
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
More information about the antlr-interest
mailing list