[antlr-interest] RE: Does tree parser have to export its own token types?

[Anakreon Mejdi]

Something should exist if it is useful not if it doesn't hurt.
I have a question related with tokens too.
Could the lexer have a tokens section like the Parser has?
The reason I would like this is because from within lexer rules I change
the type of the Token which might an imaginary Token.
The workarounds I found are two.
1) Put lexer and parser in the same file and contribute in the same 
token types.So the imaginary tokens exist for the Lexer too.
The disadvantage is that the lexer and parser have to exist in the same
file.
2)Declare a protected empty lexer rule, example
protected IMAGINARY_TOKEN:;
The rule is never called so no complains from antlr and a new Token type
exists for IMAGINARY_TOKEN.
The disadvantage is that code is generated for the empty rule as well, 
which is a waste of resources.
Terence Parr wrote:
> On Tuesday, November 19, 2002, at 10:35  PM, Daniel Gackle wrote:
> 
> 
>>Ack.  I just had the novel idea of checking the documentation
>>and found a partial answer in the "Token Vocabularies" section.
>>I now understand that if I put my tree parser, parser and lexer
>>all in the same file and give them all an "exportVocab = XXX"
>>option, only one XXXTokenTypes will be generated.
>>
>>But what if I want them in separate files?  It seems that
>>importVocab lets you make sure that one grammar has access
>>to another's tokens, but there's no easy way to stop it
>>from exporting them to its own token space.  Is that correct?
>>
>>To reiterate, in my case the new grammar isn't defining any
>>tokens of its own, so for it to export its own personalized
>>token space is superfluous.
> 
> 
> But that extra file isn't hurting anything is it?
> 
> Ter
> --
> Co-founder, http://www.jguru.com
> Creator, ANTLR Parser Generator: http://www.antlr.org
> Lecturer in Comp. Sci., University of San Francisco
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 
> 
> 



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/