[antlr-interest] Is tokenVocab an Option for Parser Grammars?

[Johannes Luber]

Jim Idle schrieb:
> On Fri, 2008-08-08 at 16:22 -0700, titech at metaorb.com wrote:
>> Jim,
>>
>>
>> >> On Fri, 2008-08-08 at 13:23 -0700, titech at metaorb.com <mailto:titech at metaorb.com> wrote:
>>
>> >>>   
>> >From the docs I've seen, it seems that the tokenVocab option should
>> work in a parser grammar, but this doesn't seem to work in 3.1b2.
>>
>> >> 
>> >> You don't need it for a combined grammar, but you DO need 
>> it for a separate parser grammar and lexer grammar.
>> >>
>> >> Jim
>>
>>
>> If that's the case, then this might be a bug, because, as far as I can tell, it doesn't work with 3.1b2--unless I'm just using the wrong syntax.
>>
>> Here's what I've got:
>>
>> Parse.g:
>>         parser grammar dsn_hdr;
>>
>>         options
>>         {
>>           tokenVocab=Lex;
>>         }
>>        
>>        /* Parser Rules */
>>
>>
>> Lex.g:
>>         lexer grammar Lex;
>>
>>
>>         /* Lexer Rules */
>>
>>
>> Of course this all works if I copy the lexer rules to the parser.  Is this the correct syntax? 
> 
> Yes, it is the correct syntax and I verified that it works on b2 on my 
> local machine. Are you generating the lexer first and is it in the same 
> directory as the parser? What error message are you getting? Your parser 
> grammar name needs to be the same as the file name by the way.

It should be mentioned that the tokenVocab-option requires the use of 
the tokens file, which must be in the same directory as the grammar 
which uses the tokenVocab-option. There is -o option for ANTLR which 
moves all the output files into another directory, included the 
generated source code files. If that doesn't work for you and you don't 
want merge the directories containing the grammar files, you have to 
include a file copy in your build setup.

Johannes

> 
> Example:
> parser grammar Parse;
> 
> options
> {
> tokenVocab = Lex;
> }
> 
> rule : FRED+ EOF;
> 
> lexer grammar Lex;
> 
> FRED : 'FRED';
> WS : (' ' | '\t')+ { skip(); } ;
> 
> $ a3 Lex.g
> ANTLR Parser Generator  Version 3.1b2 (July 17, 2008)  1989-2008
> 
> $ a3 Parse.g
> ANTLR Parser Generator  Version 3.1b2 (July 17, 2008)  1989-2008
> 
> $ grep FRED *.java
> 
> $ grep FRED *.java
> Lex.java:    public static final int FRED=4;
> ..
> Parse.java:        "<invalid>", "<EOR>", "<DOWN>", "<UP>", "FRED", "WS"
> Parse.java:    public static final int FRED=4;
> ...
> 
> 
> 
> 
> 
> Jim
> 
>