[antlr-interest] Re: ANTLR and runtime token assigning - URGENT
antlrlist
antlrlist at yahoo.com
Fri Jun 20 07:02:22 PDT 2003
> 1. How can I set "SEPARATOR" token value from my grammar file in
produced CSVLexer/CSVParser?
> I mean, I wrote class that uses this Parser/Lexer but I want the
class' user to have ability to set SEPARATOR to ';'
> instead of ",". How is it possible to define variable SEPARATOR in
Parser/Lexer that will be used to process input
> without running antlr.Tool again?
The only way I could think about is accepting both "," and ";", like this:
SEPARATOR : ';' | ',';
Now the lexer recognizes both entries like SEPARATOR at the same time.
If you don't want to recognize them simultaneously use a class member:
class CSVLexer extends Lexer;
...
{
protected char sepChar = ',';
... // your gets and sets here
}
SEPARATOR : { sepChar==',' }? ','
| { sepChar==';' }? ';'
;
> I wonder why ANTLR hard-code tokens instead of exposing them in
easy-to-modify variables...
I had the same thoughts until I begun to understand the code that
ANTLR generates, and how it is generated. The idea is that you can't
generate all those "switchs" if you don't have fixed values...
> 2. In my grammar I tried to put
> RECORD : (~('"'|SEPARATOR|'\r'|'\n'))+;
> instead of
> RECORD : (~('"'|','|'\r'|'\n'))+;
>
> but antlr.Tool throws error:
> csv.g:119:13: This subrule cannot be inverted. Only subrules of the
form:
> (T1|T2|T3...) or
> ('c1'|'c2'|'c3'...)
> may be inverted (ranges are also allowed).
> Exiting due to errors.
>
> How can I do that?
>
If you're going to have ';' or ',' as SEPARATORS, then write:
RECORD : (~('"'|','|';'|'\r'|'\n'))+;
Sorry, but you just can't have such a "customizable separator
character". You have to specify the character set you're going to use
on each rule; if not, then ANTLR just won't be able to generate the
right code. The analizer's code is based on the "constantness" of your
tokens.
If you want to have such a customizable analizer, you'll have to
create it manually; A lexer class only needs to implement the
TokenStream interface.
Cheers!
Enrique
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list