[antlr-interest] Changing Token Names

"Paul Bouché (NSN)" paul.bouche at nsn.com
Wed Apr 1 09:23:24 PDT 2009


Jim Idle schrieb:
> Paul Bouché (NSN) wrote:
>   
>> Hi,
>>
>> I would like to change the token names for error display.
>>
>> I used have this:
>> value : simpleValue | '{' property '}';
>> where a token type for the curly braces was created by ANTLR (T_XXX), 
>> but if it was missing then in the error message a '{' was generated 
>> because for that token in the Java target tokenNames contained a '{'.
>> Now I needed to create tokens for the brackets, but I would still like 
>> the tokenName display as '{'. I solve this now by a static initializer 
>> which alters the tokenNames array - this is kind of a hack - is there 
>> another way? In ANTLR v2 there was the paraphrase option?
>>   
>>     
> You have to not use the literals in the parser and create lexer tokens, 
> then the token name will be fixed. Then you can use the array that ANTLR 
> generates for token names that make sense, and override with a switch or 
> a Map lookup for those tokens that don't. The problem with literals is 
> that if you add one somewhere in the parser, then all the Tnnn names 
> will change and break your messages.
>
> There is no paraphrase option in v3, so you are otherwise doing the 
> right thing.
>
> Jim
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>   
I am doing this:
public class ... {
    // generated...
        public static final String[] tokenNames = new String[] {
        "<invalid>", "<EOR>", "<DOWN>", "<UP>", "PROPSMESSAGE", 
"PROPERTYNODE", "PROPERTIES", "PROPERTY", "ATTRIBUTES", "ATTRIBUTE", 
"KEY", "TYPE", "CAST", "VALUE", "LIST", "NUMBER", "HEX", "BUFREF", 
"TEXT", "UTEXT", "PASSWORD", "IPADDRESS", "PEERADDRESS", "BOOL", 
"SIGNAL", "SIGNALNAME", "Signal", "NL", "DELIM", "NAME", "NUMERIC", 
"PATH", "STRING", "CB", "CA", "COLON", "DOT", "EQ", "OA", "OL", "CL", 
"OB", "HEXDIGIT", "BREF", "BOOLVAL", "IPADDR", "PEERADDR", "PWD", 
"NEWLINE_", "OB_", "CB_", "OL_", "CL_", "OA_", "CA_", "IP4", "IP6", 
"DIGIT_", "Quad4", "Quad6", "SIGNS", "HEXDIGIT_", "SIGNALNAME_", 
"COLON_", "NAME_", "STRING_", "ESCAPE", "UNICODE", "WHITESPACE", 
"HEXESCAPE", "NAMECHAR_", "NAMEchar_", "SIGNALNAMECHAR_", "SIGNALCHAR_", 
"HEXCHAR_"
    };
    // kind of a hack because tokenNames is final
        static {
            tokenNames[OL] = "'['";
            tokenNames[CL] = "'['";
            tokenNames[OB] = "'{'";
            tokenNames[CB] = "'}'";
            tokenNames[OA] = "'('";
            tokenNames[CA] = "')'";
            tokenNames[EQ] = "'='";
            tokenNames[DOT] = "'.'";
            tokenNames[DELIM] = "','";
            tokenNames[COLON] = "':'";
        }
}

Thx,
Paul

-- 
Paul Bouché
Voice: +49 30 590080-1284
 
Nokia Siemens Networks GmbH & Co. KG, An den Treptowers 1, 12435 Berlin, Germany
Sitz der Gesellschaft: München / Registered office: Munich
Registergericht: München / Commercial registry: Munich, HRA 88537
WEEE-Reg.-Nr.: DE 52984304

Persönlich haftende Gesellschafterin / General Partner: Nokia Siemens Networks Management GmbH
Geschäftsleitung / Board of Directors: Lydia Sommer, Olaf Horsthemke
Vorsitzender des Aufsichtsrats / Chairman of supervisory board: Lauri Kivinen
Sitz der Gesellschaft: München / Registered office: Munich
Registergericht: München / Commercial registry: Munich, HRB 163416

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090401/3fb17849/attachment.html 


More information about the antlr-interest mailing list