[antlr-interest] lexing multiple literals to one token

Robert Anderson randerson110 at gmail.com
Wed Jul 27 18:54:25 PDT 2005


I can't quite figure out the syntax for this:

I want to lex two different (interchangeable) keywords into the same
token.  I want to use the tokens {..} mechanism because I want both of
these to be considered by a testLiterals=true identifier rule option. 
How do I do this?  The following don't seem to work:

tokens {
   MYTOK="form1";
   MYTOK="form2";
}

generates a "redefinition" warning.

tokens {
  MYTOK = "form1","form2";
}

generates an "unexpected token: ," error.

tokens {
  MYTOK = ("form1"|"form2");
}

generates a "expecting STRING_LITERAL, found '('" error.

I cannot seem to find any real documentation on what is allowable in a
tokens {...} block.

I would create a rule, but this ends up causing problems with
ambiguity with my identifier rule.  I would use the "literals option"
described in the documentation to get around this, but that appears to
have been removed and replaced with the token {..} mechanism.

Any hints?

Thanks,
Bob


More information about the antlr-interest mailing list