[antlr-interest] enums in v4 ANTLR Java code generation considered useless

Scott Stanchfield scott at javadude.com
Tue May 18 15:20:24 PDT 2010


A little clarification...

Start with (as Edgar said)

  public interface TokenType { }
  public enum CommonTypes implements TokenType {
      EOF;  // others?
  }

  public class Token {
    private TokenType type;
    public TokenType getType() {return type;}
    ...
  }

  public class Parser ... {
      void match(TokenType type) {
        if (LA(1).getType() == type) {
          ...
        }
      }
      ...
  }


then for the specific parser, generate

  public enum FooParserTokenType implements TokenType {
    IDENT, INT, ...;
  }


What might be nice would be to have a "description" available for the
type, something like

  public interface TokenType {
    String getDescription();
  }

  public enum FooParserTokenType implements TokenType {
        IDENT("identifier (lower camel case word of doom)"),
        INT("integer (digits, no decimal points)", ...;
    private String description;
    private FooParserTokenType(String description) {
      this.description = description;
    }
    public String getDescription() {
      return description;
    }
  }

this could be fed by something like

  IDENT
    options {description="identifier (lower camel case word of doom)"} : ...

and could be used to really help error messages by using
token.getType().getDescription().

This is where Java enums get really cool -- carrying extra data.

I've gotta upload an enumeration talk I gave at work (which described
other cool things you can do, like the GoF state pattern and something
kinda close to function pointers in java)

Hope this helps!
-- Scott

----------------------------------------
Scott Stanchfield
http://javadude.com



On Tue, May 18, 2010 at 5:58 PM, Scott Stanchfield <scott at javadude.com> wrote:
> There are several advantages to enums:
> * there is a discrete set of values that can be used (no accidental
> 42's passed in when 42 isn't a token type)
> * the enum value can carry extra information
> * the enum values can override methods differently
>
> OH - one of the things that's clouding this is that you really don't
> need the numeric type identifers anymore. You can just have
>
>  public enum TokenType {
>    IDENT, INT ...;
>  }
>
> then in your match method:
>
>  void match(TokenType type) {
>    if (LA(1).getType() == type) {
>        ...
>    }
>  }
>
> And you can use the types in a switch statement:
>
>  switch(type) {
>    case INT:
>    case IDENT:
>    ...
>  }
>
> No more magic numbers! Woohoo!
>
> -- Scott
>
> ----------------------------------------
> Scott Stanchfield
> http://javadude.com
>
>
>
> On Tue, May 18, 2010 at 4:52 PM, Terence Parr <parrt at cs.usfca.edu> wrote:
>> hi Edgar :)
>>
>> Don't i have to pass objects around then instead of ints with extra method calls etc...?
>>
>> void match(TokenType t) {
>>  int ttype = t.getType();
>> }
>>
>> no autoboxing etc...
>>
>> match(GeneratedTokenType.ID);
>>
>> Also i need Parser.GeneratedTokenType.ID in outside code.  pretty ugly.
>>
>> I guess i don't see enum advantages for token types and they cause confusion. "enum GeneratedTokenType implements TokenType" vs "int", in other words.
>>
>> Ter
>>
>> On May 18, 2010, at 1:47 PM, Edgar Espina wrote:
>>
>>> Hi,
>>>
>>> TokenType can be an interface. Then your generated tokens can implemented
>>> this interface.
>>>
>>> enum GeneratedTokenType implements TokenType {
>>> ID(3);
>>> }
>>>
>>> Thanks,
>>> edgar
>>>
>>> On Tue, May 18, 2010 at 5:42 PM, Terence Parr <parrt at cs.usfca.edu> wrote:
>>>
>>>> A lot of you have wondered why I'd don't use enum types for token types in
>>>> the generated Java-based parsers:
>>>>
>>>> enum TokenType { ID(3), INT(4) };
>>>>
>>>> I gave it my best shot but once again I found that enums just don't seem to
>>>> work for me. For example, without inheritance how can I define the match
>>>> method?
>>>>
>>>> void match(TokenType t) {....}
>>>>
>>>> Unfortunately this is in the superclass not the generated class so
>>>> TokenType is not yet defined. Without inheritance, I have to use match(int).
>>>> That means that there's no point in using the enum.
>>>>
>>>> But, you say, I should be using generics on the parser to pass in the token
>>>> type enum type. Fair enough, And I'll try once I tried to generic'ize the v3
>>>> runtime for v4. We've had some discussion of this on the dev list.
>>>>
>>>> Ter
>>>>
>>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>>> Unsubscribe:
>>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>>>
>>>
>>>
>>>
>>> --
>>> edgar
>>>
>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>


More information about the antlr-interest mailing list