[antlr-interest] Are token aliases possible?

Sam Barnett-Cormack s.barnett-cormack at lancaster.ac.uk
Fri Oct 28 15:42:19 PDT 2011


The problem is what you're asking for isn't consistent with the way 
ANTLR works. For instance, say it were another constant with the same 
value... in that case, prediction would work correctly, but the tree 
that's created would be indistinguishable because CLASS and CLASS_DECL 
are the same thing, so you wouldn't be able to differentiate between 
them in the AST.

You need to define the node type in the AST, however awkward you find 
the code for that.

Sam

On 28/10/2011 21:49, Christian wrote:
> If CLASS_DECL should only be another name with the same type, than, in
> Java, it could be easily declared as another constant with the same value:
>
> private final static int CLASS_DECL = CLASS;
>
> However, ANTLR does not support it, at least not independent of the
> target language.
>
> Am 28.10.2011 16:38, schrieb Sam Barnett-Cormack:
>> That looks a very handy feature. I wonder if there's any chance of it
>> arriving in the Java version?
>>
>> Sam
>>
>> On 28/10/2011 14:50, Sam Harwell wrote:
>>> In the Java version, the rewrite syntax is the definitive way to
>>> specify the
>>> type of a tree node.
>>>
>>> In the newer versions of the C# port, you can specify tree node types
>>> as a
>>> token option, so you could do this and be equivalent to your rewrite
>>> rule.
>>> Note that I added a ^ as well; your first example did not include it but
>>> that's what makes a particular node the root of the created tree.
>>> Since the
>>> AST operators (^ and !) perform better in a number of ways than the
>>> rewrite
>>> operators, I added this feature to allow me to use AST operators in a
>>> number
>>> of places where I previously used rewrite syntax for the sole purpose of
>>> changing a node type.
>>>
>>> class_declaration
>>>     : CLASS<CLASS_DECL>^  type_or_generic   class_base?
>>> type_parameter_constraints_clauses?   class_body   ';'!?
>>>
>>>
>>> -----Original Message-----
>>> From: antlr-interest-bounces at antlr.org
>>> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Christian
>>> Sent: Friday, October 28, 2011 6:46 AM
>>> To: antlr-interest at antlr.org
>>> Subject: Re: [antlr-interest] Are token aliases possible?
>>>
>>> Yes, I know that one can use rewrite rules. Thank you anyway. In some
>>> cases
>>> however, I do not want to repeat the rules following the CLASS token and
>>> write an arrow etc. In such cases, it is shorter to just replace the
>>> CLASS
>>> by CLASS_DECL and add a few operators instead of a rewrite rule.
>>>
>>> As an example, I want to transform the following rule:
>>>
>>> class_declaration
>>>     : CLASS  type_or_generic   class_base?
>>> type_parameter_constraints_clauses?   class_body   ';'?
>>>
>>> into
>>>
>>> class_declaration
>>>     : CLASS_DECL  type_or_generic   class_base?
>>> type_parameter_constraints_clauses?   class_body   ';'!?
>>>
>>> versus
>>>
>>> class_declaration
>>>     : CLASS  type_or_generic   class_base?
>>> type_parameter_constraints_clauses?   class_body   ';'?
>>>         ->   ^(CLASS_DECL type_or_generic class_base?
>>> type_parameter_constraints_clauses? class_body)
>>>       ;
>>>
>>> I think everyone, especially me, would prefer the first solution.
>>>
>>> Am 28.10.2011 13:34, schrieb Sam Barnett-Cormack:
>>>> I don't know for sure, buut my immediate thought is that it isn't
>>>> likely to be possible (as tokens are generated without reference to
>>>> parse context, and must be unambiguous except where code manually
>>>> deals with the ambiguity). You could however create it as a dummy
>>>> token name, and rewrite to it in the parser, such that the AST
>>>> generated has a 'virtual' CLASS_DECL token generated from the 'real'
>>>> CLASS token. Pretty sure there are examples of this in the wiki.
>>>>
>>>> On 28/10/2011 12:27, Christian wrote:
>>>>> CLASS is only the token name. However, CLASS_DECL represents the
>>>>> semantics of what was parsed. It allows to be more consequent when
>>>>> analyzing the generated AST by names like CLASS_DECL, METHOD_DECL,
>>>>> FIELD_DECL etc.
>>>>>
>>>>> Example:
>>>>>
>>>>> switch (node.getType()) {
>>>>>       case CsRewriteRulesParser.NAMESPACE_DECL:
>>>>>          // TODO
>>>>>          break;
>>>>>       case CsRewriteRulesParser.METHOD_DECL:
>>>>>          // TODO
>>>>>          break;
>>>>>       case CsRewriteRulesParser.CLASS_DECL:
>>>>>          // TODO
>>>>>          break;
>>>>> }
>>>>>
>>>>> It would be just another name for the integer constant CLASS.
>>>>>
>>>>> Am 28.10.2011 13:19, schrieb Sam Barnett-Cormack:
>>>>>> I think the immediate answer is, "why?"
>>>>>>
>>>>>> On 28/10/2011 09:02, Christian wrote:
>>>>>>> Hi community,
>>>>>>>
>>>>>>> if there is a real token, say
>>>>>>>         CLASS : 'class';
>>>>>>> can a define an alias, say
>>>>>>>         CLASS_DECL = CLASS
>>>>>>> in some way?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Christian
>>>>>>>
>>>>>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>>>>>> Unsubscribe:
>>>>>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-addr
>>>>>>> ess
>>>>>>
>>>>>
>>>>>
>>>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>>>> Unsubscribe:
>>>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-addres
>>>>> s
>>>>
>>>
>>>
>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>> Unsubscribe:
>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>>
>>>
>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>> Unsubscribe:
>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address


More information about the antlr-interest mailing list