[antlr-interest] Tracking Character Position in antlr3

Jan van der Ven jhvdven at xs4all.nl
Fri Jan 19 08:37:13 PST 2007


Edson Tirelli wrote:
>   Thanks for the quick answer Terence.
>
>   I will go with the casts then.
>
>   []s
>   Edson
>
> Terence Parr wrote:
>
>>
>> On Jan 19, 2007, at 7:43 AM, Edson Tirelli wrote:
>>
>>>
>>>    Hello All,
>>>
>>>    I think this is probably a recurring frequent question, but the  
>>> answers I was able to find, both in the FAQ( http://www.jguru.com/ 
>>> faq/view.jsp?EID=550800 ) and the list archives seems to refer to  
>>> antlr 2.x version. So, here we go again:
>>>
>>>    We have a parser generated by antlr 3. We also have an IDE built  
>>> over eclipse. We have a requirement from the IDE guys that the  
>>> parser must provide location within the source files as character  
>>> position from the beggining of the file, instead of line/columns.
>>>    Looking through antlr source code, it seems that CommonToken has  
>>> this information encoded as "start" and "stop" indexes and you can  
>>> get them through methods: getStartIndex() and getStopIndex(). I  
>>> developed some unit tests (hardcoding class casts from Token to  
>>> CommonToken) and these attributes are consistent throughout the  
>>> parsing.
>>>
>>>    So, my questions:
>>>
>>> 1) Is there any reason for these methods to not be exposed in the  
>>> Token interface? In other words, can they be exposed in the Token  
>>> interface implemented by CommonToken? This would avoid the nasty  
>>> class casts...
>>
>>
>> A minimal token doesn't need those...can just answer getText.
>>
>> I can see your need though.
>>
>>> 2) If not, do you see any reason for me to not do the casts and get  
>>> the info from the CommonToken class?
>>
>>
>> That is ok.
>>
>>> 3) Is there any other better alternative way of doing it?
>>
>>
>> Not at the moment...we could extend Token, but it's getting complex.
>>
>> Ter
>>
>
>
What I did was to remove all newline() calls. Then everything is deemed 
to be on one line and that meets your criterium I think.

Jan



More information about the antlr-interest mailing list