[stringtemplate-interest] Re: [antlr-interest] StringTemplate + preserving comments

Terence Parr parrt at cs.usfca.edu
Tue Nov 8 22:19:26 PST 2005


On Nov 8, 2005, at 10:10 PM, Sohail Somani wrote:

> On Tue, 2005-08-11 at 21:54 -0800, Terence Parr wrote:
>> On Nov 8, 2005, at 9:51 PM, Sohail Somani wrote:
>>>
>>> This is quite possible. The language I'm parsing is very whitespace
>>> happy (sometimes!). No this was not fun to parse. So it could be  
>>> that
>>> the lexer passes:
>>>
>>> COMMENT EOL SOME_THING
>>>
>>> And of course when I try to print the text of SOME_THING, there  
>>> is no
>>> hidden before.
>>>
>>> Any suggestions?
>>
>> Yes.  Make that IF a WHILE loop to print ALL hidden tokens between
>> your token and the previous :)  It chains them all together :)
>
> Ah, but isn't this information available only in the tokenstream  
> and not
> the token? Or am I missing something...
>
> If I try to do SOME_THING.getText(), getHiddenBefore() will be null
> because there is no hidden token before (as it is EOL)... So I don't
> know where to get this chain from.

Each real token (in v2) will have a linked list between it and the  
previous real token with all hidden tokens.  If you send newlines and  
comments to the parser as hidden, there will be a linked list (right  
to left) from your token to the previous token.  Are you stripping  
out newlines perhaps?

Ter


More information about the antlr-interest mailing list