[stringtemplate-interest] Re: [antlr-interest] StringTemplate +
preserving comments
Terence Parr
parrt at cs.usfca.edu
Tue Nov 8 22:19:26 PST 2005
On Nov 8, 2005, at 10:10 PM, Sohail Somani wrote:
> On Tue, 2005-08-11 at 21:54 -0800, Terence Parr wrote:
>> On Nov 8, 2005, at 9:51 PM, Sohail Somani wrote:
>>>
>>> This is quite possible. The language I'm parsing is very whitespace
>>> happy (sometimes!). No this was not fun to parse. So it could be
>>> that
>>> the lexer passes:
>>>
>>> COMMENT EOL SOME_THING
>>>
>>> And of course when I try to print the text of SOME_THING, there
>>> is no
>>> hidden before.
>>>
>>> Any suggestions?
>>
>> Yes. Make that IF a WHILE loop to print ALL hidden tokens between
>> your token and the previous :) It chains them all together :)
>
> Ah, but isn't this information available only in the tokenstream
> and not
> the token? Or am I missing something...
>
> If I try to do SOME_THING.getText(), getHiddenBefore() will be null
> because there is no hidden token before (as it is EOL)... So I don't
> know where to get this chain from.
Each real token (in v2) will have a linked list between it and the
previous real token with all hidden tokens. If you send newlines and
comments to the parser as hidden, there will be a linked list (right
to left) from your token to the previous token. Are you stripping
out newlines perhaps?
Ter
More information about the antlr-interest
mailing list