[antlr-interest] Fwd: Re: Trying to keep whitespace in an AST

Johannes Luber jaluber at gmx.de
Fri Feb 8 08:54:47 PST 2008


Artem Portnoy schrieb:
> I'm trying to solve the exact same problem. I found some sample code 
> here http://www.antlr.org/article/whitespace/index.html. The problem is 
> that the way the existing Java grammar is written, not all the tokens 
> end up in the AST tree, such as semicolons, curly brakets, etc. The 
> result of this is some of the comments end up getting weaved onto these 
> "missing" tokens, and there's no way to get them back from AST. If 
> somebody knows how to resolve this issue please share the information.

The description is still for ANTLR 2.7.7. Did you manage to translate 
into ANTLR 3 syntax? I am nonetheless wondering, why you get the pure 
syntactical tokens with the whitespace. My understanding is that each 
token includes only the text it actually matched. Did you check the 
"intermediate" tokens for being only whitespace and comments? If that 
doesn't work the only option is to include the neglected tokens into the 
AST.

Johannes



More information about the antlr-interest mailing list