[antlr-interest] Comments / White-Space and All The Rest
Terence Parr
parrt at cs.usfca.edu
Fri Nov 24 10:45:12 PST 2006
Hi. You are probably trying to execute code with in the lexer, but
you should really simply matched lexical items in the lexer and leave
processing to the parser. Place comments on a hidden token channel
so that the parser does not see them in terms of parsing, but actions
can still access the comments. All non-skipped tokens are sent to
the parser in the token stream and just by looking backwards and
forwards in the token stream you can see all of the hidden channel
tokens. For example, at the start of the method, look one token to
the left of the first token of the method definition and you will
find out if there is a comment there. :)
CommonTokenStream is your friend. :)
Ter
On Nov 24, 2006, at 8:21 AM, Randall R Schulz wrote:
> Hi,
>
> I would like to integrate comments into the stream of top-level
> constructs recognized by my ANTLR (v3) grammar.
>
> Specifically, I'd like to treat comments in the input something like
> JavaDoc comments in that I accumulate them (merge their contents in a
> single StringBuilder instance) as they appear and then when the parser
> encounters a top-level construct (the analogy would be a method or
> field definition in Java source code) associate the accumulated
> comment, if any, with that construct.
>
> I've had partial success, but there's one problem: It appears all the
> comments are processed before any of the parsing happens. Thus the
> comments don't properly interleave with the non-comment content and
> don't get associated with the proper constructs. Instead, they all
> appear at the beginning, merged into a single comment block.
>
> How can I get the kind of behavior I want? Namely that comment action
> code is processed in sequence with the parsing process?
>
>
> Thanks.
>
> Randall Schulz
More information about the antlr-interest
mailing list