[antlr-interest] Comments / White-Space and All The Rest
Randall R Schulz
rschulz at sonic.net
Fri Nov 24 11:30:31 PST 2006
Terence,
On Friday 24 November 2006 10:45, Terence Parr wrote:
> Hi. You are probably trying to execute code with in the lexer, but
> you should really simply matched lexical items in the lexer and leave
> processing to the parser. Place comments on a hidden token channel
> so that the parser does not see them in terms of parsing,
Naturally, that's what I do.
> but actions
> can still access the comments.
How? The Wiki page
<http://www.antlr.org/wiki/pages/viewpage.action?pageId=1057> alludes
to doing this, but lack specifics on how to code such actions.
> All non-skipped tokens are sent to
> the parser in the token stream and just by looking backwards and
> forwards in the token stream you can see all of the hidden channel
> tokens. For example, at the start of the method, look one token to
> the left of the first token of the method definition and you will
> find out if there is a comment there. :)
How do I get at the tokens? How do I navigate the token stream? I notice
that TokenStream.LB(int) is protected, so I cannot even call it from
parse action code.
I also notice that rules that return a single value (which is all but
two of the rules in my grammar) don't trigger the generation of return
types that descend from ParserRuleReturnScope. In those cases, how do I
get at the start and stop tokens?
> CommonTokenStream is your friend. :)
>
> Ter
Randall Schulz
More information about the antlr-interest
mailing list