[antlr-interest] Catch all characters token
Pete Forman
pete.forman at westerngeco.com
Thu Jul 8 01:38:50 PDT 2004
At 2004-07-07 20:01 +0100, Steve Jones wrote:
>The problem is complicated by the fact I need to return three tokens, eg:
>
>BEGIN
>SOME_TEXT
>END
>
>If I use greedy then then I will only produce one token that contains the
>terminals BEGIN and END as well as the interposing text.
It looks like one token's worth of info to me.
Some solutions to consider:
1) Return one token from lexer, parser puts three tokens into the AST.
textBlock: SOME_TEXT {## = #([BEGIN, "Begin"], ##, [END, "End"]);};
2) One token from lexer, use a TokenStream to make the three tokens.
http://www.antlr.org/doc/streams.html
3) Switch to a second lexer on BEGIN.
That is probably too heavyweight for your simple SOME_TEXT.
--
Pete Forman -./\.- Disclaimer: This post is originated
WesternGeco -./\.- by myself and does not represent
pete.forman at westerngeco.com -./\.- opinion of Schlumberger, Baker
http://petef.port5.com -./\.- Hughes or their divisions.
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list