[antlr-interest] Stripping Tokens, Skipping leading text

Christian Schladetsch christian.schladetsch at gmail.com
Fri May 8 16:33:34 PDT 2009


Hello,

Using ANTLR 3.1.3 with DOT-NET-runtime-3.1.3 and "CSharp" target,
output=template.

I have a situation where I'd like to gather all text between two delimiters,
but excluding those delimiters:

    [[
           any text
    ]]

My attempts so far have failed:

    CODE_BLOCK: '[[' (options{greedy=false;}:.)* ']]' ;

This correctly parses the entire token, but the token value in the lexer
contains the enclosing delimiters '[[' and ']]'

Moving on, to strip the delimiters:

    CODE_BLOCK: OPEN_CODE (options{greedy=false;}.*) CLOSE_CODE ;
    OPEN_CODE : '[[' {$channel=HIDDEN;} ;
    CLOSE_CODE : '[[' {$channel=HIDDEN;} ;

but the delimiters remain visible in the parser:

    code_block : cb=CODE_BLOCK -> code(text={$cb.Text}) ; // the code string
template gets the delimiters :(

I assume this is because CODE_BLOCK is not outputting to the HIDDEN channel,
overriding the channel used for OPEN_CODE and CLOSE_CODE.

How can I ensure that the parser only sees the inner text of the block?

I also tried:

    CODE_BLOCK: '[[' inner=(options{greedy=false;}.*) ']]' { Text =
$inner.Text; } ;

But the C# runtime fails, because it declares 'inner' as an int and does
nothing with it.

I am missing something simple here I know. I would greatly appreciate your
help. I'd like to say something like we can for trees:

    CODE_BLOCK: '[['! ... ']]'! ;

To remote the two delimiters in the token text.

While I'm here, I have a similar problem. I'd like to skip all input until a
starting token is found:

    any text here that is not parsed lah di dah /** text here is parsed **/
no text parsing here

I tried:

    start: START parse_stuff END ;
    START : (options{greedy=false;}:.)* '/**' ;
    END : '**/' .* ;

But the start token '/**' is always skipped, despite the non-greedy prefix.
Any ideas greatly appreciated.

Regards and Thanks in advance,
Christian.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090509/2f5c102f/attachment.html 


More information about the antlr-interest mailing list