[antlr-interest] Lexer for Ruby's heredoc syntax

[Andrea Polci]

I'm trying to write a parser that allow the Ruby's heredoc syntax that
allow to write something like this:

method_name(<<DELIM, 123, <<OTHERDELIM)
    This is the content for the
    first argument of the method
DELIM
   This is the content for the third argument of the method
OTHERDELIM
otherMethod()
....


The lexer should produce the following token stream:
IDENTIFIER LPAREN HEREDOC COMMA INTEGER COMMA HEREDOC RPAREN EOL IDENTIFIER
...

What the lexer should do when it found a heredoc tag (<<XXXX) is to mark
the current position, skip to the following line, than consume all the
characters until it finds the delimiter matching the tag. After that it
should rewind to the mark previously set.
The problem is that it should then skip all the lines part of the heredoc
already analysed.

Is there a way to do something similar?
All I can think of is to wrap the input CharStream and allow to mark lines
that have to be ignored by calls to input.consume() (and other methods of
the CharStream interface).

Thanks for any help.

Andrea