[antlr-interest] Comment rule matches links

[Jenny Balfer]

> > If you don't need the regular expression structure you can just match
> > them as a single token. If you do need structure you could either just
> > use a seperate grammar in a later phase to process the single tokens
> > in your main token stream or use the technique from the island-grammar
> > example, which shows island grammars under lexer control,
> 
> Island grammars under lexer control will probably not cut it, as the  
> '/' token is ambiguous in many languages, e.g.
> "int x = 5 / 3;" vs "match(/3/, ...);". The grammar/lexer switching  
> has to be done in the parser.
> 
> Regards,
> Martin

That's right, but how can I implement an island grammar under parser
control if the string matching already was done in the lexer? 

Due to the fact comments are not part of the program statements, they have
to be skpped in the lexer, and to avoid strings containing //s to be
skipped, I implemented the string token rule also in the lexer. So I really
need a way to handle my regexp problem in the lexer, too - or is there
another way?