[antlr-interest] Comment rule matches links

[Gavin Lambert]

At 21:22 26/08/2008, Jenny Balfer wrote:
 >That's right, but how can I implement an island grammar under
 >parser control if the string matching already was done in the
 >lexer?
 >
 >Due to the fact comments are not part of the program statements, 

 >they have to be skpped in the lexer, and to avoid strings
 >containing //s to be skipped, I implemented the string token
 >rule also in the lexer. So I really need a way to handle my
 >regexp problem in the lexer, too - or is there another way?

I vote for slapping the language authors around the head with an 
old fish ;)

Regexp literals like that are a hard problem, since you'll 
definitely need parser context to determine that it even is a 
regexp in the first place.  You might be able to cobble something 
together using a non-greedy token source (so it doesn't slurp the 
whole thing up in one go) combined with the "island grammar under 
parser control" example, so you can switch out the lexer and then 
switch back with the input position just after the regexp and keep 
going from there.  But it'll get messy.

Is there a good Perl grammar for v3 around somewhere?  That has 
the same kind of regexp literals, so that might give you some 
ideas.