[antlr-interest] 4.0 daily builds

[Terence Parr]

On Jan 1, 2012, at 1:53 PM, Sam Barnett-Cormack wrote:
> Then again, it might be possible to do something like that with lexer 
> modes and multiple parsers.

Sure. You could send the different tokens on different channels and then use different parsers to handle each channel.

> What I immediately wonder is, in the special rules for inside the mode, 
> can you generate tokens within it, or does a mode always have to return 
> a single token?

Well, the lexer always returns a single token unless you explicitly tell it to emit more than one for every call to nextToken.  the modes do nothing but group the rules. I create a different state machine for each mode and so matching only finds tokens that are valid for that mode.

> Can the special rules modify the text they're matching 
> in terms of the text the eventual token gets?

sure by setting this.text.

> That would be particularly 
> useful in the string example, to actually process escape sequences right 
> there in the lexer.

Ter