[antlr-interest] Island grammars in Antlr 2?

[Fiid Williams]

Yup.  I don't need the banner information so I'm just tossing it, so  
spaces or anything else aren't on my list of things to try and  
understand.

I was wondering if it's possible to use the multiplexing lexer stuff  
to deal with cases like this and switch to another lexer for the  
piece in-between...  there are times where I may want to do that in  
my application since IOS doesn't have any reserved words and it also  
supports short versions of some of the literals.

Fiid.


On Nov 17, 2005, at 10:08 AM, Pete Gonzalez wrote:

> Fiid Williams wrote:
>> "banner"  delim=one_token  {
>>             // eat tokens until the end of the banner.
>>             String delimchar = delim.getText().substring(0,1);
>>             while(true) {
>>                 Token t = LT(1);
>>                 if(t.getText().contains(delimchar)) {
>>                     break;
>>                 }
>>                 consume();
>>             }
>>             consume();
>>         };
>> You could concatenate the strings you get back from token t to  
>> get  the full text of what's between the braces.  one_token  
>> matches  everything except a newline (which is significant in my  
>> grammar.).
>
> Interesting idea.  But with this approach I think parts of the text  
> (e.g. comments and whitespace) will be discarded by the lexer.
>
> I'm experimenting with another approach where a public flag  
> controls a semantic predicate in the lexer -- the parser would set  
> this flag when it is expecting the island token.   I'm not sure if  
> it will work however, since other tokens could accidentally invoke  
> the lexer as part of their lookahead.
>
> Cheers,
> -Pete
>
>