[antlr-interest] Island grammars in Antlr 2?

Pete Gonzalez pgonzalez at bluel.com
Thu Nov 17 10:08:47 PST 2005


Fiid Williams wrote:
> "banner"  delim=one_token  {
>             // eat tokens until the end of the banner.
>             String delimchar = delim.getText().substring(0,1);
>             while(true) {
>                 Token t = LT(1);
>                 if(t.getText().contains(delimchar)) {
>                     break;
>                 }
>                 consume();
>             }
>             consume();
>         };
> 
> You could concatenate the strings you get back from token t to get  the 
> full text of what's between the braces.  one_token matches  everything 
> except a newline (which is significant in my grammar.).

Interesting idea.  But with this approach I think parts of the text (e.g. 
comments and whitespace) will be discarded by the lexer.

I'm experimenting with another approach where a public flag controls a 
semantic predicate in the lexer -- the parser would set this flag when it 
is expecting the island token.   I'm not sure if it will work however, 
since other tokens could accidentally invoke the lexer as part of their 
lookahead.

Cheers,
-Pete


More information about the antlr-interest mailing list