[antlr-interest] Re: Lexer problem: Double meaning of single quote character

[andreasklimke]

Hi Monty,

thanks to your advice, I solved the double meaning of the quote
character problem. With k=2, I had to make a minor change to make it
work, since the generated lexer code newer reached the second part of
the mSTRING_LITERAL function due to the initial test (if LA(1)==...".

This was your initial suggestion:

STRING_LITERAL
    : ('\'')=>{transposePrefixMatched()}? '\'' {$setType(TRANSPOSE);}  
      | '\'' (~'\'')* '\''


With k=2, the generated code then looked like this:

public final void mSTRING_LITERAL(boolean _createToken) throws 
...
if ((LA(1)=='\'')) {
  match('\'');
  {
   _loop71:
   do {
      ...
    } while (true);
  }
  match('\'');
}
else {
   boolean synPredMatched73 = false;
   if ((true&&(transposePrefixMatched()))) {
     int _m73 = mark();
     synPredMatched73 = true;
     inputState.guessing++;
...


I don't know if this behavior is intentional or a bug. Anyway, I
changed the grammar to look like this, and it seems to work fine now.

STRING_LITERAL
    : {!transposePrefixMatched()}? ('\'') (~'\'')* '\''
    | ('\'') {$setType(CTRANSPOSE); }
    ;

I am including the complete solution to the problem below, which is
very close to your initial suggestion.

Thank you very much again for your excellent advice & help,
Andreas



class QuoteAmbiguityLexer extends Lexer;

options {
    testLiterals=false;
    k=2;
}

{
    int lastMatched = 0;
    
    protected Token makeToken(int t) {
        if ( t != Token.SKIP) {
          lastMatched=t;
        }
        Token tok = (Token) super.makeToken(t);
        return tok;
    }

    protected boolean transposePrefixMatched() 
    {
        System.out.println("Testing...");
        return lastMatched==RPAREN;
    }
}

RPAREN		:   ')'	    ;

STRING_LITERAL
    : {!transposePrefixMatched()}? ('\'') (~'\'')* '\''
    | ('\'') {$setType(TRANSPOSE); }
    ;




 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/