[antlr-interest] Lexer Problem: Double meaning of single quote character

[andreasklimke]

Matlab allows the single quote character (') to have two meanings:
  1) The conjugate transpose operator (a matrix operation)
  2) Delimiter of a string literal

Example:
(A+B)' : Transpose of A+B
'hello' : String hello

The quote should be recognized as transpose operator only following
certain tokens, such as ), ], an indentifier IDENT, etc.
Otherwise, it should be recognized as the beginning of a string
literal. It looks like having lexer states/lexer multiplexing does not
really fit this problem, I guess there is a better way of doing it.

I have written the following lexer rules, which detect the transpose
after a right parenthesis correctly; however, instead of returning two
tokens RPAREN and CTRANSPOSE (this is what I really want), it looks
like I can only return ONE token indicating both (which I did below).

Another option: Is it possible for a rule that's being processed to
know the previously returned token, or can I somehow set a flag (e.g.
R_PAREN=true) so I can manually transport this information? This would
make this problem easy to solve.

A little hint to point me in the right direction would be great.

- Andreas



RPAREN_OR_CTRANSPOSE
    :   RPAREN                      { $setType(RPAREN); }
    |   (RPAREN '\'') => CTRANSPOSE { $setType(RPAREN_CTRANSPOSE); }
    ;

protected
CTRANSPOSE
    :   (RPAREN '\'')
        { System.out.println("Matched transpose"); }
    ;

protected
RPAREN	: ')' ;

STRING_LITERAL
    :   '\'' (~'\'')* '\''
    ;


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/