[antlr-interest] TokenStreamRecognitionException

[Hugo Leeney]

In that link you posted Koehne, it refers to the literal option in the
Lexer and provides a link. However on the page linked to there is no
entry for such an option (althought the tesLiterals option is listed).
Any ideas as to why? Deprecated or something?

Thanks,

Hugo Leeney

On 6/6/06, Koehne Kai <Kai.Koehne at student.hpi.uni-potsdam.de> wrote:
> Hi,
>
> the problem is that the keywords defined in the tokens section are only detected if there is a lexer rule catching it already!
>
> "The literals table [...] is checked after each token is matched, so that the literals effectively override the more general identifier pattern."
> http://www.antlr.org/doc/lexer.html#Keywords_and_literals
>
> There is no lexer rule that starts with "-", so the keyword MINUS is not detected either.
>
> Regards,
>
> Kai Koehne
>
> ________________________________
>
> Von: antlr-interest-bounces at antlr.org im Auftrag von sunjigang8 at netscape.net
> Gesendet: Di 06.06.2006 11:59
> An: antlr-interest at antlr.org
> Betreff: [antlr-interest] TokenStreamRecognitionException
>
>
>
> I defined a token in lexer as  (  MINUS = "-"; ). I use ( minus:MINUS; ) in parser to parser the symbol "-". While TokenStreamRecognitionException was thrown and popped up a window says ( unexpected char: '-' ). I think the char is recongnised but could not form the expected token.
>
> Literals in lexer are consistent with that of parser, that is,  ( public const int MINUS = 31; ) are generated at the beginning of both lexer and parser.
>
> This must be a silly question. I wish you could give a clue.
>
> Robert
>
>
> options {
>     language  = "CSharp";
>     namespace = "MiniClean";
> }
> class MiniCleanLexer extends Lexer;
> options {
>             k=2;
>         }
> tokens
> {
>     IMPLEMENTATION="IMPLEMENTATION";
>     FUNCTIONAL="FUNCTIONAL";
>     STRATEGY="STRATEGY";
>     ABSTYPE="ABSTYPE";
>     FROM="FROM";
>     IMPORT="IMPORT";
>     SYSTEM="SYSTEM";
>     DEFINITION="DEFINITION";
>     MODULE="MODULE";
>     MACRO="MACRO";
>     TYPE="TYPE";
>     RULE="RULE";
>     IF="IF";
>     OR="||";
>     AND="&&";
>     DB_COLUMN= "::";
>     BAR= "|";
>     DERIVES= "->";
>     SEMI= ";";
>     COMMA= ",";
>     LPAREN="(";
>     RPAREN= ")";
>     COLUMN= ":";
>     LBRACKET= "[";
>     RBRACKET= "]";
>     PredefinedType = "=>";
>     PLUS ="+";
>     MINUS = "-";
>     DOT=".";
>     E="E";
>     INT= "INT";
>     REAL="REAL";
>     CHAR="CHAR";
>     BOOL="BOOL";
>     STRING="STRING";
>     FILE="FILE";
>     PROCID="PROCID";
>     WORLD="WORLD" ;
>     TRUE="TRUE";
>     FALSE="FALSE";
>     ID_LOWER;
>     ID_UPPER;
>     FunnyID1;
>     NUM;
> }
>
>
> //ID_LOWER: LowerCaseChar (RestChar)*;
> //ID_UPPER: UpperCaseChar (RestChar)*;
> //FunnyID1:  '@' | '#' | '$' | '%' | '^' | '&'| '?' |'*' | '/' | '='| '<' | '>' | '_' | '`' | '\''|'\"' ;
>
> //NUM:(Digit)+;
>
> protected UpperCaseChar: 'A'..'Z';
> protected LowerCaseChar: 'a'..'z';
> protected Digit: OctDigit |'8'|'9';
> protected OctDigit: '0'..'7' ;
> protected RestChar: LowerCaseChar | Digit| UpperCaseChar; //|CharDel|StringDel ; | Class1Char
>
> //protected StringDenot: StringDel (Char)* StringDel ;
>
> //protected Class1Char: '@' | '#' | '$' | '%' | '^' | '&'| '?' | '*' | '+' | '-' | '/' | '='| '<' | '>' | '_' | '.'| '`' | '\''|'\"' ;
>
> protected Class2Char: '|' | '!' | '(' | ')' | '{' | '}'| '[' | ']' | ':' | ';' | '~' | ',' ;
>
> protected Special: '\n' | '\r' | '\f' | '\b' | '\t'| '\\' ;//| StringDel | CharDel; // OctDigit OctDigit OctDigit ;
>
> WS:
> (
>      ' '
>     |  {setTabSize(4);} '\t'         //NOTE (UpperCaseChar|LowerCaseChar|Class1Char|Class2Char)* ('\r''\n' ){newline();}
>     | '\n'
>     | '\r' '\n'{newline();}
>
> ) {$setType(Token.SKIP);} ;
>
>
>
>
>
> __________________________________________________________________
> Switch to Netscape Internet Service.
> As low as $9.95 a month -- Sign up today at http://isp.netscape.com/register
>
> Netscape. Just the Net You Need.
>
> New! Netscape Toolbar for Internet Explorer
> Search from anywhere on the Web and block those annoying pop-ups.
> Download now at http://channels.netscape.com/ns/search/install.jsp
>
>
>


-- 
*******************************************
This document is strictly confidential and is intended for use by
 the addressee unless otherwise indicated.
*******************************************