[antlr-interest] Token Stream Question

[colettekirwan]

Hi

I was wondering if some could please explain what I am doing wrong.

Using the "Lexical Analysis With Antlr" Lecture Notes, I copied the 
example that explains how to ignore but not throw away whitespaces.

But everytime I run the example I get the following error.
D:\antlrTest>java -classpath D:\antlrTest\antlr.jar;. TestMain < 
test.in
a = 2;
{

      a=3;

}

b=aException in thread "main" line 9:8: unexpected char: 0x?F
        at TestLexer.nextToken(TestLexer.java:148)
        at antlr.TokenStreamHiddenTokenFilter.consume
(TokenStreamHiddenTokenFilt
er.java:38)
        at antlr.TokenStreamHiddenTokenFilter.nextToken
(TokenStreamHiddenTokenFi
lter.java:148)
        at antlr.TokenBuffer.fill(TokenBuffer.java:69)
        at antlr.TokenBuffer.LT(TokenBuffer.java:86)
        at antlr.LLkParser.LT(LLkParser.java:56)
        at TestParser.stat(TestParser.java:121)
        at TestParser.slist(TestParser.java:61)
        at TestMain.main(TestMain.java:25)



I am using the latest version of antlr, and believe I have copied the 
grammar code from the article correctly.

Below is a copy of the grammar code I am using.
class TestParser extends Parser;

{
public void hidden(Token tok) 
{
antlr.CommonHiddenStreamToken t = (antlr.CommonHiddenStreamToken)tok;
for ( ; t!=null ; t=TestMain.filter.getHiddenAfter(t) ) 
  {
  System.out.print(t.getText());
  }
}
}

slist
: ( stat )+
;

stat: l:LBRACE {hidden(l);} (stat)+ r:RBRACE {hidden(r);}
| id:ID {hidden(id);} a:ASSIGN {hidden(a);} expr s:SEMI {hidden(s);}
;

expr: i1:INT {hidden(i1);}
| i2:ID {hidden(i2);}
;

class TestLexer extends Lexer;
options {
charVocabulary='\u0000'..'\uFFFE';
}
WS : (' '
| '\t'
| ('\n'|'\r'('\n')?) {newline();}
)+
;



SL_COMMENT
: "//"
(~('\n'|'\r'))* ('\n'|'\r'('\n')?)
{newline();}
;

LBRACE: '{'
;

RBRACE: '}'
;

LPAREN: '('
;
RPAREN: ')'
;

STAR: '*'
;

PLUS: '+'
;
SEMI: ';'
;

ASSIGN: '='
;

protected
DIGIT
: '0'..'9'
;

INT : (DIGIT)+
;

ID : ('a'..'z')+
;


Any help with the above quesion, one be greatly appreciated.
Also if any one knows the new location to the Description of the 
preserve Whitespace example, that  would be great


Thanks



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/