[antlr-interest] Mismatched Token with String and Newline
Robert.Klaus at innovations.de
Robert.Klaus at innovations.de
Fri Oct 26 05:03:30 PDT 2007
Ahh,
Thanks! Skipping tokens helps alot if you don't forget it ...
Bye
Robert
> -----Original Message-----
> From: Gavin Lambert [mailto:antlr at mirality.co.nz]
> Sent: Friday, October 26, 2007 11:54 AM
> To: Klaus Robert; antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Mismatched Token with String and Newline
>
> At 22:25 26/10/2007, Robert.Klaus at innovations.de wrote:
> >I have a problem with lexing Strings and I'm thinking I
> have done >something wrong. When entering the grammar below
> in ANTLRWorks and >use the interpreter with the input "\n"
> it works fine. When I >enter a line break after the "\n" I
> get a MismatchedTokenException.
>
> First: ANTLRworks isn't really designed for debugging lexers,
> just parsers. So when you're facing a lexer problem it's
> better to write a test harness outside of ANTLRworks and step
> through it with your debugger of choice.
>
> >fragment CHARACTER
> > :
> >
> > ('\\' ('n'|'t'|'r'|'\\'|'"'))
> > | ('\\u' HEX_CHAR HEX_CHAR HEX_CHAR HEX_CHAR)
> > ;
>
> "ESCAPE" would probably be a better name than "CHARACTER".
>
> Also, given ANTLR's trouble dealing with common prefixes, you
> should probably rewrite this to:
>
> fragment CHARACTER
> : '\\'
> ( 'n' | 't' | 'r' | '\\' | '"'
> | 'u' HEX_CHAR HEX_CHAR HEX_CHAR HEX_CHAR
> )
> ;
>
> >NEWLINE : '\r'? '\n' ;
> >WS : (' ' | '\t' | '\f')+ ;
>
> You're not skipping or hiding the NEWLINE, so it's going to
> get emitted. This means that your given input string ought
> to get lexed to STRING NEWLINE EOF, and you're only accepting
> STRING EOF. Hence the second of the two errors you reported.
>
>
More information about the antlr-interest
mailing list