[antlr-interest] Re: Problem w/ string rule
chiplastique
jeremyj at imtc.gatech.edu
Thu May 20 07:32:49 PDT 2004
I need my string rule to recognize unicode strings with with 'exotic'
characters from other languages, so defining a token vocabulary for
all of the possibilities would be problematic.
So that I understand clearly, will negation only recognize characters
defined in lexer rules?
Also, are there any other internationalization/unicode issues I
should consider with ANTLR? Again, I'm using the C# ANTLR
implementation so I would think that unicode support is a freebie.
The only rule that needs to recognize funky characters is this string
rule.
--- In antlr-interest at yahoogroups.com, Monty Zukowski <monty at c...>
wrote:
> You need to define a "tokenVocabulary" -- by default ANTLR will
only
> recognize characters matched explicitly in lexer rules (not
including
> negations).
>
> Monty
>
> On May 19, 2004, at 3:17 PM, chiplastique wrote:
>
> > I have the following simple lexer rule for identifying strings:
> >
> > STRING
> > : '"' (~'"')* '"'
> > ;
> >
> > This rule should recognize any sequence of characters enclosed by
> > double quotes. However, I've discovered that strings including
> > certain punctuation characters such as '\', '`', or ';' cause
ANTLR
> > to throw a TokenStreamRecognitionException in the STRING rule.
> >
> > Is this a bug or am I doing something wrong? I'm using the 2.7.2
> > runtime for C#.
> >
> > Thanks!
> >
> >
> >
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> >
> >
> >
> >
> Monty Zukowski
>
> ANTLR & Java Consultant -- http://www.codetransform.com
> ANSI C/GCC transformation toolkit --
> http://www.codetransform.com/gcc.html
> Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list