[antlr-interest] Re: Problem w/ string rule

[chiplastique]

I need my string rule to recognize unicode strings with with 'exotic' 
characters from other languages, so defining a token vocabulary for 
all of the possibilities would be problematic.

So that I understand clearly, will negation only recognize characters 
defined in lexer rules?

Also, are there any other internationalization/unicode issues I 
should consider with ANTLR?  Again, I'm using the C# ANTLR 
implementation so I would think that unicode support is a freebie.  
The only rule that needs to recognize funky characters is this string 
rule.

--- In antlr-interest at yahoogroups.com, Monty Zukowski <monty at c...> 
wrote:
> You need to define a "tokenVocabulary" -- by default ANTLR will 
only 
> recognize characters matched explicitly in lexer rules (not 
including 
> negations).
> 
> Monty
> 
> On May 19, 2004, at 3:17 PM, chiplastique wrote:
> 
> > I have the following simple lexer rule for identifying strings:
> >
> > STRING
> > 	:	'"' (~'"')* '"'
> > 	;
> >
> > This rule should recognize any sequence of characters enclosed by
> > double quotes.  However, I've discovered that strings including
> > certain punctuation characters such as '\', '`', or ';' cause 
ANTLR
> > to throw a TokenStreamRecognitionException in the STRING rule.
> >
> > Is this a bug or am I doing something wrong?  I'm using the 2.7.2
> > runtime for C#.
> >
> > Thanks!
> >
> >
> >
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> >
> >
> >
> >
> Monty Zukowski
> 
> ANTLR & Java Consultant -- http://www.codetransform.com
> ANSI C/GCC transformation toolkit -- 
> http://www.codetransform.com/gcc.html
> Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/