[antlr-interest] Re: A couple of questions regarding literals and unicode

[davidjpenton2002]

Duh...now I feel silly. That seems pretty obvious. 

I misunderstood what I saw in other sample grammars.  In grammars I
looked at to teach myself antlr, I saw that there was no *exact*
lexical rule corresponding to a parser literal in many cases.  But I
suppose that in such cases the *whole point* is that another (more
general) lexer rule matches the literal, thereby necessitating an
option to cause a check against the literal table.  Is that it?

I guess I'll ask if I can get my ditch-digging job back, or else maybe
I just need more sleep :-)

Thanks!

- Dave -

--- In antlr-interest at y..., Terence Parr <parrt at j...> wrote:
> 
> On Friday, December 6, 2002, at 12:47  PM, davidjpenton2002 wrote:
> 
> > Greetings.  I am struggling a little with getting literals recognized.
> > I seem to have problems getting non-alphabetic characters to be
> > recognized in literals. For example:
> >
> > class P extends Parser;
> >
> > startRule
> >   :  "<?xml" SOMETHING
> >   ;
> >
> > class L extends Lexer;
> > options
> > {
> >   charVocabulary="\003'..'\377';
> > }
> >
> > SOMETHING : "abcd";
> >
> > The inclusion of the non-alphabetic characters "<?" in the literal
> > seems to cause problems.
> 
> The literals in the parser are tested in the lexer, but you have to 
> have a rule that matches those char.  <? is not matched by any rule so 
> the lexer cannot return that token.
> 
> >
> > As you might guess, I am trying to parse some xml.  So this leads to a
> > more general question. Does antlr handle unicode?  The info on the
> > website does not seem to make it clear whether it does or not.
> 
> It does and I'm thinking of making enhancements real quick before 2.7.2 
> comes out.
> 
> Ter
> --
> Co-founder, http://www.jguru.com
> Creator, ANTLR Parser Generator: http://www.antlr.org
> Lecturer in Comp. Sci., University of San Francisco


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/