[antlr-interest] Re: Token.filename being ignored - solutions
antlrlist
antlrlist at yahoo.com
Mon Jun 9 07:30:09 PDT 2003
Anyway the CharScanner should fill the filename information in
antlr.CharScanner.makeToken, should't it? If not, how do I add lex
info to my Tokens?
Enrique.
--- In antlr-interest at yahoogroups.com, Matt Benson <gudnabrsam at y...>
wrote:
> Since tokens are an open interface it is quite
> possible that a token would not come from a file at
> all. If you need this why don't you subclass token to
> have a filename and have your TokenStream
> implementation then populate that attribute?
>
> -Matt
>
> --- antlrlist <antlrlist at y...> wrote:
> >
> >
> > Terrence,
> >
> > I've had this problem for a little while: filenames
> > are usually
> > ignored by default in antlr. CommonToken does not
> > take it in account,
> > and what's worse antlr.CharScanner.makeToken() does
> > not fill it.
> >
> > Could you please explain this? Is it a matter of
> > efficiency?
> >
> > According to this link:
> >
> >
>
http://java.sun.com/docs/books/tutorial/java/data/stringsAndJavac.html
> >
> > storing a String in CommonToken would add the
> > overhead of managing 1
> > pointer - String objects are not created unless you
> > modify them.
> >
> > This is shown specially in this piece of code :
> >
> > String s1 = "hello";
> > String s2 = s1;
> > System.out.println("s1 = " + s1
> > + "; s2 = " + s2);
> > System.out.println("System.identityHashCode(s1) =
> > "
> > + System.identityHashCode(s1));
> > System.out.println("System.identityHashCode(s2) =
> > "
> > + System.identityHashCode(s2));
> >
> > s1 += " world";
> > System.out.println("\ns1 = " + s1
> > + "; s2 = " + s2);
> > System.out.println("System.identityHashCode(s1) =
> > "
> > + System.identityHashCode(s1));
> > System.out.println("System.identityHashCode(s2) =
> > "
> > + System.identityHashCode(s2));
> >
> > Here's the output:
> >
> > s1 = hello; s2 = hello
> > System.identityHashCode(s1) = 2452092
> > System.identityHashCode(s2) = 2452092
> >
> > s1 = hello world; s2 = hello
> > System.identityHashCode(s1) = 7474923
> > System.identityHashCode(s2) = 2452092
> >
> > s1 points to a new address after " world" is
> > appended.
> >
> > Cheers,
> >
> > Enrique
> >
> >
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
> > http://docs.yahoo.com/info/terms/
> >
> >
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
> http://calendar.yahoo.com
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list