[antlr-interest] specialStateTransition too big for Java(AntLR3)

[Jim Idle]

Show us your lexer rules and maybe we can help simplify them a little.
That usually helps as you can express the differences in a more natural
manner and reduce the number of lexer states.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Ulf Ochsenfahrt
> Sent: Saturday, August 11, 2007 11:51 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] specialStateTransition too big for
> Java(AntLR3)
> 
> Hi there,
> 
> I'm also currently building an antlr php grammar and have run into the
> very same problem. The problem is that antlr generates an extremely
> long
> specialStateTransition method (over 10000 lines) when a couple of
lexer
> rules (13 rules) have semantic predicates.
> 
> Has anyone solved this problem?
> 
> Cheers,
> 
> -- Ulf
> 
> > Hi,
> >
> > The problem is common and it has to do with the generated method tht
> > is is too big.
> > In fact, found this on the jvm spec:
> >
>
http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.h
> tml
> > (..)
> > The static constraints on the instructions in the code array are as
> follows:
> >     * The value of the code_length item must be less than 65536.
> > (...)
> >
> > You can break the rule that's giving you that error in two rules. I
> > just though on posting the jvm spec here so people don't think it's
a
> > antlr error, but instead a feature specified by the jvm spec.
> >
> > Greetings,
> > Miguel Ping
> >
> > On 5/4/07, Ruben Fonseca <rubenfonseca at di.uminho.pt> wrote:
> >> Hi!
> >>
> >> I'm using AntLR3 to build a PHP parser. I need to get rid of HTML.
> In
> >> order to do this I implemented a syntatic predicate on my Lexer, to
> see
> >> if I'm getting HTML or PHP tokens (see an example
> >>
>
http://svn.perl.org/viewcvs/parrot/trunk/languages/plumhead/src/antlr3/
> Plumhead.g?view=markup
> >> )
> >>
> >> The problem is that when I generate the Java code with AntLR3, I
> can't
> >> compile the code, because of the following error:
> >>
> >> The code of method specialStateTransition(int, IntStream) is
> exceeding
> >> the 65535 bytes limit	WebAppViewer/antlr3/php	phpLexer.java
> 	line 3894
> >>
> >> I'm guessing my Lexer is too complex or has too many tokens? Does
> anyone
> >> experienced the same problem before? Thanks for the advice.
> >>
> >> Ruben.
> >>
> >>