[antlr-interest] Pushdown lexer

[Jim Idle]

It's difficult to say without seeing your lexer grammar Peter. But do
remover that you don't have to use an ANTLR lexer, you could hand
craft a lexer that provides the same interface.

I have usually found that the limits are only exceeded by lexers if
they are trying to do "too much". Such as trying to be too specific
about the character ranges for tokens.

If you can post your grammar, then there might

Jim

On Nov 11, 2012, at 13:52, "Peter S. May" <me at psmay.com> wrote:

> Hello again.
>
> I've been in the process of writing a lexer that could rightly be described as a pushdown automaton with some utilization of lookahead; it variously pushes and pops states on a stack depending on the input and uses the top as a start conditions. This essentially makes it so that every single rule begins with a semantic predicate, since ANTLR doesn't do start conditions on its own. Naturally, "The code of method specialStateTransition(int, IntStream) is exceeding the 65535 bytes limit". Is there any general advice on refactoring a lexer to get around this? (Or, dare I ask, should I be researching a tool that handles this sort of language more efficiently?)
>
> Thanks
> PSM
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address