[antlr-interest] Problem Lexer

David Johnson cdcttr at gmail.com
Fri Jan 16 20:58:33 PST 2009


I am working on a lexer where I need to know the context of the input to
determine the proper tokens to generate.

While attempting to solve the problem, my lexer triggered what appears to be
a never ending recursive call - eventually exiting with an Out of Memory
exception.

I ripped out the problem parts and have included it below.  This lexer does
the same thing.  I am running this using antlr 3.1.1

Thanks for any help.

-Dave

--- Example Test Case ---

    public static void main(String [] args) {
        CharStream stream = new ANTLRStringStream("|small");
        error e = new error(stream);

        Token t;
        while((t=e.nextToken()) != Token.EOF_TOKEN) {
            System.out.println(t);
        }
    }

--- output from test case ---

line 1:0 rule CHECK failed predicate: {check}?
line 1:0 rule CHECK failed predicate: {check}?
line 1:0 rule CHECK failed predicate: {check}?
line 1:0 rule CHECK failed predicate: {check}?
line 1:0 rule CHECK failed predicate: {check}?
line 1:0 rule CHECK failed predicate: {check}?
line 1:0 rule CHECK failed predicate: {check}?
.... (forever)

--- error.g ---

lexer grammar error;

@members {
    boolean check = false;
}

fragment Letter
    :    'A'..'Z'|'a'..'z';

fragment Number
    :    '1'..'9'|'0';

fragment Id
    :    Letter(Letter|Number|'_'|'-')+;

fragment Eq
    :    '=';

fragment Pipe
    :    '|';

NOT_CHECK:
    {!check}?=> Pipe Id Eq {
    }
    {!check}?=> Pipe Id {
    }
    {!check}?=> p=Pipe {
    } ;

CHECK:
    {check}?=> ~(' ' | ']')+ ;
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090116/52ea5216/attachment.html 


More information about the antlr-interest mailing list