[antlr-interest] Context-sensitive lexer
jonas.hagmar at gmail.com
jonas.hagmar at gmail.com
Fri Jun 17 05:30:56 PDT 2011
The parser still gets into an infinite loop if I enable backtracking.
Best Regards,
Jonas
On Jun 17, 2011 2:23pm, "Strobl, Robert"
<Robert.Strobl at student.hpi.uni-potsdam.de> wrote:
> Have you tried to enable the backtracking?
> Best regards,
> Robert
> Am 17.06.2011 um 14:15 schrieb Jonas:
> > Hi,
> >
> > I'm developing a parser for a file format where context is very
> > important. I'm looking to
> > 1) understand why my ANTLR parser gets into infinite loops
> > 2) find out if there is any better way to implement context
> > sensitivity than what I am doing with semantic predicates.
> >
> > A typical beginning of a file looks like this:
> > TITLE
> > some title text
> >
> > SECTION1
> > a=b*c
> > END
> >
> > SECTION2
> > ...
> >
> > SECTION3
> > ...
> >
> > The syntax differs from section to section; the 'TITLE' section is
> > terminated by the newline after the title text line, while other
> > sections can eg use single quote string literals and be terminated
> > by a keyword like 'END'. Here is a sample grammar, that gets into an
> > infinite loop:
> >
> > grammar test;
> >
> > options {
> > output=AST;
> > }
> >
> > @lexer::members {
> > static final int STATE_AT_BEGINNING = 0;
> > static final int STATE_IN_TITLE = 1;
> > static final int STATE_AFTER_TITLE = 2;
> > int lexerState = STATE_AT_BEGINNING;
> > }
> >
> > file : title;
> >
> > title : BEGIN_TITLE TITLE_TEXT END_TITLE;
> >
> > BEGIN_TITLE
> > : {(lexerState == STATE_AT_BEGINNING)}? 'TITLE' WS_NL
> > {lexerState=STATE_IN_TITLE;}
> > ;
> >
> > TITLE_TEXT
> > : {lexerState == STATE_IN_TITLE}? TEXT
> > ;
> >
> > END_TITLE
> > : {lexerState == STATE_IN_TITLE}? NL {lexerState=STATE_AFTER_TITLE;}
> > ;
> >
> > BLANK_ROW
> > : {!(lexerState == STATE_IN_TITLE)}? WS_NL
> > ;
> >
> > REMARK : {!(lexerState == STATE_IN_TITLE)}? 'REMA' .* NL
> > ;
> >
> > fragment
> > WS_NL : (' ' | '\t')* NL;
> >
> > fragment
> > NL : '\r'? '\n';
> >
> > fragment
> > TEXT : (~('\r' | '\n'))*;
> >
> > Best Regards,
> > Jonas
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
More information about the antlr-interest
mailing list