[antlr-interest] Antlr 3 Lexer problem

[Geoffrey Zhu]

The syntactic predicate does not seem to work. The lexer chokes on
exactly the same location 'c' if I pass in "( security".


In mTokens() it still looks ahead only one step to determine what should
be the next token.

  else if ( (LA3_0=3D=3D'(') ) {  //if it sees a '(' followed by s', it will
try to use rule mLP_SELECT()
            int LA3_2 =3D input.LA(2);

            if ( (LA3_2=3D=3D'\t'||LA3_2=3D=3D' '||LA3_2=3D=3D's') ) {
                alt3=3D3;
            }
            else {
                alt3=3D2;}
        }

//mLP_SELECT simply calls mLSELECT and returns. 
  public final void mLP_SELECT() throws RecognitionException {
        try {
            int _type =3D LP_SELECT;
            // C:\\Documents and Settings\\gzhu\\My
Documents\\T.g:13:13: ( ( LSELECT )=3D> LSELECT )
            // C:\\Documents and Settings\\gzhu\\My
Documents\\T.g:13:13: ( LSELECT )=3D> LSELECT
            {
            mLSELECT(); if (failed) return ;

            }

            this.type =3D _type;
        }
        finally {
        }
    } 

 

-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Jim Idle
Sent: Tuesday, June 26, 2007 1:52 PM
To: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Antlr 3 Lexer problem



> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest- 
> bounces at antlr.org] On Behalf Of Geoffrey Zhu
> Sent: Tuesday, June 26, 2007 11:44 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Antlr 3 Lexer problem
> 
> I can't do this. I am constructing complicated lexer rules because in 
> my input there are code blocks within which the grammar is totally 
> different from the rest of the file. My file is a custom
mini-language.
> Embeded in it there are some SQL statements with surrounding 
> parentheses. My program does not need to deal with full SQL syntax. I 
> only need to correctly recognize the beginning and ending of the SQL 
> statements.

OK:

fragment WS : ' ' | '\t' ;
fragment LSELECT : '(' WS* 'select' ;
LPAREN    : '(' ;
LP_SELECT : (LSELECT)=3D> LSELECT;


Should do it. ANTLR
Perhaps you can tokenize the whole SQL statement to one token, unless
you need to look in it for host variables or something?

Jim




_______________________________________________________=0A=
=0A=
The  information in this email or in any file attached=0A=
hereto is intended only for the personal and confiden-=0A=
tial  use  of  the individual or entity to which it is=0A=
addressed and may contain information that is  propri-=0A=
etary  and  confidential.  If you are not the intended=0A=
recipient of this message you are hereby notified that=0A=
any  review, dissemination, distribution or copying of=0A=
this message is strictly prohibited.  This  communica-=0A=
tion  is  for information purposes only and should not=0A=
be regarded as an offer to sell or as  a  solicitation=0A=
of an offer to buy any financial product. Email trans-=0A=
mission cannot be guaranteed to be  secure  or  error-=0A=
free. P6070214