[antlr-interest] Antlr 3 Lexer problem
Geoffrey Zhu
gzhu at peak6.com
Tue Jun 26 11:43:47 PDT 2007
I can't do this. I am constructing complicated lexer rules because in my
input there are code blocks within which the grammar is totally
different from the rest of the file. My file is a custom mini-language.
Embeded in it there are some SQL statements with surrounding
parentheses. My program does not need to deal with full SQL syntax. I
only need to correctly recognize the beginning and ending of the SQL
statements.
What I am thinking is to match one code block with one single token, as
ANTLR did with the "action" token (
http://fisheye2.cenqua.com/browse/antlr/src/org/antlr/tool/antlr.g?r=324
4#l1186 ). Below is just a simplified example to show that the lexer
seems to choke...
I just need to figure out how to make it not to choke.
-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Jim Idle
Sent: Tuesday, June 26, 2007 1:32 PM
To: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Antlr 3 Lexer problem
Don't try to construct syntactical context in the lexer basically:
grammar T;
options {
language=CSharp;
backtracking=true;
}
ID : ('a'..'z')+;
LPAREN : '(';
SELECT : 'select';
prog: (ID | LPAREN SELECT?)+ ;
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Geoffrey Zhu
> Sent: Tuesday, June 26, 2007 11:28 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] Antlr 3 Lexer problem
>
> Hi,
>
> I have a simple lexer problem. The below grammar will fail on "(
> security" in the lexer, because when the lexer sees '(' and then sees
> 's' , it will try to match the LP_SELECT rule, which cannot be
matched.
> In this case, I want the lexer to return LPAREN and then ID. How can I
> do that?
>
> Thanks a lot,
> Geoffrey
>
>
> grammar T;
>
> options {
> language=CSharp;
> backtracking=true;
> }
>
> ID : ('a'..'z')+;
>
> LPAREN : '(';
>
> LP_SELECT : LPAREN 'select';
>
> prog: (ID | LPAREN | LP_SELECT)+ ;
>
> _______________________________________________________
>
> The information in this email or in any file attached hereto is
> intended only for the personal and confiden- tial use of the
> individual or entity to which it is addressed and may contain
> information that is propri- etary and confidential. If you are not
> the intended recipient of this message you are hereby notified that
> any review, dissemination, distribution or copying of this message is
> strictly prohibited. This communica- tion is for information
> purposes only and should not be regarded as an offer to sell or as a
> solicitation of an offer to buy any financial product. Email trans-
> mission cannot be guaranteed to be secure or error- free. P6070214
More information about the antlr-interest
mailing list