[antlr-interest] Re: Representing state in lexer
cgodfrey86
cgodfrey at epnet.com
Wed May 14 08:50:32 PDT 2003
I can do a definition like the following, where a PATTERN token is
returned as either quoted or not, but I still have issues when
scanning gt. If gt appears after a quote (not necessarily immediately
after the quote), I want to return gt as type PATTERN.
PATTERN : QUOTE (ALLOWCHARS)+ QUOTE ;
TERM :
("gt")=> "gt"
{$setType(GT_OP);}
| (">")=> ">"
{$setType(GT_OP);}
|("ge")=> "ge"
{$setType(GE_OP);}
|(">=")=> ">="
{$setType(GE_OP);}
|("lt")=>"lt"
{$setType(LT_OP);}
|("<")=>"<"
{$setType(LT_OP);}
|("le")=>"le"
{$setType(LE_OP);}
|("<=")=>"<="
{$setType(LE_OP);}
|("eq")=>"eq"
{$setType(EQ_OP);}
|("=")=>"="
{$setType(EQ_OP);}
|("-")=>"-"
{$setType(DASH);}
| ("or") => "or"
{$setType(OR_OP);}
| ("and") => "and"
{$setType(AND_OP);}
| ("not") => "not"
{$setType(NOT_OP);}
|(('a'..'z')('a'..'z') WS) => ('a'..'z')('a'..'z')
{
$setType(S_TAG);
}
| ('w'INT)=>'w'INT
{$setType(W_OP);}
| ('n'INT)=>'n'INT
{$setType(N_OP);}
|
(ALLOWCHARS)+
{$setType(PATTERN);}
;
--- In antlr-interest at yahoogroups.com, "Anthony W Youngman"
<Anthony.Youngman at E...> wrote:
> What is it non-deterministic with? In other words, what other
construct
> recognised by the lexer could a PATTERN be confused with?
>
> You will have to define an unquoted pattern as a subset of that
other
> construct, and then use $settype to change the type to PATTERN if
that's
> what you've got.
>
> Cheers,
> Wol
>
> -----Original Message-----
> From: cgodfrey86 [mailto:cgodfrey at e...]
> Sent: 14 May 2003 15:56
> To: antlr-interest at yahoogroups.com
> Subject: [antlr-interest] Re: Representing state in lexer
>
>
> Thank you for the suggestion.
>
> The main issue that I am running into with this is that a PATTERN
> token can appear either inside or outside of quotes - so I won't
> recognize a PATTERN outside of quotes. When I try and define it
> without quotes, I have lexical nondeterminism
>
> PATTERN : QUOTE (ALLOWCHARS)+ QUOTE ;
>
>
>
>
>
>
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
>
>
>
> This transmission is intended for the named recipient only. It may
contain private and confidential information. If this has come to you
in error you must not act on anything disclosed in it, nor must you
copy it, modify it, disseminate it in any way, or show it to anyone.
Please e-mail the sender to inform us of the transmission error or
telephone ECA International immediately and delete the e-mail from
your information system.
>
> Telephone numbers for ECA International offices are: Sydney +61 (0)
2 9911 7799, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000
and New York +1 212 582 2333.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list