[antlr-interest] Re: Syntactic predicates question
shmuel siegel
antlr at shmuelhome.mine.nu
Tue Jan 31 00:33:24 PST 2006
Artem Dmytrenko wrote:
> match rule 'A' so (1) should fail and not consume the first character of
> the string. Shouldn't ANTLR examine at least k characters (in my case
> k=2, so it should be looking at 'A' and '1') from input stream before
> making a decision about which token matched? The generated code for
> matching 'A' in lexer is as follows:
>
> if ((LA(1)=='A') && (true)) {
> match('A');
> }
>
> Shouldn't it be something similar to the following?
>
> if ((LA(1)=='A') && (LA(2)==END_OF_TOKEN) {
> match('A');
> }
>
Why would you expect a lexer to look for more characters than was asked
for. "A" is a valid ActionToken so when the lexer saw "A", it returned
the "A". I would have called it an error if it had done anything else.
If instead of ActionToken you used ActionToken WS in your predicate, or
whatever could legally terminate ActionToken, the lexer should have
returned the token that you wanted. There is nothing in your description
that says that A of A12345 is not a valid ActionToken.
At least, this is how I understand lexers in general. Can someone with
more knowledge verify or reject my assertion.
Shmuel
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.375 / Virus Database: 267.14.25/246 - Release Date: 1/30/2006
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.375 / Virus Database: 267.14.25/246 - Release Date: 1/30/2006
More information about the antlr-interest
mailing list