Fwd: [antlr-interest] Lexer not as smart as it should be?

[Martin Probst]

> This seems to work as well(atleast no non-determinism error). As antlr
> generates codes, which itself is means that there is no
> non-determinism.

That is not correct. ANTLR will generate code even if there are non-
determinisms. That's why they are warnings, not errors. The code might
still not work as you expect it to.

> However, when i give an input i get an exception that is in bold. It
> can be fixed by adding a condition for WS.

Well, that's not surprising, is it? It encounters a character (0xA aka
'\n' or the line terminator) which it doesn't know how to handle.

> From what i understand syntactic predictes slow down the
> lexer .Correct me if i am wrong.

Syntactic predicates are indeed slow, as they currently require the
Parser to throw an exception while parsing, rewind to the place where it
started guessing and start again with the next alternative.

However, what you have ( if (int(...) < )...) is not a syntactic but a
semantic predicate. That's just a posh name for host-code injected into
the parser at certain places. Depending on what you do, that will be
slow or fast. E.g. calculating 1.000.000 digits of Pi will be slow, but
a single integer comparison is usually quite fast ;-)

Martin