[antlr-interest] Parsing the Keywords/Identifiers

Martin Probst mail at martin-probst.com
Wed Oct 19 15:20:04 PDT 2005


Hi,

> For example when writing a pl/sql parser, PASSWORD is a keyword and it
> can come as a column name too. Is there a way around for this?  Hope i
> am clear in explaining the problem. 

You have a keyword free language. The normal way to work around that is
to have a lexer that is stateful. E.g. if in SQL SELECT is not always a
keyword, so you can do:
SELECT SELECT FROM SELECT WHERE SELECT = 'SELECT'
You have to keep track of e.g. an operator/operand state:
first SELECT is an operator, afterwards the Lexer switches to the
operand state. The next SELECT is then recognised as an operand (because
of the state), and the Lexer switches back to operator sate, and so on.

This is somewhat annoying and messy, but at least it keeps the Parser
out of those lexical details. You might need to add additional Lexer
states if you language is more complex. If you can't resolve a problem
that way your language is simply ambiguous (at least not LL(1)) and
you're boned ;-)

Martin



More information about the antlr-interest mailing list