[antlr-interest] RE: lexical nondeterminism question

[mzukowski at yci.com]

Trey,
	Silvain had a good solution there.  You may also consider doing some
things in a later stage of processing too, in between the lexer and the
parser.  I was recently experimenting with a PICK BASIC parser and trying to
convert it to AREV.  A common idiom was "GOTO Exit" where Exit is a reserved
word in BASIC, and here it is also a label.
	So what I wanted to do was say "If I see something like a label
after a GOTO then force it to be a LABEL_REFERENCE."  Doing this in the
lexer was awkward because of having to deal with whitespace and the various
jump statements.  "GOTO" and "GO" "TO" are valid jumps.  Also the "ON X
GOSUB A,B,C" thing meant I had to deal with a comma separated list of
labels, not just one.
	Doing it in the parser lead to lots of ambiguities because labels
could be keywords and the GOTO and GO TO variants skew the possibilites at
k=2 and k=3 and ANTLR's linear approximate lookahead lead to many
ambiguities because of that.
	So I wrote a TokenFilter that used a parser to find if a jump was
happening and if so make the label into a LABEL_REFERENCE.  See the complete
example at http://www.codetransform.com/filterexample.html.  Please let me
know of any typos or if anything is unclear.

Monty



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/