[antlr-interest] Manipulating lexer text output
Jim Idle
jimi at temporal-wave.com
Tue Apr 3 14:37:45 PDT 2007
Well, perhaps, but the lexer is just looking at one character at a time
anyway and does not produce a string with that stuff in it unless you
ask for it - if you don't then you just get a token that says where the
start and stop is. So it isn't overly inefficient and if you set the
text explicitly then there is probably little difference in execution
time.
Jim
-----Original Message-----
From: Gavin Lambert [mailto:antlr at mirality.co.nz]
Sent: Tuesday, April 03, 2007 2:00 PM
To: Jim Idle; antlr-interest at antlr.org
Subject: RE: [antlr-interest] Manipulating lexer text output
At 06:54 4/04/2007, Jim Idle wrote:
>Yeah - this has come up a lot and really the solution at the
moment
>is to do this in the parser. However you could at least do this
in
>the STRING rule with a call to a small function that removed
this stuff
>and then emit the token with the result as the text.
That'd still require double-parsing the input though (once via
ANTLR's normal processing, and once after the fact to resolve
escape sequences). Which makes it no better than doing it outside
ANTLR entirely.
More information about the antlr-interest
mailing list