[antlr-interest] set max number of characters in a string literal

[Gavin Lambert]

At 15:48 12/09/2008, Olya Krachina wrote:
 >I am working on a lexer and i was wondering how i could set a 
max
 >limit on the number of characters that make up a string literal, 

 >i.e. it is valid when there are n (let's say n = 20) or less 
chars.
 >I tried setting lookahead to 20 (options k = 20) but it did not
 >have any effect. I am using antlr 2.7.

If you need to parse exactly 20 chars and stop dead (eg. for 
fixed-width data formats), you'll need to spell it out explicitly 
-- eg. repeat CHAR 20 times.  (ANTLR only supports cardinalities 
for zero, one, or many.)

Of course you can be a bit more clever about it, eg. making a rule 
that contains five CHARs and then use that rule four times, etc.


If your input language isn't actually ambiguous, though, and you 
just want to do this for validation purposes, then your best bet 
is to just successfully match however many characters happen to 
appear (even if more than 20), and use a semantic action in the 
parser or tree parser to validate the length after the fact.

Generally speaking the lexer should be built to be as tolerant as 
it possibly can -- wait until parsing time to detect and report 
errors.