[antlr-interest] Line continuation character problem

[Anthony Youngman]

Okay.
 
Define a lexer rule as '&' 'linefeed', and mark it with token type
"skip". That should get round your original continuation line problem
(unless you can split in the middle of a token). I do note you say
you've solved this, though ...
 
And look at the example in the manual of lexing ESCape characters inside
of a character string. You'll probably need a different lexer rule
(almost identical to the one above, though). But if this rule is
triggered inside a string, you want to finish it with something like
"token.setlength( token.getlength() - 3)" which will lose the &\r\n
that's just been added ...
 
Cheers,
Wol

________________________________

From: Craig Barker [mailto:cb at phoebussoftware.com] 
Sent: 10 June 2004 10:10
To: antlr-interest at yahoogroups.com
Subject: RE: [antlr-interest] Line continuation character problem


Hi Folks,
 
I have a small problem that, although trawling all the manuals, examples
etc, i can't find an example of (although i'm sure i've read about it
somewhere!).
 
I'm trying to analyse a language where each statement appears on a
single line, apart from when the '&' character is explicitly used as a
line continuation character. 
 
A statement like:
 
and(a>1,b<2) can also be written
 
and (a>1,&
b<2)
 
The only other issue is that a string token can actually be validly
split by '& \r \n' so we may have:
 
and(a>"abc",b<"def")
 
written as:
 
and(a>"abc",b<"d&
ef")
 
 
I've tried using a lexer filter and that works fine for the first
example but either i'm incorrectly using it in the second example or as
the lexer will already be partway through lexing the string (done as per
the default java grammer in the distro.) it cannot invoke the filter
rule.
 
Is the best way of dealing with this a change in the lexing class to
automatically filter out any '&\r\n''s from the character stream before
it hits the lexer?
 
If there's another way or if anyone can point me to an example I would
be most grateful.
 
Many thanks
 
Craig


****************************************************************************

This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system.

Telephone numbers for ECA International offices are: Sydney +61 (0)2 8272 5300, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333.

****************************************************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20040610/6dc09498/attachment.html