[antlr-interest] Spaces in names
Gavin Lambert
antlr at mirality.co.nz
Wed Aug 8 13:49:15 PDT 2007
At 01:56 9/08/2007, Dave Raskin wrote:
>Gavin, thanks for the reply!
First of all, make sure you hit the "Reply to All" button so that
your reply goes to the list as well. You're more likely to get a
response that way.
>BTW, I am using C# generated code - could this be an issue?
No, this is a grammar problem, not a target problem.
>name_blank_name
>: NAME ( BLANK NAME )*
>;
That won't work, because BLANK tokens will still be getting
skipped (or if they're not, then it'll mess up other bits of your
grammar). As I said, if you want to skip whitespace most of the
time but treat it as significant sometimes, then you have to do it
in the lexer.
>I also tried changing the lexer rule, but got a compile error:
What was the error?
>NAME_BLANK_NAME
>: NAME ( BLANK NAME )*
>;
It's not quite as easy as that; since NAME can contain an
arbitrary number of characters, there's ambiguity between NAME and
NAME_BLANK_NAME. You'd have to write a hybrid type-changing rule
if you wanted to do it this way.
But as I said in my original message, if you're trying to
represent a string value then this is completely the wrong
approach, since it will collapse whitespace. ie. it would have no
way of representing "hello world" (two spaces) -- it would always
translate it into "hello world" (one space). Plus, the above
doesn't take leading or trailing space into account.
You need to make the lexer match an entire string (including the
quotes) as a whole. That way you can preserve the exact contents
of the string.
>Is there an example grammer you can point to, I haven't found one
>yet...
Have a look at examples/csharp/tinyc/lexer.g, specifically at the
CHAR_LITERAL and STRING_LITERAL rules. You basically want a
hybrid of the two, though you can probably simplify the escape
handling (only supporting escaping backslashes and single quotes;
though that depends on your particular dialect).
More information about the antlr-interest
mailing list