[antlr-interest] Sometimes significant new lines

[Nicholas Clare]

Hi,

Sorry, I read my email again, and it doesn't really make sense. It was
written very late last night. The semicolons aren't hidden, but new
lines are (since most of the time I don't want whitespace to be
significant). I want to allow either a semicolon, or a new line to
mark the end of the configItem. So, any of the following should be
valid.

one = 1; two = 2;

OR

one = 1
two = 2;

OR

one = 1;
two = 2;

I haven't tried using the idea you've laid out, and I will do now, but
I just wanted to clear up what I meant.

Thanks,
Nick

On Fri, May 8, 2009 at 4:37 AM, Indhu  Bharathi <indhu.b at s7software.com> wrote:
> Did you evaluate the option of making semicolon a default token itself instead of hidden token? Unless there is compelling reason to make semicolon hidden token, let's make it default token. That will make the work easy.
>
> Now assuming there is some compelling reason to make semicolon hidden token, you can access hidden tokens while parsing using input.LA(int) or input.LT(int). Former returns token type while later returns the token itself.
>
> Maybe you can write something like:
>
> configItem: ID '=' VALUE { input.LA(1)==SEMICOLON }?=> /*nothing*/
>
>
> Now your immediate question will be what if LA(1) is some other hidden token and LA(2) is semicolon? You can iterate through all hidden tokens and see if there is a semicolon before the next default token. You can extract it out into a new method for convenience.
>
> http://www.antlr.org/pipermail/antlr-interest/2009-April/033895.html
>
> The above link contains a sample of recognizing newline from hidden token. It is a small sample grammar which will recognize "foo<NEWLINE>" but reject "foo" where <newline> is hidden token.
>
> Hope that helps.
>
> Cheers, Indhu
>
>
> ----- Original Message -----
> From: Nicholas Clare <nickclare at gmail.com>
> To: antlr-interest at antlr.org
> Sent: Friday, May 8, 2009 5:24:45 AM GMT+0530 Asia/Calcutta
> Subject: [antlr-interest] Sometimes significant new lines
>
> Hi Everyone,
>
> I'm writing a simple parser for configuration files. Most of it is
> working beautifully, but there's one thing I can't figure out. The
> basic idea is that there are configuration items, the syntax of which
> are:
>
> configItem: ID '=' VALUE terminator
>
> (It's obviously a touch more complicated than that, but I don't want
> to bore you with details). The terminator rule is what I'm having
> trouble with. What I want is to allow either semicolons or new lines.
> The semicolons are easy. Since most of the time I don't want to worry
> about extra new lines, I'm currently sending them to the HIDDEN
> channel. Is there a way to detect them here (preferably without java
> actions, but that would be acceptable if there's no other option)?
>
> Thanks very much,
> Nick
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>