[antlr-interest] Sometimes significant new lines

[Indhu Bharathi]

Did you evaluate the option of making semicolon a default token itself instead of hidden token? Unless there is compelling reason to make semicolon hidden token, let's make it default token. That will make the work easy.

Now assuming there is some compelling reason to make semicolon hidden token, you can access hidden tokens while parsing using input.LA(int) or input.LT(int). Former returns token type while later returns the token itself.

Maybe you can write something like:

configItem: ID '=' VALUE { input.LA(1)==SEMICOLON }?=> /*nothing*/


Now your immediate question will be what if LA(1) is some other hidden token and LA(2) is semicolon? You can iterate through all hidden tokens and see if there is a semicolon before the next default token. You can extract it out into a new method for convenience. 

http://www.antlr.org/pipermail/antlr-interest/2009-April/033895.html

The above link contains a sample of recognizing newline from hidden token. It is a small sample grammar which will recognize "foo<NEWLINE>" but reject "foo" where <newline> is hidden token.

Hope that helps.

Cheers, Indhu


----- Original Message -----
From: Nicholas Clare <nickclare at gmail.com>
To: antlr-interest at antlr.org
Sent: Friday, May 8, 2009 5:24:45 AM GMT+0530 Asia/Calcutta
Subject: [antlr-interest] Sometimes significant new lines

Hi Everyone,

I'm writing a simple parser for configuration files. Most of it is
working beautifully, but there's one thing I can't figure out. The
basic idea is that there are configuration items, the syntax of which
are:

configItem: ID '=' VALUE terminator

(It's obviously a touch more complicated than that, but I don't want
to bore you with details). The terminator rule is what I'm having
trouble with. What I want is to allow either semicolons or new lines.
The semicolons are easy. Since most of the time I don't want to worry
about extra new lines, I'm currently sending them to the HIDDEN
channel. Is there a way to detect them here (preferably without java
actions, but that would be acceptable if there's no other option)?

Thanks very much,
Nick

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address