[antlr-interest] Optional spaces question

Gokulakannan Somasundaram gokul007 at gmail.com
Wed Jan 18 05:53:32 PST 2012


Hi,
   First of all, you cannot have a lexer rule inside another lexer rule on
the left. Either you need to make ID as a fragment lexer rule / make the
FUNCTION_CALL as a parser rule.
   Remember, even if there is a skip rule for spaces, it won't skip the
space that occurs in between fragment lexer rules inside a lexer rule. So
following the first suggestion, if ID is a fragment lexer rule, you can
write

FUNCTION_CALL : ID ( SPACE { /* do whatever you want here*/ } )*  '(';

Hope it helps.

Gokul.

On Wed, Jan 18, 2012 at 9:17 PM, Thomas Thomsen <thomas at t-t.dk> wrote:

> I am pretty new to ANTLR, doing a DSL language. I like ANTLR a lot, but I
> am struggling with a problem regarding optional whitespaces. My problem is
> that I need to distinguish between "f(x)" and "f  (x)" -- note the space
> between "f" and "(x)" in the latter (I am putting whitespace on the hidden
> channel, and I want to continue to do that). The former is a function call,
> the latter something different.
>
> I found a post on this list from 2007 ("Handling optional spaces") which
> addresses the exact same question. One suggestion was to have the lexer
> absorb the left parenthesis if there is no space in between:
>
> ID : ('a'..'z') + ;
> FUNCTION_CALL: ID '(' ;
>
> Then the lexer would return "f(" as a FUNCTION_CALL-token if there is not
> space in between. This works, but it is not too pretty and complicates
> things elsewhere in my code. The other suggestion was to check the hidden
> channel for whitespace-tokens by means of Java code (actually C# in my
> case). But since I am not yet too familiar with the inner workings of
> ANTLR, this scares me a bit.
>
> So I was thinking of a third strategy: Have a simple preprocessor look
> through the input file, and if a letter is directly followed by a left
> parenthesis, put some special character in between. So the preprocessor
> transforms "f(x)" into "f&(x)", where "&" is a (glue) character not used
> elsewhere in the grammar. And afterwards, it would be much easier to
> distinguish between "f&(x)" and "f  (x)" in ANTLR.
>
> Is this question or strategy completely stupid for some reason?
>
> Best regards, and thanks for all the good work on ANTLR,
>
> -Thomas Thomsen
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>


More information about the antlr-interest mailing list