[antlr-interest] Optional spaces question

Kirby Bohling kirby.bohling at gmail.com
Thu Jan 19 07:36:53 PST 2012


On Wed, Jan 18, 2012 at 7:17 AM, Thomas Thomsen <thomas at t-t.dk> wrote:
> I am pretty new to ANTLR, doing a DSL language. I like ANTLR a lot, but I
> am struggling with a problem regarding optional whitespaces. My problem is
> that I need to distinguish between "f(x)" and "f  (x)" -- note the space
> between "f" and "(x)" in the latter (I am putting whitespace on the hidden
> channel, and I want to continue to do that). The former is a function call,
> the latter something different.
>
> I found a post on this list from 2007 ("Handling optional spaces") which
> addresses the exact same question. One suggestion was to have the lexer
> absorb the left parenthesis if there is no space in between:

You might try looking at the C/C++ Pre-processor grammar, it uses
states and context to accomplish this.  In the C Pre-processor:
#define FOO (...)

defines a macro which contains parenthesis.

#define FOO(...)

defines a macro which takes parameters.

See here:
http://www.antlr.org/grammar/1166665121622/Cpp.tar






>
> ID : ('a'..'z') + ;
> FUNCTION_CALL: ID '(' ;
>
> Then the lexer would return "f(" as a FUNCTION_CALL-token if there is not
> space in between. This works, but it is not too pretty and complicates
> things elsewhere in my code. The other suggestion was to check the hidden
> channel for whitespace-tokens by means of Java code (actually C# in my
> case). But since I am not yet too familiar with the inner workings of
> ANTLR, this scares me a bit.
>
> So I was thinking of a third strategy: Have a simple preprocessor look
> through the input file, and if a letter is directly followed by a left
> parenthesis, put some special character in between. So the preprocessor
> transforms "f(x)" into "f&(x)", where "&" is a (glue) character not used
> elsewhere in the grammar. And afterwards, it would be much easier to
> distinguish between "f&(x)" and "f  (x)" in ANTLR.
>
> Is this question or strategy completely stupid for some reason?
>
> Best regards, and thanks for all the good work on ANTLR,
>
> -Thomas Thomsen
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address


More information about the antlr-interest mailing list