[antlr-interest] distinguish "3 + 4" from "3 +4"

[Andreas Liebig]

Thank you for the reply, Sam.
Is this the int/float wiki article you mentioned: http://www.antlr.org/wiki/pages/viewpage.action?pageId=3604497 ("How can I emit more than a single token per lexer rule?")?
Unfortunately I cannot see how it helps in my case. The wiki article uses the difference in non-whitespace characters, whereas I have to make use of the hidden whitespace characters. I read somewhere that it is actually possible to check the hidden channel if necessary, but I cannot find any details.

One more example for my situation:
"3+4" should be parsed as NUMBER PLUS NUMBER, the same as "3 + 4".

Still looking forward to more suggestions.
Andreas



----- Original Message -----
From: Sam Barnett-Cormack <s.barnett-cormack at lancaster.ac.uk>
To: Andreas Liebig <liebigandreas at yahoo.com>
Cc: "antlr-interest at antlr.org" <antlr-interest at antlr.org>
Sent: Wednesday, October 5, 2011 1:20 PM
Subject: Re: [antlr-interest] distinguish "3 + 4" from "3 +4"

On 05/10/2011 12:14, Andreas Liebig wrote:
> Hello,
> I am not very experienced with ANTLR, and I would like to ask for some ideas how to solve this task:
> 
> I have to distinguish input streams like
> "3 + 4" (parsed as three tokens NUMBER PLUS NUMBER) from
> "3 +4" (parsed as NUMBER NUMBER, because the + is part of the number +4).
> 
> I would like to ignore whitespace in general using the "$channel=HIDDEN;" syntax. But only in this situation whitespace does matter.
> Can you guide me to a good explanation of a possible solution?

Don't really know what docs are where, but off the top of my head... you need to have your NUMBER lexer rule start with an optional +, or presumably actually + or -, so a human-readable version of the grammar would have something like

NUMBER : ('+'|'-')? DIGIT+;
PLUS : '+';
MINUS : '-';

Of course, ANTLR won't like that, because it's ambiguous. There are a few ways to resolve the ambiguity, you'll see one if you look on the wiki about how to differentiate between ints and floats in the lexer. I can't remember the syntax off the top of my head.

Sam