[antlr-interest] Not processing a newline always

[Cliff Hudson]

Since you are operating much like an immediate-mode interpreter, probably
the way to do this is to write a special token stream which provides a set
of tokens to the parser when you have given each line (basically tokenize
each line separately.)

 

ANTLR’s parser uses a pull model to get tokens from the token stream, and
you are probably using a push model to provide strings to the lexer for
tokenization, so you will have to handle that hookup yourself.

 

From: Francesc d'Assis Massanés [mailto:f.massanes at gmail.com] 
Sent: Tuesday, April 27, 2010 5:41 PM
To: Cliff Hudson
Cc: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Not processing a newline always

 

I will be feeding the lines one by one and expecting to be processed when a
full instruction is computed (not after or before).

 

I never want the whitespace to be processed, it is the newlines that I am
worried about. But I have "fix" the problem using:

 
 expression3:  expression4 ( ('+'|'-') (NEWLINE)? expression4)* ;

I know is not a good solution but works, and then I will code my ast tree to
disregard the token newline ....


Francesc Massanés



On Tue, Apr 27, 2010 at 7:12 PM, Cliff Hudson <cliff.s.hudson at gmail.com>
wrote:

Are you sending expressions to ANTLR a line at a time, or are you sending a
while 'file' which would be complete?  Do you have a lexer rule which
consumes whitespace?  Under what situations do you not want whitespace
ignored?


-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Francesc d'Assis
Massanés
Sent: Monday, April 26, 2010 10:04 AM
To: antlr-interest at antlr.org
Subject: [antlr-interest] Not processing a newline always

Hello,

I am trying to program a "complex" math program and to process the user
input I have been working in a grammar in antlr.

At this point it works almost as I want it to work but I am missing
something: I do not want to process newlines always.

Let me explain it better: when an expression is finished ( 4+5 ) you can
find a semi-colon or a newline, in any case you just finished this
expression.
But what happens if the expression is incomplete: ( 4+5+ ) then if you find
a semi-colon you have an error (you should not be able to parse this input)
but if you find a newline you should look the next line to follow
processing.

What I want is a rule like: Newline is HIDDEN if some other token is need to
finish the actual rule.

I do not know if it is even possible to do this with antlr or I shall go to
modify the output code.

Thanks,
Francesc Massanés

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address