[antlr-interest] Line terminator.

mzukowski at yci.com mzukowski at yci.com
Thu Apr 10 08:21:43 PDT 2003


In this specific case you may be able to handle this with a syntactic
predicate in the lexer, but I don't have enough of your parser and lexer to
say for sure.
	
You may be better off using a filter type of approach.  See
http://www.codetransform.com/filterexample.html

Basically you lex everything without skipping newlines, then have a filter
on the token stream which checks and only keeps NEWLINE if it preceeds
CONTINUE (to force the parser to have a recognition error), otherwise
NEWLINES are discarded.

Monty

-----Original Message-----
From: Cesar Octavio López Nataren [mailto:cesar at ciencias.unam.mx]
Sent: Thursday, April 10, 2003 1:15 AM
To: antlr-interest at yahoogroups.com
Subject: [antlr-interest] Line terminator.


Hello, in my grammar the simbols:  \u000A, \u000D, \u2028, \2029 are called 
Line terminator. 

Now my problem is: 

There are some small cases where I don't want to ignore that a line 
terminator appears. 

Here is an example: 

ContinueStatement: continue [no LineTerminator here] Identifier_opt; 

By now, I had been ignoring these "corner cases" by just ignoring on the 
lexer those symbols with the very well known Token.SKIP but now I need more 
control. 

what do you suggest? 

Erasing the Token.SKIP from the lineTerminators lexing rules, and modifying 
all my parser rules to take care of lineTerminators? (looks like lot's of 
work for me). 

is possible a some kind of "lexical" predicates to make a solution for it? 

or? 


Cheers 


césar

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list