[antlr-interest] JavaScript grammar

[Patrick Hulsmeijer]

The function promoteEOL checks the current token and when it is not
SEMIC, EOF, RBRACE or already EOL it scans backwards evaluating the
off-channel tokens up to the previous (on-channel) token. When it
encounters an EOL token that is off-channel it sets its channel to
DEFAULT_CHANNEL and repositions the input stream on it.
The current token can already be EOL (on-channel) when the promoteEOL
function is called previously from e.g. the return statement:

returnStatement
	: RETURN^ { promoteEOL(null); } expression? semic!
	;

This will result in a AST that is compliant to the ECMAScript 3
specification without actually inserting a semicolon token in the token
stream.

Regards,
patrick.

-----Original Message-----
From: Johannes Luber [mailto:jaluber at gmx.de] 
Sent: maandag 31 maart 2008 15:12
To: Patrick Hulsmeijer
Cc: antlr-interest at antlr.org
Subject: Re: [antlr-interest] JavaScript grammar

Patrick Hulsmeijer schrieb:
> Chris,
> 
> For the semicolon insertion issue I've taken the following approach. 
>  The line terminators are left on the hidden channel. In the parser
the 
> semicolon (e.g. in statements) is defined as a rule. In this rule I
scan 
> the token stream for line terminators and promote the first
encountered 
> to the default channel. Line terminator are also an alternative in
this 
> rule. Something like this:
> 
> semic:
> @init
> {
>                 int marker = input.mark();
>                 promoteEOL();
> }
>                 : SEMIC
>                 | EOL
>                 | RBRACE { input.rewind(marker); }
>                 | EOF
>                 ;
>  

Can you elaborate more on the function promoteEOL()?

Johannes