[antlr-interest] Diferent output when running ANTLRWorks IDE and a test rig in both Java and C# versions

[Hélder Silva]

This is my current grammar:

grammar Oven;

options 
{
    language=CSharp2;
//    language=Java;
    output=AST;
    ASTLabelType=CommonTree; // Cria arvore para ser utilizada por outro
parser a executar funções
}

@lexer::members
{
	bool emailMode = false;
}
@members 
{
//to create more understandable error phrases for users
Stack paraphrases = new Stack();

public override string GetErrorMessage(RecognitionException e,
string[] tokenNames)
{
	IList stack = GetRuleInvocationStack(e, this.GetType().ToString());
	string msg = null;
	if ( e.GetType()==typeof(NoViableAltException) ) 
	{
		NoViableAltException nvae = (NoViableAltException)e;
		msg = " no viable alt; token="+e.Token+
		" (decision="+nvae.decisionNumber+
		" state "+nvae.stateNumber+")"+	
		" decision=<<"+nvae.grammarDecisionDescription+">>";
	}
	else 
	{
		msg = base.GetErrorMessage(e, tokenNames);
	}
//	return stack+" "+msg;
	for(int i=0;i<stack.Count;i++)
		msg = msg+" at "+stack[i];
	for(int i=0;i<paraphrases.Count;i++ ) 
	{
		String paraphrase = (String)paraphrases.Peek();
		msg = msg+" "+paraphrase;
	}
	return msg;
}

//To stop parsing after first error happens!!
protected override void Mismatch(IIntStream input, int ttype, BitSet follow)
//throws RecognitionException
{
	throw new MismatchedTokenException(ttype, input);
}

public override object RecoverFromMismatchedSet(IIntStream input,
RecognitionException e,
BitSet follow)
//throws RecognitionException
{
	throw e;
}

protected override object RecoverFromMismatchedToken(IIntStream input, int
ttype, BitSet follow)
{
    	throw new MismatchedTokenException(ttype,input);
}

}

// Alter code generation so catch-clauses get replace with
// this action.
@rulecatch
{
catch(RecognitionException e)
{
	throw e;
}
}

program:	command+;

command
@init { paraphrases.Push("in command"); }
@after { paraphrases.Pop(); }
	:	cmd_set
	|	cmd_turn
	|	cmd_wait
	|	cmd_sync
	|	cmd_email
	|	cmd_log
	;
cmd_set		
@init { paraphrases.Push("set"); }
@after { paraphrases.Pop(); }
	: SET VALVE TO? INT FLUX TERMINATOR -> ^(SET VALVE INT)
	| SET ZONE a=INT TO b=INT DEGREES? TEMPUNITS TERMINATOR -> ^(ZONE $a
$b TEMPUNITS)
	;
cmd_turn	
@init { paraphrases.Push("turn"); }
@after { paraphrases.Pop(); }
	: TURN VALVE STATE TERMINATOR -> ^(TURN VALVE STATE)
	| TURN ZONE INT STATE TERMINATOR -> ^(TURN ZONE INT STATE)
	;
cmd_wait
@init { paraphrases.Push("wait"); }
@after { paraphrases.Pop(); }
	: WAIT FOR INT TIMEUNITS TERMINATOR -> ^(WAIT INT TIMEUNITS)
	| WAIT UNTIL ZONE a=INT TEMPERATURE LESS_THAN b=INT DEGREES
TEMPUNITS TERMINATOR -> ^(WAIT ZONE $a LESS_THAN $b TEMPUNITS)
	| WAIT UNTIL ZONE a=INT TEMPERATURE GREATER_THAN b=INT DEGREES
TEMPUNITS TERMINATOR -> ^(WAIT ZONE $a GREATER_THAN $b TEMPUNITS)
	| WAIT UNTIL ZONE a=INT TEMPERATURE EQUAL b=INT DEGREES TEMPUNITS
TERMINATOR -> ^(WAIT ZONE $a EQUAL $b TEMPUNITS)
	;
cmd_sync
@init { paraphrases.Push("sync"); }
@after { paraphrases.Pop(); }
	: SYNC ZONES TERMINATOR -> ^(SYNC ZONES)
	;
cmd_email
@init { paraphrases.Push("email"); }
@after { paraphrases.Pop(); }
	: EMAIL MAILADDRESS MESSAGE TERMINATOR -> ^(EMAIL MAILADDRESS
MESSAGE) 
	;
cmd_log	
@init { paraphrases.Push("log"); }
@after { paraphrases.Pop(); }	
	: TURN LOG STATE TERMINATOR -> ^(TURN LOG STATE)
	;	

// START:tokens
SET 	:	'set';
TURN	:	'turn';
WAIT	:	'wait';
EMAIL	:	'email' {emailMode=true;};
SYNC	:	'sync';
TO	:	'to';
DEGREES	:	'deg'|'degrees';
FOR	:	'for';
UNTIL	:	'until';
ZONE	:	'zone';
ZONES	:	'zones';
TEMPERATURE
	:	'temperature';
LOG	:	'log';

STATE	:	'on'
	|	'off'
	;
FLUX	:	'sccm';
TIMEUNITS 
	:	('sec'|'seconds')
	|	('min'|'minutes')
	|	('hr'|'hours')
	;
TEMPUNITS
	:	('ºC'|'C'|'ºc'|'c')
	|	('ºF'|'F'|'ºf'|'f')
	;
LESS_THAN
	:	'<';
GREATER_THAN
	:	'>';
EQUAL	:	'=';

VALVE:'helium'|'ethylene'|'hydrogen';
MAILADDRESS:{emailMode}?=>('a'..'z'|'A'..'Z'|'0'..'9'|'.'|'_')+ '@'
('a'..'z'|'A'..'Z'|'0'..'9'|'.'|'_')+;
INT:   ('0'..'9')+ ;
MESSAGE	
@init{emailMode=false;}
	:	'"' ((options{greedy=false;}: .)*) '"';
WS  : (' '|'\r'|'\t'|'\u000C'|'\n') {$channel=HIDDEN;};
COMMENT	:	 '\'' ~('\'') '\'' {$channel=HIDDEN;};
LINE_COMMENT : '\'' ~('\n'|'\r')* '\r'? '\n' {$channel=HIDDEN;};
TERMINATOR: ';';
// END:tokens


For input such as:

set helium to 220 sccm;
on off turn helium on;
turn helium off;

The C# parser doesn’t generate a NoViableAltException, which is what happens
in ANTLRWorks. All commands after first erroneous one are disregarded
also...
During C# code debug, I can see it fail to match any command after the first
one (because token stream is wrong), and just exit without exception,
apparently not consuming all of the tokens available.
Lexer seems to be doing its job correctly I think...
I've tried to comment some rules out so I could isolate the ambiguous lexer
rule, but I can't see why this is happening here...


-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Jim Idle
Sent: sexta-feira, 8 de Abril de 2011 17:14
To: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Diferent output when running ANTLRWorks IDE
and a test rig in both Java and C# versions

Well the hint is that your grammar is wrong.

But you need to post your grammar to get more help :-). I think, from the
way you describe your problem, that perhaps you have ambiguous lexer rules
and think that the parser influences what lexer rule is run, which it does
not.

Also, make sure you run the debugger in AntrlWorks and check any warning
messages when you generate the code.

Jim



> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest- 
> bounces at antlr.org] On Behalf Of Hélder Silva
> Sent: Friday, April 08, 2011 3:46 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] Diferent output when running ANTLRWorks IDE 
> and a test rig in both Java and C# versions
>
> I’m getting different results when i run a certain input on the IDE in 
> comparison to a test rig in either Java or C# as well.
>
>
>
> In the IDE, for a given input, I get a NoViableAltException, which is 
> what I want to happen.
>
>
>
> When running the same input on Java test rig I get this:
>
> line 1:3 mismatched character ' ' expecting '@'
>
> line 1:4 required (...)+ loop did not match anything at input 'helium'
>
> The “expecting ‘@’” i suppose the parser is looking for another token 
> I have in which it matches an email address, but no rule I have starts 
> with an email address either.
>
>
>
> In the C# test rig, the parser complains there is no input with the
> message:
>
> line 1:4 required (...)+ loop did not match anything at input 'helium'
>
> can someone give me a hint on what’s happening here?
>
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address