[antlr-interest] Newbie: Token/Rule Difference [again..]

Trey Cool Shonuff trey_shonuff at freemail.hu
Fri May 23 09:03:28 PDT 2008


Hi,
yes indeed it works this way, thank you very much!
I will however go and dig deeper into the actual Java Parser/Lexer code ANTLR generates to get a better understanding of how things really work.
You gave me a good kickstart!
Cheers,
-Trevor.

> -----Original Message-----
> From: Andreas Bartho [mailto:andreas.bartho at inf.tu-dresden.de] 
> Sent: 2008. május 23. 17:18
> To: Trey Cool Shonuff
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Newbie: Token/Rule Difference [again..]
> 
> Hi,
> 
> in your first grammar a single letter is matched by LETTER, 
> i.e. the parser receives the LETTER token, but expects the 
> WORD token, which causes the error. As you probably do not 
> want LETTER to be a token but only a helper rule, try adding 
> the fragment keyword. The same holds for
> digit:
> 
> grammar G1;
> q	:	WORD;
> 
> fragment
> LETTER 	:	('a'..'z' | 'A'..'Z');
> 
> fragment
> DIGIT 	:	'0'..'9';
> 
> WORD	:	(DIGIT | LETTER)+;
> 
> This should work
> 	Andreas
> 
> 
> Trey Cool Shonuff wrote:
> > Hi all,
> > I'm totally new to ANTLR and the last time I've really 
> fiddled with language processing was back in college.
> > Please help me tell why the following two, similar grammars 
> behave differently:
> > 
> > grammar G1;
> > q	:	WORD;
> > LETTER 	:	('a'..'z' | 'A'..'Z');
> > DIGIT 	:	'0'..'9';
> > WORD	:	(DIGIT | LETTER)+;
> > --------
> > 
> > grammar G2;
> > q	:	(DIGIT | LETTER)+;
> > LETTER 	:	('a'..'z' | 'A'..'Z');
> > DIGIT 	:	'0'..'9';
> > --------
> > 
> > I thought that these two were equivalent, however, G1 does 
> not recognize the single-character sentence (throws 
> MismatchedTokenException), while G2 does.
> > 
> > Can you tell me what the actual difference is?
> > Than You,
> > -Trevor.
> 
> 

_______________________________________________________
TÉRÍTÉSMENTES ÉRETTSÉGI UTÁNI SZAKKÉPZÉSEK!

2 év alatt akár 3 szakma!

EUROPASS, EBCL ÉS ECDL!

Jövő Európába - EFEB
http://ad.adverticum.net/b/cl,1,6022,259041,315847/click.prm



More information about the antlr-interest mailing list