[antlr-interest] assigning token LABEL

Jim Idle jimi at temporal-wave.com
Wed Jun 20 08:01:26 PDT 2007


Move this semantic into the parser:

ASS: '=' ;
SEMI : ';' ;
COLON: ':';
ID: ('a'..'z'|'A'..'Z')+ ;

label	: ID COLON ;

Then when it is unambiguous you can just use the label rule and if you
find yourself with ambiguities, just use ((label)=>label) ...

As in the artificial:

thang: ((label)=>label)? ID ASS ID SEMI label SEMI
	;

This should do everything you need and allow better error recovery etc.
In general you should do as little interpretation as possible in the
lexer (unless you have ONLY a filtering lexer), only perform semantic
checking in the parser if you definitely don't need an AST and if you
do, leave that to the tree walk/parse.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of chintan rao
> Sent: Wednesday, June 20, 2007 7:39 AM
> To: Johannes Luber
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] assigning token LABEL
> 
> Hi thanks for the solution again
> 
> >LABEL
> >       :       ('a'..'z'|'A'..'Z')+ ':'
> >       ;
> 
> The problem with the solution is that colon gets assigned with label
> too
> for example
> input:
> lb:
> output:
> now LABEL token will be 'lb:'
> I want to it to be 'lb'
> and may be assign a diffrent token for ':'
> 
> 
> 
> On 6/21/07, Johannes Luber <jaluber at gmx.de> wrote:
> > chintan rao wrote:
> > > Hi,
> > > firstly thanks for your time
> > >
> > >> As you haven't sent your grammar with your question I have to
> improvise
> > >> the snippets. Create first an imaginary LABEL token and have
rules
> like
> > >> this:
> > >
> > > What wanted was some thing like this
> > > input :
> > > la :
> > > ouput given by lexer :
> > > la gets assigned token LABEL
> > >
> > > input:
> > > la
> > > ouput given by lexer :
> > > la gets assigned token ID
> >
> > Looking at your sample input I discovered that I misread your
> > specification. I thought that after the label had to come an ID. My
> > mistake. The following grammar works with ANTLRworks and parses my
> test
> > input correctly. Just don't forget a whitespace rule in your actual
> grammar.
> >
> > LABEL
> > 	:	('a'..'z'|'A'..'Z')+ ':'
> > 	;
> >
> > ID
> > 	:	('a'..'z'|'A'..'Z')+
> > 	;
> >
> > test
> > 	:	(LABEL|ID)*
> > 	;
> >
> > Best regards,
> > Johannes Luber
> >


More information about the antlr-interest mailing list