[antlr-interest] Re: Multiple lexers (c++ code gen.)

marcschellens m_schellens at hotmail.com
Thu Nov 20 22:22:02 PST 2003


Thanks Arnar,

but looking at the TokenStreamSelector Code,
it works the same way like I did.
Ie. if you define the lexers by labels, the push method
does the same as a direct push of the Lexer*.
I found out now, that the trick here was to kick the
selector via retry().
Thanks anyway,
marc

--- In antlr-interest at yahoogroups.com, "Arnar Birgisson" <arnarb at o...>
wrote:
> You have to associate each lexer in the selector with a identifying
> string, and then use that string as an argument to selctor->push.
> 
> Here's how I do it:
> 
>  istream& input = cin;
> 
>  TokenStreamSelector selector;
> 
>  LexerA aLexer(input);
>  aLexer.initialize(&selector);
> 
>  LexerB bLexer(forritLexer.getInputState());
>  bLexer.initialize(&selector);
> 
>  selector.addInputStream(&aLexer, "LexerA");
>  selector.addInputStream(&bLexer, "LexerB");
>  selector.select("LexerA");
> 
>  ASTFactory my_factory;
>  MyParser parser(selector);
> 
>  parser.initializeASTFactory(my_factory);
>  parser.setASTFactory(&my_factory);
> 
>  parser.start();
> 
> and then use it like this in LexerA:
> 
>  UNIT_LBRACE
>    : '{' { selector->push("eininglexer"); }
>    ;
> 
> Hope this helps.
> 
> Arnar
> 
> > -----Original Message-----
> > From: marcschellens [mailto:m_schellens at h...] 
> > Sent: 20. nóvember 2003 15:08
> > To: antlr-interest at yahoogroups.com
> > Subject: [antlr-interest] Multiple lexers (c++ code gen.)
> > 
> > 
> > Somehow the switch between two lexers seems not to work:
> > 
> > setup is like this:
> > 
> >     TokenStreamSelector selector;
> >     
> >     FMTLexer   lexer( istr);
> >     lexer.SetSelector( selector);
> >     
> >     CFMTLexer  cLexer( lexer.getInputState());
> >     cLexer.SetSelector( selector);
> > 
> >     lexer.SetCLexer( cLexer);
> > 
> >     selector.select( &lexer);
> > 
> >     FMTParser  parser( selector);
> > 
> >     parser.initializeASTFactory( FMTNodeFactory);
> >     parser.setASTFactory( &FMTNodeFactory );
> >    
> >     parser.format( 1);
> > 
> > And usage:
> > 
> > in FMTLexer (switch to CFMTLexer):
> > 
> > CSTRING!
> > 	: 'c' '\"' { cLexer->DoubleQuotes( true); 
> > selector->push(cLexer);}
> >     | 'c' '\'' { cLexer->DoubleQuotes( false); 
> > selector->push( cLexer);}
> > 	;	
> > 
> > cLexer is CFMTLexer* (private member in lexer).
> > 
> > Do I make an obvious mistake?
> > thanks,
> > marc
> > 
> > in CFMTLexer (switch back)
> > 
> > protected
> > ENDSTR1
> >     : ('\"') => '\"' 
> >     | { selector->pop();}
> >     ;
> > 
> > 
> > 
> > 
> > 
> > 
> >  
> > 
> > Your use of Yahoo! Groups is subject to 
> > http://docs.yahoo.com/info/terms/ 
> > 
> >


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list