[antlr-interest] Re: Troubel With Keywords / Whitespaces

[Jason]

Loring,

Thanks for taking the time to reply.  The rule wasn't
actually:

EAL_SELECT: "select"   { };

but something like:

EAL_SELECT: {mState = INITAL || mState = IDLIST}? { };

The lex grammar I was using as my base, made heavy use
of flex's start states and I ported them to antlr
using semantic predicates, thus the need for the
explicit rule.  The real grammar (as opposed to the
one I included in my email) actually has about 80
similar rules.  In any case, I just moved them all to
the token sections.  What happens when things blow up
because I've lost the state information?  I'll cross
that bridge when I get to it :)

One general comment to no one in particular.  I
started with a pretty complicated pair of FLEX/BISON
grammars which I intended to 'port' to ANTLR.  In
practice the differing features/facilities of the two
have made this very difficult.  If I had it to do
over, I'd scratch the FLEX/Bison grammars and start
fresh with ANTLR right from the source description of
the language.

-jason


--- lgcraymer <lgc at mail1.jpl.nasa.gov> wrote:
> Jason--
> 
> In your EAL_SELECT, "select" is not matched as a
> literal--it is
> matched as 's' 'e' 'l' 'e' 'c' 't'.
> 
> To get the behavior that you want, delete the
> EAL_SELECT rule and add
> EAL_SELECT="select" to your "tokens" in the lexer. 
> Or use "select" in
> your parser grammar
> 
> --Loring
> 
> --- In antlr-interest at yahoogroups.com, Jason
> <jasonriz at y...> wrote:
> > Hello,
> > 
> > I've run into a problem when I run the lexer
> generated
> > with the grammar which follows on, for instance,
> this
> > input stream:  'select foo bar bar selectbar foo'.
> 
> > Everything works ducky except that rather than
> > matching the 'REG_ID' rule, 'selectbar' matches
> the
> > 'EAL_SELECT' rule followed by the "REG_ID' rule. 
> It's
> > obvious why this is happening but I can't figure
> out
> > the best way to fix it.  I know it's unusual to
> make
> > each Keyword a rule, but I make heavy use of
> semantic
> > predicates to determine whether a keyword has been
> > matched or not.  The only solution I've come up
> with
> > is to change 'select' to 'select ' but there's got
> to
> > be something better.  If anyone has any thoughts
> I'd
> > greatly appreciate it.
> > 
> > class TestLexer extends Lexer;
> > 
> > options
> > {
> >   k = 8;
> >   exportVocab=Test;
> > }
> > 
> > EAL_SELECT: "select"   { };
> > 
> > protected ALPHA_CHAR: ('a'..'z' | 'A'..'Z') { };
> > protected DIGIT: ('0'..'9');
> > 
> > REG_ID: ALPHA_CHAR ( ALPHA_CHAR | DIGIT )* { };
> > 
> > // Whitespace -- ignored
> > WS	:	
> >   (	' ' |	'\t' |	'\f' |	(options
> > {generateAmbigWarnings=false;}
> > 			:	"\r\n" 
> > 			|	'\r'   
> > 			|	'\n'   
> > 			)
> > 			{ newline(); }
> > 		)+
> > 		{ _ttype = Token.SKIP; }
> >   ;
> > 
> > -jason
> > 
> > -jason
> > 
> > __________________________________
> > Do you Yahoo!?
> > Yahoo! Search - Find what you're looking for
> faster
> > http://search.yahoo.com
> 
> 
> 
>  
> Yahoo! Groups Links
> 
> 
>      antlr-interest-unsubscribe at yahoogroups.com
> 
>  
> 


__________________________________
Do you Yahoo!?
Yahoo! Search - Find what you’re looking for faster
http://search.yahoo.com


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/