[antlr-interest] Re: catching an exception in the parser and cont inuing - will this work?

David Penton djplists at arrowsash.com
Fri Jan 24 12:48:17 PST 2003


Thanks Monty.

- Dave -
On Fri, 2003-01-24 at 12:47, mzukowski at yci.com wrote:
> The single spot you are asking about is the consume() method.  Everything
> goes through there.  Look at the generated code.  
> 
> Monty
> 
> -----Original Message-----
> From: davidjpenton2002 <djplists at arrowsash.com>
> [mailto:djplists at arrowsash.com]
> Sent: Friday, January 24, 2003 9:23 AM
> To: antlr-interest at yahoogroups.com
> Subject: [antlr-interest] Re: catching an exception in the parser and
> cont inuing - will this work?
> 
> 
> Thanks for your help.
> 
> I had hoped that the "bad" token could be handled by a catch block 
> somewhere.  The "somewhere" part may be the catch.  It appears to me 
> that the normal behavior of an antlr parser is to report an 
> unexpected token, and then to forge ahead.  That's the behavior I 
> observe if I place a token (or rather a string that will match a 
> known lexer rule) in the wrong spot in my input.  The parser reports 
> it as an unexpected token, but continues to parse.  That is the 
> behavior that I want to use by returning a "special" token unknown to 
> any parser rule. I do not (yet) understand just where and how the 
> normal processing happens in the parser code - I'm busy looking 
> through it to find out, especially as regards exception handling.  
> Any hints on this would help, although I am also re-reading the docs 
> on exceptions.
> 
> The main thing that would make life easier for me is if I can 
> identify a single spot to handle the information about source of 
> tokens.  That is, it would be a pain in the neck if such processing 
> had to be associated with each parser rule separately.  I'm not sure 
> if I am being clear about that.
> 
> The other two strategies that you suggest (hidden token stream; 
> embedding file information in the token) may also be fruitful for me, 
> so I'll look into both. Thanks for the excellent insights.
> 
> - Dave Penton -
> 
> --- In antlr-interest at yahoogroups.com, mzukowski at y... wrote:
> > What I do in the gcc project is have tokens aware of the file they 
> came from
> > as well.  Then it is pretty easy to tell whether a new file has been
> > included or not.
> > 
> > There are other ways too, like Ter's fieldguide on whitespace 
> preservation
> > by using a hidden token stream.
> > 
> > You'll have to be more specific about what you mean by making the 
> parser
> > "aware" of the token change.  There are other ways to do this.  
> > 
> > Your way might work, but exceptions are expensive.  Wait, it won't 
> work
> > because it will screw up lookahead.  No rule will know about your 
> bad token
> > and that would be "bad".
> > 
> > If you don't like having the filename in the token, I would instead 
> put that
> > "bad" token in a hidden token stream.  You can override consume() 
> to check
> > the hidden token stream and set your state.  Or it can wait in the 
> hidden
> > stream and everytime you check that state you could pull in the 
> tokens from
> > the hidden stream and set state before returning it.
> > 
> > Monty
> > 
> > -----Original Message-----
> > From: davidjpenton2002 <djplists at a...>
> > [mailto:djplists at a...]
> > Sent: Friday, January 24, 2003 7:32 AM
> > To: antlr-interest at yahoogroups.com
> > Subject: [antlr-interest] catching an exception in the parser and
> > continuing - will this work?
> > 
> > 
> > I am handling "include files" in my lexer, as in the java example 
> > provided with antlr.  This means that fact the lexer is in an 
> > included file is transparent to the parser.
> > 
> > However, it turns out that I want the parser to be made aware of 
> the 
> > fact that the lexer is in an included file. (Kindly assume that I 
> > have a sound design reason for this ;-) Given the amount of 
> lookahead 
> > that may occur, I am sure that the only safe way to do this is by 
> > putting the correct "notification" on the stream of tokens passed 
> to 
> > the parser.
> > 
> > Is it possible to have the lexer pass a back a "bad" token (i.e. 
> one 
> > that the parser can never recognize), catch a resultant exception, 
> > and then (here's the important part) force the parser to take up 
> > right where is was when it got the "bad" token?  It would amount to 
> > skipping over the "bad" token, except that the exception handling 
> > code would set a flag or two that would be available later to 
> parser 
> > semantic actions.
> > 
> > I have looked at the exception handling documentation, and sifted 
> > through my generated parser.  It is not clear to me if my strategy 
> > will work.
> > 
> > BTW, I am still using antlr 2.7.1 - I want to try to finish what I 
> am 
> > doing before trying the switch to 2.7.2.
> > 
> > Thanks.
> > 
> > - Dave Penton -
> > 
> > 
> >  
> > 
> > Your use of Yahoo! Groups is subject to 
> http://docs.yahoo.com/info/terms/
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 
> 
> 



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list