[antlr-interest] Re : Re : How to get a list of all valid options for the next token?

[Terence Parr]

Ok,I added a faq entry:

http://www.antlr.org/wiki/pages/viewpage.action?pageId=11567208

  see if this helps.

Ter
On Aug 15, 2008, at 2:28 AM, Niemeijer, R.A. wrote:

> Right, I think I've localized all the data I need. All you need is a  
> bit of reflection. All the bitsets that determine the possible next  
> tokens are called FOLLOW_{0}_in_{1}{2}, where {0} is the previous  
> token (can be gotten from the Parser's state.following variable),  
> {1} is the name of the method the NoViableAltException is generated  
> in (use something like MethodBase.GetCurrentMethod().Name) and {2}  
> is un unimportant 3-digit number. So get the list of all Bitsets of  
> the Parser via reflection and choose the one that starts with  
> FOLLOW_{0}_in_{1}.
>
> However, in order to be able to use this I have to insert this code  
> at the point the NoViableAltException is thrown so I can dump the  
> list of tokens in its grammarDecisionDescription field. That means  
> overriding all the bits of code that look like this:
>
> NoViableAltException nvae_d1s2 =
>    new NoViableAltException("", 1, 2, input);
>
>    throw nvae_d1s2;
>
> Now obviously this can be done manually, but then you'd lose it  
> after every grammar generation. So my question: is there a way to  
> override this part of the generated grammar?
>
> Thanks.
>
> -----Original Message-----
> From: Niemeijer, R.A.
> Sent: donderdag 14 augustus 2008 17:39
> To: 'Stanislas Rusinsky'; Thomas Brandon
> Cc: antlr-interest at antlr.org
> Subject: RE: Re : [antlr-interest] Re : How to get a list of all  
> valid options for the next token?
>
> The FOLLOW_ variables do indeed seem to contain all the data I need.  
> I'm going to have a look at it tomorrow to see if I can convert to  
> bitset data to something more useful to me.
>
> -----Original Message-----
> From: Stanislas Rusinsky [mailto:rusinskystanislas at yahoo.fr]
> Sent: woensdag 13 augustus 2008 19:27
> To: Thomas Brandon; Niemeijer, R.A.
> Cc: antlr-interest at antlr.org
> Subject: Re : [antlr-interest] Re : How to get a list of all valid  
> options for the next token?
>
> oops, when making such attention mistakes I feel like hiding under  
> the table :) they are LL of course
>
> Mr. Niemeijer, could you please keep me informed if you reach a  
> satisfactory solution with the "FOLLOW_" hint? I am interested too  
> in that topic.
>
> Stanislas.
>
>
>
> ----- Message d'origine ----
> De : Thomas Brandon <tbrandonau at gmail.com>
> À : "Niemeijer, R.A." <r.a.niemeijer at tue.nl>
> Cc : antlr-interest at antlr.org
> Envoyé le : Mercredi, 13 Août 2008, 16h32mn 28s
> Objet : Re: [antlr-interest] Re : How to get a list of all valid  
> options for the next token?
>
> ANTLR grammars are LL not LR, but they are recursive descent and thus
> don't use an explicit internal representation of the grammar to
> control parse decisions. They do however have follow sets available at
> parse time, used for error recovery and reporting. See the various
> FOLLOW_ bitsets in generated parsers and the computeErrorRecoverySet
> and computeContextSensitiveRuleFOLLOW methods in BaseRecognizer. You
> may be able to get what you need from them. Though as they only
> support a limited purpose they may collapse alternatives you need to
> keep distinct.
> Another option would be to create a custom generator that would then
> have the richer information ANTLR uses at generation time.
>
> Tom.
>
> On Wed, Aug 13, 2008 at 10:48 PM, Niemeijer, R.A. <r.a.niemeijer at tue.nl 
> > wrote:
>> Well, so much for that idea then :(
>>
>> Guess the only way to do this then is to indeed duplicate the  
>> knowledge
>> inherent in the parser to a different checker. Oh well.
>>
>>
>>
>> From: Stanislas Rusinsky [mailto:rusinskystanislas at yahoo.fr]
>> Sent: woensdag 13 augustus 2008 14:42
>> To: Niemeijer, R.A.; Foust
>> Cc: antlr-interest at antlr.org
>> Subject: Re : [antlr-interest] How to get a list of all valid  
>> options for
>> the next token?
>>
>>
>>
>> I have found one series of 3 articles that might be interesting for  
>> such
>> situation:
>>
>> http://www.ibm.com/developerworks/views/opensource/libraryview.jsp?search_by=Create+commercial-quality+eclipse+ide
>>
>> But I don't know if you could achieve that easily (if someone knows  
>> a way or
>> tool I would be interested too in knowing it). ANTLR parsers  
>> generated from
>> grammars are not 'self-aware' if I understood correctly: they are  
>> LR parsers
>> but don't hold as such a graph based on the grammar that could  
>> determine the
>> next allowed token.
>>
>> Yours sincerely,
>>
>> Stanislas Herman.
>>
>
>
>
>       
> _____________________________________________________________________________
> Envoyez avec Yahoo! Mail. Une boite mail plus intelligente http://mail.yahoo.fr