[antlr-interest] Re : How to get a list of all valid options for the next token?

Wed Aug 13 05:42:11 PDT 2008

I have found one series of 3 articles that might be interesting for such situation: 

http://www.ibm.com/developerworks/views/opensource/libraryview.jsp?search_by=Create+commercial-quality+eclipse+ide

But I don't know if you could achieve that easily (if someone knows a way or tool I
would be interested too in knowing it). ANTLR parsers generated
from grammars are not 'self-aware' if I understood correctly: they are LR parsers but don't hold as such a graph based on the grammar that could determine the next allowed token.

Yours sincerely,

Stanislas Herman.

----- Message d'origine ----
De : "Niemeijer, R.A." <r.a.niemeijer at tue.nl>
À : Foust <javafoust at gmail.com>
Cc : antlr-interest at antlr.org
Envoyé le : Mercredi, 13 Août 2008, 13h02mn 41s
Objet : Re: [antlr-interest] How to get a list of all valid options for the next token?

No problem, thanks anyway.

Does anyone else have any idea
how to do this?

Thanks in advance.

From:Foust
[mailto:javafoust at gmail.com] 
Sent: woensdag 13 augustus 2008 12:44
To: Niemeijer, R.A.
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

> Yes, it’s certainly
possible to make my own check. However, that means I’m basically duplicating 
> my grammar, which makes it
more difficult to maintain. The reason I wanted to get the information 
> from Antlr is that it
should already know all the possible token combinations.

OK, well it sounds like you know
what you’re doing. 

Sorry I wasn’t able to help
more,

Brent

From:Niemeijer, R.A.
[mailto:r.a.niemeijer at tue.nl] 
Sent: Wednesday, August 13, 2008 3:06 AM
To: Foust
Cc: 
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

Yes, it’s certainly possible to
make my own check. However, that means I’m basically duplicating my grammar,
which makes it more difficult to maintain. The reason I wanted to get the information
from Antlr is that it should already know all the possible token combinations.

From:Foust
[mailto:javafoust at gmail.com] 
Sent: woensdag 13 augustus 2008 11:58
To: Niemeijer, R.A.
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

Yes, I didn’t understand exactly
what you’re trying to do (and probably still don’t). If you’re thinking of it
like an Intellisense problem, then there are other ways of solving that kind of
problem without using a parser-generator like Antlr. Have you already
considered those? 

It sounds like your problem is a
bit different than the code editor’s – simpler, actually. Whatever rules you
use to decide what pieces can go where may be more easily expressed with a
custom solution.  You could use Antlr to parse any text input, but call
custom code to make sense of it and decide what the set of following pieces is.
This would not only be more efficient, but would be far less frustrating to
develop, since you have full control over the decision mechanism and can
customize the underlying data structures to fit the task. Depending upon how
simple your sequencing rules are, you might even be able to use a regular
expression solution, if you can choose the text representation of the pieces.

Brent   

From:Niemeijer, R.A.
[mailto:r.a.niemeijer at tue.nl] 
Sent: Wednesday, August 13, 2008 12:13 AM
To: Foust
Cc: antlr-interest at antlr.org
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

Hello Brent,

It seems my question wasn’t
clear enough. I have no intention of having a dynamic grammar. What I meant by
disabling puzzle pieces is this: on the left side of the screen there is a list
of all the possible puzzle pieces, a library if you will. From there the user
drags pieces to the right side of the screen to form sequences of puzzle
pieces. What I want to do is disable the pieces in the library. An alternative
way of looking at it is seeing it as Intellisense/code completion. While the
user is typing (dragging pieces), I want to display a popup list of all the
possible words (pieces) that can come at the current position in the sentence.
The grammar itself is completely fixed, just like the grammar of programming
languages that allow code completion.

From:Foust [mailto:javafoust at gmail.com] 
Sent: dinsdag 12 augustus 2008 22:14
To: Niemeijer, R.A.
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

> I want to disable
all the pieces that cannot come at that point in the sentence.
> I want to disable all the pieces that cannot come at
that point in the sentence.

My guess is that this will be
pretty difficult to implement using even an augmented context-free grammar.
Usually, you would parse any valid sentence (a list of puzzle pieces, in
your case), then do the semantic analysis as a later pass (iterate over the
actual puzzle pieces entered and check each one for validity). 

Multiple passes should make the
code easier to write (as well as maintain). And error reporting should be more
intuitive. It’s pretty difficult to enforce all semantic meaning at the parse
phase and still output meaningful error messages.

If you really want to experiment
with changing the grammar dynamically, though, you could generate an AST in
your parser for any valid sentence, and then use semantic predicates in your
tree grammar to turn on and off certain rules. But I think you’ll find that
just adding validity checks in your tree grammar actions will be the most
straightforward, providing all the advantages of a multi-pass interpreter.

Brent

From:antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On
Behalf Of Niemeijer, R.A.
Sent: Tuesday, August 12, 2008 2:57 AM
To: antlr-interest at antlr.org
Subject: [antlr-interest] How to get a list of all valid options for the
next token?

Hello everyone,

I’m making a program where users construct short sentences
by combining puzzle pieces. This series of puzzle pieces is then converted to
text and parsed by Antlr. Currently the user has access to all the different
pieces at any point in the sentence, even if the grammar would not be able to
parse this. I want to disable all the pieces that cannot come at that point in
the sentence.
My idea was to do this with the help of the exceptions the
parser generates. For example, when it throws a MismatchedTokenException I can
find the expected token through the Expecting property of the exception.
Unfortunately this only works when there’s only one possibility. When there are
multiple it instead throws a NoViableAltException or a MismatchedSetException.
I haven’t been able to find a list of valid following tokens in their
properties.

Can anyone tell me if there is any way of determining the
list of valid tokens after the current token in the sentence?

Thanks in advance.

      _____________________________________________________________________________ 
Envoyez avec Yahoo! Mail. Une boite mail plus intelligente http://mail.yahoo.fr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080813/643a71ea/attachment-0001.html 