[antlr-interest] How to get a list of all valid options for the next token?

Niemeijer, R.A. r.a.niemeijer at tue.nl
Wed Aug 13 03:06:22 PDT 2008


Yes, it's certainly possible to make my own check. However, that means
I'm basically duplicating my grammar, which makes it more difficult to
maintain. The reason I wanted to get the information from Antlr is that
it should already know all the possible token combinations.

 

From: Foust [mailto:javafoust at gmail.com] 
Sent: woensdag 13 augustus 2008 11:58
To: Niemeijer, R.A.
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

 

Yes, I didn't understand exactly what you're trying to do (and probably
still don't). If you're thinking of it like an Intellisense problem,
then there are other ways of solving that kind of problem without using
a parser-generator like Antlr. Have you already considered those? 

 

It sounds like your problem is a bit different than the code editor's -
simpler, actually. Whatever rules you use to decide what pieces can go
where may be more easily expressed with a custom solution.  You could
use Antlr to parse any text input, but call custom code to make sense of
it and decide what the set of following pieces is. This would not only
be more efficient, but would be far less frustrating to develop, since
you have full control over the decision mechanism and can customize the
underlying data structures to fit the task. Depending upon how simple
your sequencing rules are, you might even be able to use a regular
expression solution, if you can choose the text representation of the
pieces.

 

Brent   

 

From: Niemeijer, R.A. [mailto:r.a.niemeijer at tue.nl] 
Sent: Wednesday, August 13, 2008 12:13 AM
To: Foust
Cc: antlr-interest at antlr.org
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

 

Hello Brent,

 

It seems my question wasn't clear enough. I have no intention of having
a dynamic grammar. What I meant by disabling puzzle pieces is this: on
the left side of the screen there is a list of all the possible puzzle
pieces, a library if you will. From there the user drags pieces to the
right side of the screen to form sequences of puzzle pieces. What I want
to do is disable the pieces in the library. An alternative way of
looking at it is seeing it as Intellisense/code completion. While the
user is typing (dragging pieces), I want to display a popup list of all
the possible words (pieces) that can come at the current position in the
sentence. The grammar itself is completely fixed, just like the grammar
of programming languages that allow code completion.

 

From: Foust [mailto:javafoust at gmail.com] 
Sent: dinsdag 12 augustus 2008 22:14
To: Niemeijer, R.A.
Subject: RE: [antlr-interest] How to get a list of all valid options for
the next token?

 

> I want to disable all the pieces that cannot come at that point in the
sentence.

> I want to disable all the pieces that cannot come at that point in the
sentence.

 

My guess is that this will be pretty difficult to implement using even
an augmented context-free grammar. Usually, you would parse any valid
sentence (a list of puzzle pieces, in your case), then do the semantic
analysis as a later pass (iterate over the actual puzzle pieces entered
and check each one for validity). 

 

Multiple passes should make the code easier to write (as well as
maintain). And error reporting should be more intuitive. It's pretty
difficult to enforce all semantic meaning at the parse phase and still
output meaningful error messages.

 

If you really want to experiment with changing the grammar dynamically,
though, you could generate an AST in your parser for any valid sentence,
and then use semantic predicates in your tree grammar to turn on and off
certain rules. But I think you'll find that just adding validity checks
in your tree grammar actions will be the most straightforward, providing
all the advantages of a multi-pass interpreter.

 

Brent

 

From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Niemeijer, R.A.
Sent: Tuesday, August 12, 2008 2:57 AM
To: antlr-interest at antlr.org
Subject: [antlr-interest] How to get a list of all valid options for the
next token?

 

Hello everyone,

 

I'm making a program where users construct short sentences by combining
puzzle pieces. This series of puzzle pieces is then converted to text
and parsed by Antlr. Currently the user has access to all the different
pieces at any point in the sentence, even if the grammar would not be
able to parse this. I want to disable all the pieces that cannot come at
that point in the sentence.

My idea was to do this with the help of the exceptions the parser
generates. For example, when it throws a MismatchedTokenException I can
find the expected token through the Expecting property of the exception.
Unfortunately this only works when there's only one possibility. When
there are multiple it instead throws a NoViableAltException or a
MismatchedSetException. I haven't been able to find a list of valid
following tokens in their properties.

 

Can anyone tell me if there is any way of determining the list of valid
tokens after the current token in the sentence?

 

Thanks in advance.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080813/da55ad4f/attachment.html 


More information about the antlr-interest mailing list