[antlr-interest] How to get a list of all valid options for the next token?

[Raphael Reitzig]

I understand that ANTLR does _not_ know which alternatives are  
possible in next step, but it tries one alternative after the other,  
taking the first that works. So, your request would mean to look ahead  
in all alternatives. This is, I suppose a rather special thing you  
won't find in ANTLR.

You mentioned syntax highlighting. This is a different thing. Syntax  
highlighting does not show you: "Well, you could start an variable  
declaration, an assignment, an if statement, ..., here", but only  
available methods on objects (at least those IDEs I know). This is  
stuff you can gather from sources/object files, store somewhere  
central staticaly. No need to look something up in any gramar while  
you're typing.

Regards

Raphael

"Niemeijer, R.A." <r.a.niemeijer at tue.nl> wrote (Wed Aug 13 13:02:41 2008):

> No problem, thanks anyway.
>
>
>
> Does anyone else have any idea how to do this?
>
>
>
> Thanks in advance.
>
>
>
> From: Foust [mailto:javafoust at gmail.com]
> Sent: woensdag 13 augustus 2008 12:44
> To: Niemeijer, R.A.
> Subject: RE: [antlr-interest] How to get a list of all valid options for
> the next token?
>
>
>
>> Yes, it's certainly possible to make my own check. However, that means
> I'm basically duplicating
>
>> my grammar, which makes it more difficult to maintain. The reason I
> wanted to get the information
>
>> from Antlr is that it should already know all the possible token
> combinations.
>
>
>
> OK, well it sounds like you know what you're doing.
>
>
>
> Sorry I wasn't able to help more,
>
>
>
> Brent
>
>
>
> From: Niemeijer, R.A. [mailto:r.a.niemeijer at tue.nl]
> Sent: Wednesday, August 13, 2008 3:06 AM
> To: Foust
> Cc:
> Subject: RE: [antlr-interest] How to get a list of all valid options for
> the next token?
>
>
>
> Yes, it's certainly possible to make my own check. However, that means
> I'm basically duplicating my grammar, which makes it more difficult to
> maintain. The reason I wanted to get the information from Antlr is that
> it should already know all the possible token combinations.
>
>
>
> From: Foust [mailto:javafoust at gmail.com]
> Sent: woensdag 13 augustus 2008 11:58
> To: Niemeijer, R.A.
> Subject: RE: [antlr-interest] How to get a list of all valid options for
> the next token?
>
>
>
> Yes, I didn't understand exactly what you're trying to do (and probably
> still don't). If you're thinking of it like an Intellisense problem,
> then there are other ways of solving that kind of problem without using
> a parser-generator like Antlr. Have you already considered those?
>
>
>
> It sounds like your problem is a bit different than the code editor's -
> simpler, actually. Whatever rules you use to decide what pieces can go
> where may be more easily expressed with a custom solution.  You could
> use Antlr to parse any text input, but call custom code to make sense of
> it and decide what the set of following pieces is. This would not only
> be more efficient, but would be far less frustrating to develop, since
> you have full control over the decision mechanism and can customize the
> underlying data structures to fit the task. Depending upon how simple
> your sequencing rules are, you might even be able to use a regular
> expression solution, if you can choose the text representation of the
> pieces.
>
>
>
> Brent
>
>
>
> From: Niemeijer, R.A. [mailto:r.a.niemeijer at tue.nl]
> Sent: Wednesday, August 13, 2008 12:13 AM
> To: Foust
> Cc: antlr-interest at antlr.org
> Subject: RE: [antlr-interest] How to get a list of all valid options for
> the next token?
>
>
>
> Hello Brent,
>
>
>
> It seems my question wasn't clear enough. I have no intention of having
> a dynamic grammar. What I meant by disabling puzzle pieces is this: on
> the left side of the screen there is a list of all the possible puzzle
> pieces, a library if you will. From there the user drags pieces to the
> right side of the screen to form sequences of puzzle pieces. What I want
> to do is disable the pieces in the library. An alternative way of
> looking at it is seeing it as Intellisense/code completion. While the
> user is typing (dragging pieces), I want to display a popup list of all
> the possible words (pieces) that can come at the current position in the
> sentence. The grammar itself is completely fixed, just like the grammar
> of programming languages that allow code completion.
>
>
>
> From: Foust [mailto:javafoust at gmail.com]
> Sent: dinsdag 12 augustus 2008 22:14
> To: Niemeijer, R.A.
> Subject: RE: [antlr-interest] How to get a list of all valid options for
> the next token?
>
>
>
>> I want to disable all the pieces that cannot come at that point in the
> sentence.
>
>> I want to disable all the pieces that cannot come at that point in the
> sentence.
>
>
>
> My guess is that this will be pretty difficult to implement using even
> an augmented context-free grammar. Usually, you would parse any valid
> sentence (a list of puzzle pieces, in your case), then do the semantic
> analysis as a later pass (iterate over the actual puzzle pieces entered
> and check each one for validity).
>
>
>
> Multiple passes should make the code easier to write (as well as
> maintain). And error reporting should be more intuitive. It's pretty
> difficult to enforce all semantic meaning at the parse phase and still
> output meaningful error messages.
>
>
>
> If you really want to experiment with changing the grammar dynamically,
> though, you could generate an AST in your parser for any valid sentence,
> and then use semantic predicates in your tree grammar to turn on and off
> certain rules. But I think you'll find that just adding validity checks
> in your tree grammar actions will be the most straightforward, providing
> all the advantages of a multi-pass interpreter.
>
>
>
> Brent
>
>
>
> From: antlr-interest-bounces at antlr.org
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Niemeijer, R.A.
> Sent: Tuesday, August 12, 2008 2:57 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] How to get a list of all valid options for the
> next token?
>
>
>
> Hello everyone,
>
>
>
> I'm making a program where users construct short sentences by combining
> puzzle pieces. This series of puzzle pieces is then converted to text
> and parsed by Antlr. Currently the user has access to all the different
> pieces at any point in the sentence, even if the grammar would not be
> able to parse this. I want to disable all the pieces that cannot come at
> that point in the sentence.
>
> My idea was to do this with the help of the exceptions the parser
> generates. For example, when it throws a MismatchedTokenException I can
> find the expected token through the Expecting property of the exception.
> Unfortunately this only works when there's only one possibility. When
> there are multiple it instead throws a NoViableAltException or a
> MismatchedSetException. I haven't been able to find a list of valid
> following tokens in their properties.
>
>
>
> Can anyone tell me if there is any way of determining the list of valid
> tokens after the current token in the sentence?
>
>
>
> Thanks in advance.
>
>



----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: PGP Digital Signature
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20080814/cbe6417c/attachment.bin