[antlr-interest] i have a problem with multiplexing.Can anyone help me?

[Dean Tribble]

I think that the problem in the code below is that you must pass
getInputState() rather than selector to the newly created parser.
Otherwise, they are not sharing the look ahead buffer.

Three additional classes of problems I ran into were:

- if the token sets are not compatible (i.e., common tokens need to be
the same in all grammars) then you get strange results because a
perfectly legitimate token in one lexer shows up as a strange token in
another. You either need to explicitly declare some of the tokens in a
common vocabulary, and make sure that all the grammars import the
appropriate vocabularies.

- look-ahead timing: the lexer lookahead and therefore the lexer actions
happen before the corresponding parser actions. Thus, you could have
tokens from one lexer being visible to the other parser. Unless you
*relly* need a different identifier set, combine the grammars and just
use separate lexers. I have a slightly different approach to rewind and
such taht also help address the problem, which I will send to the list
in the next few days (I just got this stuff working a few days ago).

- it was not clear to me whether the production that invokes the other
parser will end up with the other parser's result on the local stack. To
eliminate this concern, I had something like:

pattHole : ATCURLY!
{ EParser p = new EParser(getInputState());
p.pattern();
## = p.getAST(); // get the production from the EParser to this parser.
}
"}"!
;

It may be that that falls out of the shared input state, however. I
didn't figure out because I realized that changing to a single grammar
on multiple lexers simplified things.

ºÎ³¬ wrote:

> i've realized two separate grammars, one for java code analyse, the
> other for javadoc embedded in the java file. Then i try to link them
> together using the
> technology multiplexing. The critical parts are as follows:
> ---------- in java.g ------------------
> ...
>
> JAVADOC_OPEN: "/**"
> {
> selector.select("javadocLexer");
>
> // create a parser to handle the javadoc comment
> JavadocParser jdocparser = new JavadocParser(selector);
> jdocparser.comment();
> };
>
> ...
> ---------- in javadoc.g ------------------
> ...
>
> javadoc_comment
> : (options{greedy=true;}:STAR)*
> ( (AT)=>AT javadoc_tag | description )
> javadoc_tags
> JAVADOC_CLOSE //here without EOF, and i believe no need
> ;
> ...
> ...
>
> JAVADOC_CLOSE: "*/" {selector.select("javaLexer");};
> ...
>
> ----------------------- Test.java---------------------------------
>
> import java.io.*;
> import antlr.CommonAST;
> import antlr.collections.AST;
> import antlr.debug.misc.ASTFrame;
> import java.util.*;
> import antlr.TokenStreamSelector;
>
>
> class Test{
>
> public static void main(String[] args)
> {
> try {
> TokenStreamSelector selector = new TokenStreamSelector();
>
> JavaLexer mainLexer = new JavaLexer(
> new FileInputStream(args[0]));
> mainLexer.selector = selector;
>
> JavadocLexer docLexer =
> new JavadocLexer (mainLexer.getInputState());
> docLexer.selector = selector;
>
> selector.addInputStream(mainLexer, "javaLexer");
> selector.addInputStream(docLexer, "javadocLexer");
> selector.select("javaLexer");
>
> JavaParser parser = new JavaParser(selector);
> parser.compilationUnit();
>
> }catch(Exception e) {
> System.err.println("exception: "+e);
> }
> }
> }
> ---------- java file for
> testing-------------------------------------------
> package com;
>
> import java.io.*;
>
> //single-line comment
>
> /*multi-line comment*/
>
> public class A
> {
> public A()
> {
> }
>
> /**
> * javadoc comment
> *
> * @author hc
> * @version 1.0
> * @param i the first number
> * @param j the second number
> * @return the sum of two numbers
> * @cml my own defined javadoc tag for some special purpose
> */
> public int add(int i,int j)
> {
> return i+j;
> }
> }
> -----------------------------------------------------------------------------
> Then i got the error message:
> exception: line 24:5: expecting EOF, found 'public'
>
>
> i've been bewildered by this problem for quite a few days and look
> forward to help.
>
>
>
>
>
>
>
> ¡¤ÍøÒ×126ÓÊÏä --רҵµç×ÓÓÊ¾Ö 1.5G¿Õ¼ä,30Õ׸½¼þ http://www.126.com >>>
> <http://www.126.com>