[antlr-interest] Lines that don't match a rule ...
James Ladd
james_ladd at hotmail.com
Thu Jul 7 14:45:34 PDT 2011
Thanks Jim ! I'm sure that will do the trick.
This list is so helpful.
> From: antlr-interest-request at antlr.org
> Subject: antlr-interest Digest, Vol 80, Issue 7
> To: antlr-interest at antlr.org
> Date: Thu, 7 Jul 2011 12:00:01 -0700
>
> Send antlr-interest mailing list submissions to
> antlr-interest at antlr.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://www.antlr.org/mailman/listinfo/antlr-interest
> or, via email, send a message with subject or body 'help' to
> antlr-interest-request at antlr.org
>
> You can reach the person managing the list at
> antlr-interest-owner at antlr.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of antlr-interest digest..."
>
>
> Today's Topics:
>
> 1. left recursion removal (S?bastien Kirche)
> 2. Re: left recursion removal (Jim Idle)
> 3. Lines that don't match a rule ... (James Ladd)
> 4. Re: left recursion removal (John B. Brodie)
> 5. Re: Think I found a bug. (James Reid)
> 6. Re: left recursion removal (S?bastien Kirche)
> 7. Re: Lines that don't match a rule ... (Jim Idle)
> 8. Re: left recursion removal (Jim Idle)
> 9. Re: left recursion removal (Jim Idle)
> 10. Re: Think I found a bug. (Terence Parr)
> 11. Re: left recursion removal (John B. Brodie)
> 12. Re: left recursion removal (S?bastien Kirche)
> 13. C Target won't compile with MSVC (Ivan Brezina)
> 14. Re: C Target won't compile with MSVC (Jim Idle)
> 15. loops and syntax-directed interpreter ? (Roy Metzger)
> 16. Re: left recursion removal (S?bastien Kirche)
> 17. Re: left recursion removal (John B. Brodie)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 6 Jul 2011 21:19:39 +0200
> From: S?bastien Kirche <sebastien.kirche at gmail.com>
> Subject: [antlr-interest] left recursion removal
> To: antlr-interest <antlr-interest at antlr.org>
> Message-ID:
> <CALGPt8YiQvER1svK_zfJQ7UkJXYz5ww0U0yRLj7ha_GMK+v8yw at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi,
>
> the language for which I am trying to build the grammar has 2 flavors
> of if-then-else constructs
> - a single line : if <condition> then <statement> [else <statement>]
> - a multi line : if <condition> then <statements> [else <statements>] end if
>
> I have defined the following (partial) :
>
> codeBlock
> : (compoundStatement)*
> ;
>
> compoundStatement
> : (
> ifStatement
> | singleStatement
> ) (';' | EOL)
> ;
>
> singleStatement
> : localVariableDeclaration
> | funCall
> | 'return' expression
> ;
>
> ifStatement
> : singleLineIf
> | multiLineIf
> ;
>
> singleLineIf
> : 'if' expression 'then' singleStatement EOL
> ;
>
> multiLineIf
> : 'if' expression 'then' codeBlock 'end if'
> ;
>
>
> I understand why the naive ifStatement fails with the following "
> [fatal] rule ifStatement has non-LL(*) decision due to recursive rule
> invocations reachable from alts 1,2. Resolve by left-factoring or
> using syntactic predicates or using backtrack=true option."
>
> I would like to avoid general backtracking, so after searching for a
> while and reading the article
> http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar
> I have tried first
>
> ifStatement
> : 'if' expression 'then' (singleStatement EOL)=> singleLineIf
> | multiLineIf
> ;
>
> or
>
> ifStatement
> : 'if' expression 'then' (singleStatement EOL | codeBlock 'end if')
> ;
>
> But they fail both with the same fatality.
> How this case should be processed ?
> --
> S?bastien Kirche
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 6 Jul 2011 14:27:39 -0700
> From: Jim Idle <jimi at temporal-wave.com>
> Subject: Re: [antlr-interest] left recursion removal
> To: antlr-interest <antlr-interest at antlr.org>
> Message-ID: <fc73d76295d104694253b4775b459791 at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> What language? This is usually distinguished by the fact that multi
> statement blocks must begin on a new line after THEN or ELSE. VB.Net for
> instance.
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of S?bastien Kirche
> > Sent: Wednesday, July 06, 2011 12:20 PM
> > To: antlr-interest
> > Subject: [antlr-interest] left recursion removal
> >
> > Hi,
> >
> > the language for which I am trying to build the grammar has 2 flavors
> > of if-then-else constructs
> > - a single line : if <condition> then <statement> [else <statement>]
> > - a multi line : if <condition> then <statements> [else <statements>]
> > end if
> >
> > I have defined the following (partial) :
> >
> > codeBlock
> > : (compoundStatement)*
> > ;
> >
> > compoundStatement
> > : (
> > ifStatement
> > | singleStatement
> > ) (';' | EOL)
> > ;
> >
> > singleStatement
> > : localVariableDeclaration
> > | funCall
> > | 'return' expression
> > ;
> >
> > ifStatement
> > : singleLineIf
> > | multiLineIf
> > ;
> >
> > singleLineIf
> > : 'if' expression 'then' singleStatement EOL
> > ;
> >
> > multiLineIf
> > : 'if' expression 'then' codeBlock 'end if'
> > ;
> >
> >
> > I understand why the naive ifStatement fails with the following "
> > [fatal] rule ifStatement has non-LL(*) decision due to recursive rule
> > invocations reachable from alts 1,2. Resolve by left-factoring or
> > using syntactic predicates or using backtrack=true option."
> >
> > I would like to avoid general backtracking, so after searching for a
> > while and reading the article
> > http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtrack
> > ing+from+your+grammar
> > I have tried first
> >
> > ifStatement
> > : 'if' expression 'then' (singleStatement EOL)=>
> singleLineIf
> > | multiLineIf
> > ;
> >
> > or
> >
> > ifStatement
> > : 'if' expression 'then' (singleStatement EOL | codeBlock
> 'end
> > if')
> > ;
> >
> > But they fail both with the same fatality.
> > How this case should be processed ?
> > --
> > S?bastien Kirche
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-address
>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 7 Jul 2011 07:41:41 +1000
> From: James Ladd <james_ladd at hotmail.com>
> Subject: [antlr-interest] Lines that don't match a rule ...
> To: <antlr-interest at antlr.org>
> Message-ID: <BAY149-w32B3DDCD6F82F6E204DDF9ED5E0 at phx.gbl>
> Content-Type: text/plain; charset="iso-8859-1"
>
>
> Hi All,
>
> I hope I can get a little help on the grammar I have below.
> It works to a degree but I'm having trouble with some edge cases and that is
> stressing me a little.
>
> When I write something like the following I get the right response - sequence of tokens.
>
> Im putting '\n' where I would have new lines.
>
> \n
> + foo\n
> \n
>
> What I can't seem to do is put in a rule that matches a line that doesn't start with '+' or '-'
> I get a no viable Alt Exception with all that I try.
>
>
> My goal is to be able to detect lines starting with a '+' or a '-' followed by a messagePattern.
> And lines NOT starting with a '+' or a '-'. The grammar below does not contain any rules
> for this, because I can't get them to work.
>
> Please help
>
> Rgs, James.
>
>
> grammar Preprocessor;
>
> options {
> language = Java;
> }
>
> @header {
> package compiler;
>
> }
>
> @lexer::header {
> package compiler;
> }
>
> lines
> : (lineBreak | messagePattern)* EOF
> ;
>
> messagePattern
> : ('+' | '-') (unarySelector | keywordSelector | binarySelector)
>
> ;
>
> unarySelector
> : NAME
> ;
>
> keywordSelector
> : (KEYWORD NAME)+
> ;
>
> binarySelector
> : BINARY_SYMBOL NAME
> ;
>
> lineBreak
> : LINE_BREAK ;
>
> LINE_BREAK: ('\r'? '\n' | '\r');
>
> NAME: ('a'..'z' | 'A'..'Z')('a'..'z' | 'A'..'Z' | '0'..'9')*;
> KEYWORD: NAME':';
> BINARY_SYMBOL: ('~'|'!'|'@'|'%'|'&'|'*'|'-'|'+'|'='|'\\'|'|'|'?'|'/'|'>'|'<'|',') ('~'|'!'|'@'|'%'|'&'|'*'|'-'|'+'|'='|'\\'|'|'|'?'|'/'|'>'|'<'|',')*;
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 06 Jul 2011 18:14:16 -0400
> From: "John B. Brodie" <jbb at acm.org>
> Subject: Re: [antlr-interest] left recursion removal
> To: S?bastien Kirche <sebastien.kirche at gmail.com>
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID: <1309990456.23994.21.camel at gecko>
> Content-Type: text/plain; charset="utf-8"
>
> Greetings!
>
> On Wed, 2011-07-06 at 21:19 +0200, S?bastien Kirche wrote:
> > Hi,
> >
> > the language for which I am trying to build the grammar has 2 flavors
> > of if-then-else constructs
> > - a single line : if <condition> then <statement> [else <statement>]
> > - a multi line : if <condition> then <statements> [else <statements>] end if
> >
> > I have defined the following (partial) :
> >
> > codeBlock
> > : (compoundStatement)*
> > ;
> >
> > compoundStatement
> > : (
> > ifStatement
> > | singleStatement
> > ) (';' | EOL)
> > ;
> >
> > singleStatement
> > : localVariableDeclaration
> > | funCall
> > | 'return' expression
> > ;
> >
> > ifStatement
> > : singleLineIf
> > | multiLineIf
> > ;
> >
> > singleLineIf
> > : 'if' expression 'then' singleStatement EOL
> > ;
> >
> > multiLineIf
> > : 'if' expression 'then' codeBlock 'end if'
> > ;
> >
> >
> > I understand why the naive ifStatement fails with the following "
> > [fatal] rule ifStatement has non-LL(*) decision due to recursive rule
> > invocations reachable from alts 1,2. Resolve by left-factoring or
> > using syntactic predicates or using backtrack=true option."
>
> unable to reproduce.
>
> given your admittedly partial grammar, i tried to construct a complete
> example by adding the missing elements and creating an AST (so i could
> know the resultant parse).
>
> my test rig is attached.
>
> it runs without error when Tool'd, compiled, and executed from the
> command-line (FWIW i use Ubunto 11.04 Linux running Sun Java 6 and the
> Antlr version from the antlr-3.4-complete.jar file).
>
> Please try to post the *smallest* yet *complete* example of your
> problem.
>
> Hope this helps...
> -jbb
>
> >
> > I would like to avoid general backtracking, so after searching for a
> > while and reading the article
> > http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar
> > I have tried first
> >
> > ifStatement
> > : 'if' expression 'then' (singleStatement EOL)=> singleLineIf
> > | multiLineIf
> > ;
> >
> > or
> >
> > ifStatement
> > : 'if' expression 'then' (singleStatement EOL | codeBlock 'end if')
> > ;
> >
> > But they fail both with the same fatality.
> > How this case should be processed ?
>
>
> -------------- next part --------------
> grammar Test;
>
> options {
> output = AST;
> ASTLabelType = CommonTree;
> }
>
> tokens { SINGLE; MULTI; } // imaginary tokens go here
>
> @members {
> private static final String [] x = new String[] {
> "local x\n",
> "if x then return 1\n\n",
> "if x then if y then return 1\n\nend if\n",
> "local x;if x then return 1\n;",
> };
>
> public static void main(String [] args) {
> for( int i = 0; i < x.length; ++i ) {
> try {
> System.out.println("about to parse:`"+x[i]+"`");
>
> TestLexer lexer = new TestLexer(new ANTLRStringStream(x[i]));
> CommonTokenStream tokens = new CommonTokenStream(lexer);
> TestParser parser = new TestParser(tokens);
> TestParser.test_return p_result = parser.test();
>
> // System.out.format("the token stream:\%n");
> // for( int j = 0; j < tokens.size(); ++j ) {
> // Token token = tokens.get(j);
> // System.out.format("\%d: type = \%s, text = `\%s`\%n",
> // j,
> // tokenNames[token.getType()],
> // token.getText());
> // }
>
> CommonTree ast = p_result.tree;
> if( ast == null ) {
> System.out.println("resultant tree: is NULL");
> } else {
> System.out.println("resultant tree: " + ast.toStringTree());
> }
> System.out.println();
> } catch(Exception e) {
> e.printStackTrace();
> }
> }
> }
> }
>
> test : codeBlock EOF! ;
>
> codeBlock
> : (compoundStatement)*
> ;
>
> compoundStatement
> : (
> ifStatement
> | singleStatement
> ) (';' | EOL)
> ;
>
> singleStatement
> : localVariableDeclaration
> | funCall
> | 'return'^ expression
> ;
>
> ifStatement
> : singleLineIf
> | multiLineIf
> ;
>
> singleLineIf
> : 'if' expression 'then' singleStatement EOL
> -> ^(SINGLE expression singleStatement)
> ;
>
> multiLineIf
> : 'if' expression 'then' codeBlock 'end if'
> -> ^(MULTI expression codeBlock)
> ;
>
> localVariableDeclaration : 'local'^ ID+ ;
> funCall : ID '('^ args? ')'! ;
> args : expression (','^ expression)* ;
> expression : term (op^ term)* ;
> term : ID | NUMBER ;
> op : '+' | '-' | '*' | '/' ;
>
> EOL : '\r'? '\n' ;
>
> WS : (' ' | '\t')+ { skip(); };
>
> ID : LETTER ( LETTER | DIGIT )* ;
>
> NUMBER : DIGIT+ ;
>
> fragment LETTER : ('a'..'z')|('A'..'Z') ;
> fragment DIGIT : '0'..'9' ;
>
> ------------------------------
>
> Message: 5
> Date: Wed, 6 Jul 2011 19:15:59 -0400
> From: James Reid <james1018 at gmail.com>
> Subject: Re: [antlr-interest] Think I found a bug.
> To: Terence Parr <parrt at cs.usfca.edu>
> Cc: antlr-interest at antlr.org
> Message-ID:
> <CAPtNrFbfvCY_f3hLTR62RqEh+N-uFfiZdrzJckmUnuxNqzbBuA at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> This worked...
>
> @Override
> public void reset() {
> super.reset();
> p = skipOffTokenChannels(0);
> }
>
> Glad I could contribute in some small way.
>
> James
>
> On Wed, Jul 6, 2011 at 1:57 PM, Terence Parr <parrt at cs.usfca.edu> wrote:
>
> > Hi James. ack! i think reset() should call
> >
> > skipOffTokenChannels()
> >
> > Can you override reset to call super.reset() then skipOffTokenChannels()
> > and see if that works? If so, i can fix for 3.4
> > Ter
> > On Jul 5, 2011, at 5:30 AM, James Reid wrote:
> >
> > > Hi all,
> > >
> > > I think I found a bug but I want to be sure. I have a parser grammar
> > that
> > > I run two passes on. Here are short versions of the rules.
> > >
> > >
> > >
> > > firstpass
> > > : (collect_matches
> > > | collect_labels
> > > | .)*
> > > ;
> > >
> > >
> > > script
> > > : header? matches* EOF
> > > ;
> > >
> > >
> > > When I run the code I use a CommonTokenStream and do the following
> > >
> > > CommonTokenStream tokens = new CommonTokenStream(lex);
> > > MyParser parser = new MyParser(tokens);
> > >
> > > parser.firstpass();
> > >
> > > tokens.reset();
> > >
> > > parser.script();
> > >
> > >
> > > The problem comes when I do the tokens.reset(). If the very first token
> > is
> > > a comment (i.e. on the hidden channel) it is returned in parser.script()
> > and
> > > throws a NoViableAlternativeException because nothing in my grammar is
> > > expecting a comment. To get around this I do the following...
> > >
> > > //Reset the tokens back to the begining
> > > tokens.reset();
> > >
> > > //For some reason after the tokens have been buffered up if a
> > > hidden
> > > //token is the first token it is returned instead getting the
> > > //first non-hidden token. This little hack works around that.
> > > if (tokens.LT(1).getChannel() == Token.HIDDEN_CHANNEL){
> > > tokens.consume();
> > > }
> > >
> > > //now we can build the AST
> > > r=parser.script();
> > >
> > > If I consume the hidden token then the token stream points at the next
> > > on-channel token like it is supposed to do. Is this a bug or am I doing
> > > things wrong?
> > >
> > > Thanks,
> > > James
> > >
> > > PS: This is with Antlr 3.3
> > >
> > > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >
> >
>
>
> ------------------------------
>
> Message: 6
> Date: Thu, 7 Jul 2011 01:27:33 +0200
> From: S?bastien Kirche <sebastien.kirche at gmail.com>
> Subject: Re: [antlr-interest] left recursion removal
> To: "John B. Brodie" <jbb at acm.org>
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID:
> <CALGPt8bbeQOGHJEB8zzXYbyiMbjmQJoT6bqEqJEq5SbU03z7_g at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Le 7 juillet 2011 00:14, John B. Brodie <jbb at acm.org> a ?crit :
> > Greetings!
> >
> > [...]
> >
> > unable to reproduce.
> >
> > given your admittedly partial grammar, i tried to construct a complete
> > example by adding the missing elements and creating an AST (so i could
> > know the resultant parse).
> >
> > my test rig is attached.
> >
> > it runs without error when Tool'd, compiled, and executed from the
> > command-line (FWIW i use Ubunto 11.04 Linux running Sun Java 6 and the
> > Antlr version from the antlr-3.4-complete.jar file).
> >
> > Please try to post the *smallest* yet *complete* example of your
> > problem.
>
> Sorry for not having posted a more complete code before, I though that
> should have been enough...
>
> I have worked further on my grammar, with trying to define more
> precisely what an expression should be, with operator precedence. This
> made me rewrite the singleStatement by moving the funCall rule to the
> primary rule. But my problem about the if-then-else construct is not
> gone.
>
> I have stripped down my grammar, keeping only the expression and
> subsequent rules, removed all that define loops, switches, etc.
> Given that simplified grammar, the following script should pass the
> parsing, but it can't yet :
>
> ---------------------------------------
> string s1, s2
>
> if s1='42' then s2='421'
>
> if s2='421' then
> string s3
> s3='123'
> else
> string s4
> s4='666'
> end if
>
> ---------------------------------------
>
> Also, to Jim Idle : the language I would be able to parse is
> Powerbuilder. You can see it as a sort of basic, while it accepts
> syntax like i++ or s += 'foo' He has also the ability to accept inline
> sql statements... I tried to play on the EndOfLine delimiter to handle
> the two if-then-else syntaxes but without success.
>
> --
> S?bastien Kirche
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: pbifthenelse.g
> Type: application/octet-stream
> Size: 2646 bytes
> Desc: not available
> Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20110707/4b4433d8/attachment-0001.obj
>
> ------------------------------
>
> Message: 7
> Date: Wed, 6 Jul 2011 16:30:52 -0700
> From: Jim Idle <jimi at temporal-wave.com>
> Subject: Re: [antlr-interest] Lines that don't match a rule ...
> To: antlr-interest at antlr.org
> Message-ID: <6556c9d271c906dfb308bb73f0e8e262 at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Try:
>
> lines
> : (messagePattern LINE_BREAK)* EOF
> ;
>
> messagePattern
> : ('+' | '-') (unarySelector | keywordSelector | binarySelector)
> | { while (input.LA(1) != LINE_BREAK && input.LA(1) != EOF)
> input.consume(); }
> ;
>
> This will just eat the lines that don't
>
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of James Ladd
> > Sent: Wednesday, July 06, 2011 2:42 PM
> > To: antlr-interest at antlr.org
> > Subject: [antlr-interest] Lines that don't match a rule ...
> >
> >
> > Hi All,
> >
> > I hope I can get a little help on the grammar I have below.
> > It works to a degree but I'm having trouble with some edge cases and
> > that is stressing me a little.
> >
> > When I write something like the following I get the right response -
> > sequence of tokens.
> >
> > Im putting '\n' where I would have new lines.
> >
> > \n
> > + foo\n
> > \n
> >
> > What I can't seem to do is put in a rule that matches a line that
> > doesn't start with '+' or '-'
> > I get a no viable Alt Exception with all that I try.
> >
> >
> > My goal is to be able to detect lines starting with a '+' or a '-'
> > followed by a messagePattern.
> > And lines NOT starting with a '+' or a '-'. The grammar below does not
> > contain any rules for this, because I can't get them to work.
> >
> > Please help
> >
> > Rgs, James.
> >
> >
> > grammar Preprocessor;
> >
> > options {
> > language = Java;
> > }
> >
> > @header {
> > package compiler;
> >
> > }
> >
> > @lexer::header {
> > package compiler;
> > }
> >
> > lines
> > : (lineBreak | messagePattern)* EOF
> > ;
> >
> > messagePattern
> > : ('+' | '-') (unarySelector | keywordSelector | binarySelector)
> >
> > ;
> >
> > unarySelector
> > : NAME
> > ;
> >
> > keywordSelector
> > : (KEYWORD NAME)+
> > ;
> >
> > binarySelector
> > : BINARY_SYMBOL NAME
> > ;
> >
> > lineBreak
> > : LINE_BREAK ;
> >
> > LINE_BREAK: ('\r'? '\n' | '\r');
> >
> > NAME: ('a'..'z' | 'A'..'Z')('a'..'z' | 'A'..'Z' | '0'..'9')*;
> > KEYWORD: NAME':';
> > BINARY_SYMBOL: ('~'|'!'|'@'|'%'|'&'|'*'|'-
> > '|'+'|'='|'\\'|'|'|'?'|'/'|'>'|'<'|',') ('~'|'!'|'@'|'%'|'&'|'*'|'-
> > '|'+'|'='|'\\'|'|'|'?'|'/'|'>'|'<'|',')*;
> >
> >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-address
>
>
> ------------------------------
>
> Message: 8
> Date: Wed, 6 Jul 2011 16:40:06 -0700
> From: Jim Idle <jimi at temporal-wave.com>
> Subject: Re: [antlr-interest] left recursion removal
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID: <f957627cd66bd58cdc0ac4c81c8c6234 at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> yes - I pretty sure that you need to pass in the EOL token to the parser
> for powerbuilder script to parse (it is pretty crummy but there is a lot
> of it out there). Here is a snippet from my VB.Net grammar which allows
> the various combinations. When you build the AST, it does not need to
> distinguish between single and multi lines.
>
> Also, do not worry about these things until you have the full expression
> tree working as that will sometimes make you say "Ahhhhhgggh, now I have
> to rework all my statements. Do:
>
> 1) Build the lexer, thinking ahead a bit about the parser;
> 2) Build the expression tree and generate the AST for it;
> 3) Build the language top down (though bottom up works too);
>
> Or you could pay me to do it as I have no work on right now ;-) boo hoo.
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of S?bastien Kirche
> > Sent: Wednesday, July 06, 2011 4:28 PM
> > To: John B. Brodie
> > Cc: antlr-interest
> > Subject: Re: [antlr-interest] left recursion removal
> >
> > Le 7 juillet 2011 00:14, John B. Brodie <jbb at acm.org> a ?crit :
> > > Greetings!
> > >
> > > [...]
> > >
> > > unable to reproduce.
> > >
> > > given your admittedly partial grammar, i tried to construct a
> > complete
> > > example by adding the missing elements and creating an AST (so i
> > could
> > > know the resultant parse).
> > >
> > > my test rig is attached.
> > >
> > > it runs without error when Tool'd, compiled, and executed from the
> > > command-line (FWIW i use Ubunto 11.04 Linux running Sun Java 6 and
> > the
> > > Antlr version from the antlr-3.4-complete.jar file).
> > >
> > > Please try to post the *smallest* yet *complete* example of your
> > > problem.
> >
> > Sorry for not having posted a more complete code before, I though that
> > should have been enough...
> >
> > I have worked further on my grammar, with trying to define more
> > precisely what an expression should be, with operator precedence. This
> > made me rewrite the singleStatement by moving the funCall rule to the
> > primary rule. But my problem about the if-then-else construct is not
> > gone.
> >
> > I have stripped down my grammar, keeping only the expression and
> > subsequent rules, removed all that define loops, switches, etc.
> > Given that simplified grammar, the following script should pass the
> > parsing, but it can't yet :
> >
> > ---------------------------------------
> > string s1, s2
> >
> > if s1='42' then s2='421'
> >
> > if s2='421' then
> > string s3
> > s3='123'
> > else
> > string s4
> > s4='666'
> > end if
> >
> > ---------------------------------------
> >
> > Also, to Jim Idle : the language I would be able to parse is
> > Powerbuilder. You can see it as a sort of basic, while it accepts
> > syntax like i++ or s += 'foo' He has also the ability to accept inline
> > sql statements... I tried to play on the EndOfLine delimiter to handle
> > the two if-then-else syntaxes but without success.
> >
> > --
> > S?bastien Kirche
>
>
> ------------------------------
>
> Message: 9
> Date: Wed, 6 Jul 2011 16:42:16 -0700
> From: Jim Idle <jimi at temporal-wave.com>
> Subject: Re: [antlr-interest] left recursion removal
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID: <5fecdc401ad60835d82fdc1bc3cd50d1 at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> With the example this time:
>
>
> ifStatement
> : IF e=expression
> (
> t=THEN
> (
> // Single line IF
> statement as the last token was not a NL
> //
> s1=statements
> ((ELSE)=>ELSE s2=statements)?
>
> -> {$s2.tree == null}?
> ^(IFLINE $e ^(THEN $s1))
> ->
> ^(IFLINE $e ^(THEN $s1) ^(ELSE $s2*))
>
> | (NL|COLON)+
>
> // We discovered a NL token, hence
> we process
> // a block If here as it has to be a
> block when the line with
> // the IF on it has a NL before the
> first statement.
> //
> ib=ifBlock
>
> -> ^(IF $e $ib)
> )
>
> | (NL|COLON)+
>
> // We did not see a THEN clause, but it is
> optional on block IF statements
> // (which is a bit silly, but a left over from
> VB 1 I suspect. Hence we process
> // a block If here.
> //
> ib=ifBlock
>
> -> ^(IF $e $ib)
> )
> ;
>
> ifBlock
> : tb=block
>
> (eib+=elseIfStatement)*
>
> (
> ec=elseClause // Else Clause
> handles END IF
> | END IF
> )
>
> -> ^(THEN $tb?) $eib* $ec?
> ;
>
> elseClause
> : ELSE^ (
> (NL!|COLON!)+ block END! IF!
> | statement
> )
> ;
>
>
> elseIfStatement
> : ELSEIF e=expression THEN? (NL|COLON)+ b=block
>
> -> ^(ELSEIF $e $b?)
> ;
>
> Where 'block' is one or more statements (might need zero or more for
> powerbuilder).
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of S?bastien Kirche
> > Sent: Wednesday, July 06, 2011 4:28 PM
> > To: John B. Brodie
> > Cc: antlr-interest
> > Subject: Re: [antlr-interest] left recursion removal
> >
> > Le 7 juillet 2011 00:14, John B. Brodie <jbb at acm.org> a ?crit :
> > > Greetings!
> > >
> > > [...]
> > >
> > > unable to reproduce.
> > >
> > > given your admittedly partial grammar, i tried to construct a
> > complete
> > > example by adding the missing elements and creating an AST (so i
> > could
> > > know the resultant parse).
> > >
> > > my test rig is attached.
> > >
> > > it runs without error when Tool'd, compiled, and executed from the
> > > command-line (FWIW i use Ubunto 11.04 Linux running Sun Java 6 and
> > the
> > > Antlr version from the antlr-3.4-complete.jar file).
> > >
> > > Please try to post the *smallest* yet *complete* example of your
> > > problem.
> >
> > Sorry for not having posted a more complete code before, I though that
> > should have been enough...
> >
> > I have worked further on my grammar, with trying to define more
> > precisely what an expression should be, with operator precedence. This
> > made me rewrite the singleStatement by moving the funCall rule to the
> > primary rule. But my problem about the if-then-else construct is not
> > gone.
> >
> > I have stripped down my grammar, keeping only the expression and
> > subsequent rules, removed all that define loops, switches, etc.
> > Given that simplified grammar, the following script should pass the
> > parsing, but it can't yet :
> >
> > ---------------------------------------
> > string s1, s2
> >
> > if s1='42' then s2='421'
> >
> > if s2='421' then
> > string s3
> > s3='123'
> > else
> > string s4
> > s4='666'
> > end if
> >
> > ---------------------------------------
> >
> > Also, to Jim Idle : the language I would be able to parse is
> > Powerbuilder. You can see it as a sort of basic, while it accepts
> > syntax like i++ or s += 'foo' He has also the ability to accept inline
> > sql statements... I tried to play on the EndOfLine delimiter to handle
> > the two if-then-else syntaxes but without success.
> >
> > --
> > S?bastien Kirche
>
>
> ------------------------------
>
> Message: 10
> Date: Wed, 6 Jul 2011 16:51:43 -0700
> From: Terence Parr <parrt at cs.usfca.edu>
> Subject: Re: [antlr-interest] Think I found a bug.
> To: james1018 at gmail.com
> Cc: antlr-interest at antlr.org
> Message-ID: <2804C178-36DD-4203-A1BE-4640612C1C99 at cs.usfca.edu>
> Content-Type: text/plain; charset=us-ascii
>
> tanx. fixed for 3.4
> Ter
> On Jul 6, 2011, at 4:15 PM, James Reid wrote:
>
> > This worked...
> >
> > @Override
> > public void reset() {
> > super.reset();
> > p = skipOffTokenChannels(0);
> > }
> >
> > Glad I could contribute in some small way.
> >
> > James
> >
> > On Wed, Jul 6, 2011 at 1:57 PM, Terence Parr <parrt at cs.usfca.edu> wrote:
> > Hi James. ack! i think reset() should call
> >
> > skipOffTokenChannels()
> >
> > Can you override reset to call super.reset() then skipOffTokenChannels() and see if that works? If so, i can fix for 3.4
> > Ter
> > On Jul 5, 2011, at 5:30 AM, James Reid wrote:
> >
> > > Hi all,
> > >
> > > I think I found a bug but I want to be sure. I have a parser grammar that
> > > I run two passes on. Here are short versions of the rules.
> > >
> > >
> > >
> > > firstpass
> > > : (collect_matches
> > > | collect_labels
> > > | .)*
> > > ;
> > >
> > >
> > > script
> > > : header? matches* EOF
> > > ;
> > >
> > >
> > > When I run the code I use a CommonTokenStream and do the following
> > >
> > > CommonTokenStream tokens = new CommonTokenStream(lex);
> > > MyParser parser = new MyParser(tokens);
> > >
> > > parser.firstpass();
> > >
> > > tokens.reset();
> > >
> > > parser.script();
> > >
> > >
> > > The problem comes when I do the tokens.reset(). If the very first token is
> > > a comment (i.e. on the hidden channel) it is returned in parser.script() and
> > > throws a NoViableAlternativeException because nothing in my grammar is
> > > expecting a comment. To get around this I do the following...
> > >
> > > //Reset the tokens back to the begining
> > > tokens.reset();
> > >
> > > //For some reason after the tokens have been buffered up if a
> > > hidden
> > > //token is the first token it is returned instead getting the
> > > //first non-hidden token. This little hack works around that.
> > > if (tokens.LT(1).getChannel() == Token.HIDDEN_CHANNEL){
> > > tokens.consume();
> > > }
> > >
> > > //now we can build the AST
> > > r=parser.script();
> > >
> > > If I consume the hidden token then the token stream points at the next
> > > on-channel token like it is supposed to do. Is this a bug or am I doing
> > > things wrong?
> > >
> > > Thanks,
> > > James
> > >
> > > PS: This is with Antlr 3.3
> > >
> > > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >
> >
>
>
>
> ------------------------------
>
> Message: 11
> Date: Wed, 06 Jul 2011 23:08:05 -0400
> From: "John B. Brodie" <jbb at acm.org>
> Subject: Re: [antlr-interest] left recursion removal
> To: S?bastien Kirche <sebastien.kirche at gmail.com>
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID: <1310008085.28443.1.camel at gecko>
> Content-Type: text/plain; charset="utf-8"
>
> see attached.
>
> I, also, am available for hire, if you should opt for that...
>
> -jbb
>
> On Thu, 2011-07-07 at 01:27 +0200, S?bastien Kirche wrote:
> > Le 7 juillet 2011 00:14, John B. Brodie <jbb at acm.org> a ?crit :
> > > Greetings!
> > >
> > > [...]
> > >
> > > unable to reproduce.
> > >
> > > given your admittedly partial grammar, i tried to construct a complete
> > > example by adding the missing elements and creating an AST (so i could
> > > know the resultant parse).
> > >
> > > my test rig is attached.
> > >
> > > it runs without error when Tool'd, compiled, and executed from the
> > > command-line (FWIW i use Ubunto 11.04 Linux running Sun Java 6 and the
> > > Antlr version from the antlr-3.4-complete.jar file).
> > >
> > > Please try to post the *smallest* yet *complete* example of your
> > > problem.
> >
> > Sorry for not having posted a more complete code before, I though that
> > should have been enough...
> >
> > I have worked further on my grammar, with trying to define more
> > precisely what an expression should be, with operator precedence. This
> > made me rewrite the singleStatement by moving the funCall rule to the
> > primary rule. But my problem about the if-then-else construct is not
> > gone.
> >
> > I have stripped down my grammar, keeping only the expression and
> > subsequent rules, removed all that define loops, switches, etc.
> > Given that simplified grammar, the following script should pass the
> > parsing, but it can't yet :
> >
> > ---------------------------------------
> > string s1, s2
> >
> > if s1='42' then s2='421'
> >
> > if s2='421' then
> > string s3
> > s3='123'
> > else
> > string s4
> > s4='666'
> > end if
> >
> > ---------------------------------------
> >
> > Also, to Jim Idle : the language I would be able to parse is
> > Powerbuilder. You can see it as a sort of basic, while it accepts
> > syntax like i++ or s += 'foo' He has also the ability to accept inline
> > sql statements... I tried to play on the EndOfLine delimiter to handle
> > the two if-then-else syntaxes but without success.
> >
>
>
> -------------- next part --------------
> grammar pbifthenelse;
>
> options {
> output = AST;
> ASTLabelType = CommonTree;
> }
>
> @members {
> private static final String [] x = new String[] {
> "string s1, s2\n"+
> "\n"+
> "if s1='42' then s2='421'\n"+
> "\n"+
> "if s2='421' then\n"+
> " string s3\n"+
> " s3='123'\n"+
> "else\n"+
> " string s4\n"+
> " s4='666'\n"+
> "end if\n",
> "string s1 = '42', s2='999', s3='101'\n"+
> "\n"+
> "if s1='42' then s2='421'\n"+
> "\n"+
> "if s2='421' then\n"+
> " s3='123'\n"+
> "else\n"+
> " s3='666'\n"+
> "end if\n"
> };
>
> public static void main(String [] args) {
> for( int i = 0; i < x.length; ++i ) {
> try {
> System.out.println("about to parse:`"+x[i]+"`");
>
> pbifthenelseLexer lexer =
> new pbifthenelseLexer(new ANTLRStringStream(x[i]));
> CommonTokenStream tokens = new CommonTokenStream(lexer);
> pbifthenelseParser parser = new pbifthenelseParser(tokens);
> pbifthenelseParser.pgm_return p_result = parser.pgm();
>
> // System.out.format("the token stream:\%n");
> // for( int j = 0; j < tokens.size(); ++j ) {
> // Token token = tokens.get(j);
> // System.out.format("\%d: type = \%s, text = `\%s`\%n",
> // j,
> // tokenNames[token.getType()],
> // token.getText());
> // }
>
> CommonTree ast = p_result.tree;
> if( ast == null ) {
> System.out.println("resultant tree: is NULL");
> } else {
> System.out.println("resultant tree: " + ast.toStringTree());
> }
> System.out.println();
> } catch(Exception e) {
> e.printStackTrace();
> }
> }
> }
> }
>
> pgm : codeBlock EOF!;
>
> codeBlock : ( compoundStatement )* ;
>
> compoundStatement
> : ( /* empty */
> | localVariableDeclaration
> | ifStatement
> | singleStatement
> ) ( ';' | EOL )
> ;
>
> singleStatement : assignment ;
>
> assignment : IDENTIFIER OPEQ expression ;
>
> ifStatement
> : 'if' expression 'then'
> ((singleStatement ('else' singleStatement)? EOL)=>
> singleStatement ('else' singleStatement)?
> | codeBlock ('else' codeBlock)? 'end if'
> )
> ;
> // predicate is needed to distinguish between the singleStatment for a
> // single line IF and the singleStatement within a compoundStatement
> // comprising a codeBlock that contains just one statement within a
> // multi-line IF.
>
> localVariableDeclaration
> : dataType variableDeclaration (',' variableDeclaration)*
> ;
>
> variableDeclaration
> : IDENTIFIER ( OPEQ literal)?
> ;
>
> dataType
> : 'int' | 'integer'
> | 'long'
> | 'string'
> ;
>
>
> expression : logicalORExpr ;
>
> logicalORExpr : logicalANDExpr ('or' logicalANDExpr)* ;
>
> logicalANDExpr : logicalNOTExpr ('and' logicalNOTExpr)* ;
>
> logicalNOTExpr : 'not'? relationalExpr ;
>
> relationalExpr : binaryPlusMinus (logicalOP binaryPlusMinus)* ;
>
> binaryPlusMinus : unaryExpr (('+' | '-') binaryMulDiv)* ;
>
> unaryExpr : ('+' | '-')? binaryMulDiv ;
>
> binaryMulDiv : exponentExpr (('*' | '/') exponentExpr)* ;
>
> exponentExpr : primary ( '^' primary )* ;
>
> primary
> : ( '(' expression ')' )
> | literal
> | ( IDENTIFIER postFixExpr? )
> ;
>
> postFixExpr
> : '!'
> | '++'
> | '--'
> | '(' expressionList? ')' // funCall
> // placing funCall here permits expressions like: f(x) + g(y)
> ;
>
> expressionList : expression (',' expression)* ;
>
> logicalOP
> : OPEQ
> | OPNEQ
> | OPINF
> | OPINFEQ
> | OPSUP
> | OPSUPEQ
> ;
>
> literal
> : STRING_LITERAL
> | INTEGER_LITERAL
> ;
>
> WS : (' '|'\t'|'\u000C') {$channel=HIDDEN;} ;
>
> EOL
> : '\r' '\n' // DOS
> | '\r' // Mac
> | '\n' // Unix
> ;
>
> STRING_LITERAL
> : '\'' ( options{ greedy=false; }: ( ~('\\'|'\'') | ('\\' '\'') ) )* '\''
> {
> System.out.println("string>" + getText());
> }
> ;
>
>
> INTEGER_LITERAL : '0' | ( '1'..'9' ( '0'..'9' )* ) ;
>
> IDENTIFIER
> : LETTER (LETTER|'0'..'9'|'$'|'#'|'%'|'-')*
> {
> System.out.println("identifier>" + getText());
> }
> ;
>
> fragment LETTER : ('A'..'Z'|'a'..'z'|'_') ;
>
> OPEQ : '=' ;
> OPNEQ : '<>' ;
> OPINF : '<' ;
> OPINFEQ : '<=' ;
> OPSUP : '>' ;
> OPSUPEQ : '>=' ;
>
> ------------------------------
>
> Message: 12
> Date: Thu, 7 Jul 2011 08:47:35 +0200
> From: S?bastien Kirche <sebastien.kirche at gmail.com>
> Subject: Re: [antlr-interest] left recursion removal
> To: "John B. Brodie" <jbb at acm.org>
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID:
> <CALGPt8bsE1DaBrVrLEWbJfuSJPMMiBfdyuviXseXgMaa4-_hjw at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Le 7 juillet 2011 05:08, John B. Brodie <jbb at acm.org> a ?crit :
> > see attached.
> >
> > I, also, am available for hire, if you should opt for that...
>
>
> 2011/7/7 Jim Idle <jimi at temporal-wave.com>:
> > [...]
> > Or you could pay me to do it as I have no work on right now ;-) boo hoo.
>
>
> Hehe :o)
> Thank you very much for your feedback, I will look at it later today.
> I wish I could support you, but there is no funding in that PBScript
> parser project : I am working on it in my free time and if I manage to
> have something working, I plan to publish it as free open source code.
> It could serve for building a syntax highlighter for the
> scite/scintilla editor (a bit overkill maybe) and maybe it could also
> be reused with Eclipse / IMP (though the parser generator used by IMP
> is LPG, i am not sure if / how to use a custom antlr produced parser).
> These tools are first to help my every day work with Powerbuilder.
>
> Do you suggest that I could earn some money with a parser for a dying
> language ? ;o)
> --
> S?bastien Kirche
>
>
> ------------------------------
>
> Message: 13
> Date: Thu, 07 Jul 2011 10:11:44 +0200
> From: Ivan Brezina <ibre5041 at ibrezina.net>
> Subject: [antlr-interest] C Target won't compile with MSVC
> To: antlr-interest at antlr.org
> Message-ID: <20110707101144.x5vzv6h5gk4kw0sc at webmail.kamm.cz>
> Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes";
> format="flowed"
>
> Hi
> I'm not sure whether this is a bug or not.
> In my grammar I use following rule to match Perl style
> quoted strings:
>
> /* Perl-style quoted string */
> QSTRING : ('q'|'Q') ( QS_ANGLE | QS_BRACE | QS_BRACK |
> QS_PAREN | QS_OTHER) ;
> fragment QS_ANGLE : QUOTE '<' ( options {greedy=false;} : . )* '>' QUOTE ;
> fragment QS_BRACE : QUOTE '{' ( options {greedy=false;} : . )* '}' QUOTE ;
> fragment QS_BRACK : QUOTE '[' ( options {greedy=false;} : . )* ']' QUOTE ;
> fragment QS_PAREN : QUOTE '(' ( options {greedy=false;} : . )* ')' QUOTE ;
>
> fragment QS_OTHER_CH: ~('<'|'{'|'['|'('|' '|'\t'|'\n'|'\r');
> fragment QS_OTHER
> @init {
> ANTLR3_UINT32 (*oldLA)(struct ANTLR3_INT_STREAM_struct *, ANTLR3_INT32);
> oldLA = INPUT->istream->_LA;
> INPUT->setUcaseLA(INPUT, ANTLR3_FALSE);
> }
> :
> QUOTE delimiter=QS_OTHER_CH
> /* JAVA Syntax */
> // ( { input.LT(1) != $delimiter.text.charAt(0) || ( input.LT(1) ==
> $delimiter.text.charAt(0) && input.LT(2) != '\'') }? => . )*
> // ( { input.LT(1) == $delimiter.text.charAt(0) && input.LT(2) ==
> '\'' }? => . ) QUOTE
> /* C Syntax */
> ( { LA(1) != $delimiter->getText(delimiter)->chars[0] || LA(2) !=
> '\'' }? => . )*
> ( { LA(1) == $delimiter->getText(delimiter)->chars[0] && LA(2) ==
> '\'' }? => . ) QUOTE
> { INPUT->istream->_LA = oldLA; }
> ;
>
> The problem is, that I need to declare my own variable inside @init section.
>
> Antlr generates such a C source:
>
> static ANTLR3_INLINE
> void mQS_OTHER(pOracleSQLLexer ctx)
> {
> ANTLR3_UINT32 _type;
> pANTLR3_COMMON_TOKEN delimiter;
>
>
> delimiter = NULL;
>
>
> ANTLR3_UINT32 (*oldLA)(struct ANTLR3_INT_STREAM_struct *,
> ANTLR3_INT32);
> oldLA = INPUT->istream->_LA;
> INPUT->setUcaseLA(INPUT, ANTLR3_FALSE);
>
> ... etc.
>
> MSVC complains that the variable oldLA is declared AFTER assignment
> statement. Could you modify the it into this?
>
> static ANTLR3_INLINE
> void mQS_OTHER(pOracleSQLLexer ctx)
> {
> ANTLR3_UINT32 _type;
> pANTLR3_COMMON_TOKEN delimiter = NULL;
>
> ANTLR3_UINT32 (*oldLA)(struct ANTLR3_INT_STREAM_struct *, ANTLR3_INT32);
> oldLA = INPUT->istream->_LA;
> INPUT->setUcaseLA(INPUT, ANTLR3_FALSE);
>
> ...etc
>
> Ivan
>
>
>
>
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
>
>
> ------------------------------
>
> Message: 14
> Date: Thu, 7 Jul 2011 05:36:11 -0700
> From: Jim Idle <jimi at temporal-wave.com>
> Subject: Re: [antlr-interest] C Target won't compile with MSVC
> To: antlr-interest at antlr.org
> Message-ID: <6f731513d5b601868af4f6bad31c420d at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> That is what the @declarations section is for. It will ensure the variable
> declaration is in the right place.
>
> http://antlr.markmail.org/search/C+declarations
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Ivan Brezina
> > Sent: Thursday, July 07, 2011 1:12 AM
> > To: antlr-interest at antlr.org
> > Subject: [antlr-interest] C Target won't compile with MSVC
> >
> > Hi
> > I'm not sure whether this is a bug or not.
> > In my grammar I use following rule to match Perl style quoted strings:
> >
> > /* Perl-style quoted string */
> > QSTRING : ('q'|'Q') ( QS_ANGLE | QS_BRACE | QS_BRACK |
> > QS_PAREN | QS_OTHER) ;
> > fragment QS_ANGLE : QUOTE '<' ( options {greedy=false;} : . )* '>'
> > QUOTE ;
> > fragment QS_BRACE : QUOTE '{' ( options {greedy=false;} : . )* '}'
> > QUOTE ;
> > fragment QS_BRACK : QUOTE '[' ( options {greedy=false;} : . )* ']'
> > QUOTE ;
> > fragment QS_PAREN : QUOTE '(' ( options {greedy=false;} : . )* ')'
> > QUOTE ;
> >
> > fragment QS_OTHER_CH: ~('<'|'{'|'['|'('|' '|'\t'|'\n'|'\r'); fragment
> > QS_OTHER
> > @init {
> > ANTLR3_UINT32 (*oldLA)(struct ANTLR3_INT_STREAM_struct
> > *, ANTLR3_INT32);
> > oldLA = INPUT->istream->_LA;
> > INPUT->setUcaseLA(INPUT, ANTLR3_FALSE);
> > }
> > :
> > QUOTE delimiter=QS_OTHER_CH
> > /* JAVA Syntax */
> > // ( { input.LT(1) != $delimiter.text.charAt(0) || (
> input.LT(1)
> > ==
> > $delimiter.text.charAt(0) && input.LT(2) != '\'') }? => . )*
> > // ( { input.LT(1) == $delimiter.text.charAt(0) &&
> input.LT(2)
> > ==
> > '\'' }? => . ) QUOTE
> > /* C Syntax */
> > ( { LA(1) != $delimiter->getText(delimiter)->chars[0] ||
> > LA(2) != '\'' }? => . )*
> > ( { LA(1) == $delimiter->getText(delimiter)->chars[0] &&
> > LA(2) == '\'' }? => . ) QUOTE
> > { INPUT->istream->_LA = oldLA; }
> > ;
> >
> > The problem is, that I need to declare my own variable inside @init
> > section.
> >
> > Antlr generates such a C source:
> >
> > static ANTLR3_INLINE
> > void mQS_OTHER(pOracleSQLLexer ctx)
> > {
> > ANTLR3_UINT32 _type;
> > pANTLR3_COMMON_TOKEN delimiter;
> >
> >
> > delimiter = NULL;
> >
> >
> > ANTLR3_UINT32 (*oldLA)(struct
> ANTLR3_INT_STREAM_struct
> > *, ANTLR3_INT32);
> > oldLA = INPUT->istream->_LA;
> > INPUT->setUcaseLA(INPUT, ANTLR3_FALSE);
> >
> > ... etc.
> >
> > MSVC complains that the variable oldLA is declared AFTER assignment
> > statement. Could you modify the it into this?
> >
> > static ANTLR3_INLINE
> > void mQS_OTHER(pOracleSQLLexer ctx)
> > {
> > ANTLR3_UINT32 _type;
> > pANTLR3_COMMON_TOKEN delimiter = NULL;
> >
> > ANTLR3_UINT32 (*oldLA)(struct ANTLR3_INT_STREAM_struct *,
> > ANTLR3_INT32);
> > oldLA = INPUT->istream->_LA;
> > INPUT->setUcaseLA(INPUT, ANTLR3_FALSE);
> >
> > ...etc
> >
> > Ivan
> >
> >
> >
> >
> >
> > ----------------------------------------------------------------
> > This message was sent using IMP, the Internet Messaging Program.
> >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-address
>
>
> ------------------------------
>
> Message: 15
> Date: Thu, 7 Jul 2011 06:57:41 -0700 (PDT)
> From: Roy Metzger <antlrmen at yahoo.com>
> Subject: [antlr-interest] loops and syntax-directed interpreter ?
> To: "antlr-interest at antlr.org" <antlr-interest at antlr.org>
> Message-ID:
> <1310047061.42770.YahooMailNeo at web121116.mail.ne1.yahoo.com>
> Content-Type: text/plain; charset=iso-8859-1
>
> Hello everyone and good day,
>
> I'm following Mr. Parr's tutorial on syntax-directed interpreter for Pie language
> (link: http://www.antlr.org/wiki/display/ANTLR3/Pie ). So, for this one? I'm not interested
> into building tree's and such.
>
> Now, the problem. I'm using code provided with the tutorial with minimal tweaks. My issue is with the while loop.
> It looks like this:
>
> while[boolean defer]:
> 'while' e=expr [defer]
> s=statement[defer]+
> 'stop' 'loop'
> {if (!defer) interp.allWhile($e.start, $s.start);} ;
>
>
> My issue is with starting location of the statements passed(token index).For example this works:
>
> while(true)
> print 1 //print unlimited number of 1's
> stop loop
>
>
> However, whenever I use multiple statements, only last one is executed, for example:
> while(true)
> print 1
> print 2
> print 3
> stop loop
>
> This will print non-stop 3's, and? not 1,2,3,1,2,3 as I would want etc.
>
> If there are multiple statements in the while loop, while function in the interpreter always rewinds input to
> the last statement in the while clasue(print 3 in above example, while I would like to rewind input to print 1).
>
> In the interpreter, callWhile function is not changed from the tutorial(aside statements instead of slist):
>
> ?public void callWhile(Token condStart, Token codeStart)
>
> {
> ?????? Boolean c = (Boolean)exec("expr", condStart.getTokenIndex());
>
> ??????? while ( c )
>
> {
> ??????????? exec("statement", codeStart.getTokenIndex());
> ??????????? c = (Boolean)exec("expr", condStart.getTokenIndex());
> ?? }
> ?? }
>
>
>
> Also, exec is not changed from the tutorial:
>
> ?public Object exec(String rule, int tokenIndex)
>
> {
> ??????? Object rv = null;
> ??????? try
>
> {
> ??????????? int oldPosition = parser.input.index(); // save current location
> ??????????? parser.input.seek(tokenIndex); // seek to place to start execution
> ??????????? try { // which rule are we executing?
> ??????????????? if ( rule.equals("expr") ) { rv = parser.expr(false).value; }
> ??????????????? else if ( rule.equals("statement") ) { parser.statement(false); }
> ??????????????? else listener.error("error: can't start at "+rule);
> ??? }
> ??????????? finally { parser.input.seek(oldPosition); }// restore location
> ?????? }
> ??????? catch (Exception e) {
> ??????????? listener.error("can't exec code @ index "+tokenIndex, e);
> ?????? }
> ??????? return rv;
> ?? }
>
> Now, I would really appreciate if someone could point me out why is this happening and
> how it can be fixed? Also, I've noticed that even code from the tutorial has similar issues.
>
> I would really appreciate any help, hints or tricks to help solve this problem.
> Thanks,
> Lee
>
> ------------------------------
>
> Message: 16
> Date: Thu, 7 Jul 2011 17:04:55 +0200
> From: S?bastien Kirche <sebastien.kirche at gmail.com>
> Subject: Re: [antlr-interest] left recursion removal
> To: "John B. Brodie" <jbb at acm.org>
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID:
> <CALGPt8ZeOHcgejygQsbmmbaCue2M7VvwwQ2uJnu33n-Li4x09g at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Le 7 juillet 2011 05:08, John B. Brodie <jbb at acm.org> a ?crit :
> > see attached.
>
> Many, many thanks, as your corrections on my example helped me
> considerably and the initial recursion is gone.
> I had also an obvious bug in my "NOT" expression. Now I have added
> some other structures like for/next and choose/case
>
> My current problem is that your refactor of my funCall rule by
> following an expression by a postfix notation seems not working on my
> test code.
> That looks clever, but during parsing, the Parser seem to look for a
> missing assignment between the function identifier and the parameter
> list. I did not found why yet.
>
> I join my grammar in its current state.
> --
> S?bastien Kirche
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: PowerScript.g
> Type: application/octet-stream
> Size: 8175 bytes
> Desc: not available
> Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20110707/55c5fd2c/attachment-0001.obj
>
> ------------------------------
>
> Message: 17
> Date: Thu, 07 Jul 2011 11:51:52 -0400
> From: "John B. Brodie" <jbb at acm.org>
> Subject: Re: [antlr-interest] left recursion removal
> To: S?bastien Kirche <sebastien.kirche at gmail.com>
> Cc: antlr-interest <antlr-interest at antlr.org>
> Message-ID: <1310053912.9154.2.camel at gecko>
> Content-Type: text/plain; charset="UTF-8"
>
> Greetings!
>
> On Thu, 2011-07-07 at 17:04 +0200, S?bastien Kirche wrote:
> > Le 7 juillet 2011 05:08, John B. Brodie <jbb at acm.org> a ?crit :
> > > see attached.
> >
> > Many, many thanks, as your corrections on my example helped me
> > considerably and the initial recursion is gone.
>
> you are welcome. i actually enjoy doing this stuff.
>
> > I had also an obvious bug in my "NOT" expression. Now I have added
> > some other structures like for/next and choose/case
> >
> > My current problem is that your refactor of my funCall rule by
> > following an expression by a postfix notation seems not working on my
> > test code.
> > That looks clever, but during parsing, the Parser seem to look for a
> > missing assignment between the function identifier and the parameter
> > list. I did not found why yet.
> >
> > I join my grammar in its current state.
>
> change the singleStatement rule to this:
>
> singleStatement
> : IDENT ( OPEQ expression | '(' expressionList? ')' )
> | 'return' expression
> ;
>
> and delete the now unnecessary assignment rule.
> -jbb
>
>
>
>
> ------------------------------
>
> _______________________________________________
> antlr-interest mailing list
> antlr-interest at antlr.org
> http://www.antlr.org/mailman/listinfo/antlr-interest
>
> End of antlr-interest Digest, Vol 80, Issue 7
> *********************************************
More information about the antlr-interest
mailing list