[antlr-interest] looking for rewrite rule - (UNION | EXCEPT)^ ALL? put in one token

[John B. Brodie]

Greetings!

On Mon, 2010-07-26 at 06:52 -0700, S.Eifert wrote:
> Hi everyone,
> 
> I'm working on an existing project with a SQL-Grammar. So I have strict
> limitations to watch while I have to enhance the functions.
> There's one rule which I just cant figure out how to solve. The following is
> in the grammar so far.
> 
> query_exp
>   : query_term ((UNION | EXCEPT)^ ALL?  query_term)*   
>   ;
> 
> - query_term can reference back on query_exp
> - UNION etc. are lexical rules to ignore casesensivity
> example:
> UNION : U N I O N;
> EXCEP : E X C E P T;
> The single letters are matched to their upper and lower counterpart.
> 
> I now need to change the way a UNION ALL is parsed. If I use the structure
> above ALL will be a child of UNION. But I need a single Token named
> UNIONALL. The rest of the structure needs to be the same. The treehandling
> is in a part I can not touch.
> 
> What I tried:
> 
> query_exp
>   : query_term ((UNION | UNIONALL | EXCEPT | EXCEPTALL)^ query_term)*   
>   ;
> 
> - This does not work, because the lexical rules for UNION / UNIONALL
> conflict each other. At least that seems to be the problem.

having UNIONALL and EXCEPTALL tokens produced by the lexer is (IMHO) the
best way to go (but i have an alternative below, if my lexing suggestion
does not work...)

add a tokens { UNIONALL; EXCEPTALL; } section to your grammar just after
your options {...} section but before any rule.

modify the lexer rules for UNION and EXCEPT to be:

UNION : U N I O N ( WS A L L { $type=UNIONALL; } )? ;
EXCEPT : E X C E P T ( WS A L L { $type=EXCEPTALL; } )? ;
fragment WS :
    ( ' ' | '\t' | '\r' | '\n' )+ ; // token delimiting white-space

and now your modified query_exp rule above should work....

> 
> I tried to use a rewrite rule. I used an Integer to keep track of which
> option (UNION/EXCEPT, maybe ALL) was chosen and put several conditioned
> rewrite rules. The problem was the Integer in the Parser was inside an
> if-clause and could not be read.
> 
> if ( state.backtracking==0 ) {
>               int MYINTEGER = 0;
>             }
> 
> Finally I tinkered with an additional rule where I build a String for the
> option taken (EXCEPT ALL, EXCEPT.. etc) and tried to return this. I think I
> was on the wrong track there.. it did not work at all.

the second alternative (if the above lexing changes do not work out) is
similar to this (sort of). e.g. use a parsing rule to recognize the
union/except operator.

still need the tokens { } section for UNIONALL and EXCEPTALL

no change to the lexer.

add parser rule (very much untested, but should work, i think):

query_operator : UNION | EXCEPT
   | (tok=UNION ALL -> UNIONALL[$tok,"UNIONALL"] )
   | (tok=EXCEPT ALL -> EXCEPTALL[$tok,"EXCEPTALL"] )
   ;

query_exp : query_term ( query_operator^ query_term )* ;

> 
> Probably the solution is very simple and I just cant find. I would be very
> thankfull for some input.
> I'm using ANTLR 3.1.2
> 

hope this helps...
   -jbb