[antlr-interest] Free Text mixed with rules

[Loring Craymer]

Ter might have an example, but I don't know of one.  What you need from the runtime is

CommonToken.getStopIndex()
CommonToken.setStopIndex()

Then you track the first token in an expression, ! all following tokens and rule references in expression rules, and record the stop index from the last token in the expression back in the first token.  The sensible approach for finding the last token is probably to have your expression rules return the stop index from the last token/rule invocation found, and then use the final value from the top level expression rule.

--Loring


----- Original Message ----
From: victor panizza <victor.panizza at gmail.com>
To: antlr-interest at antlr.org
Sent: Wednesday, June 18, 2008 4:25:09 PM
Subject: Re: [antlr-interest] Free Text mixed with rules


Hi Loring,
 
Could you point me to an example??
 
Thanks in advance!!!


On Tue, Jun 17, 2008 at 7:49 PM, Loring Craymer <lgcraymer at yahoo.com> wrote:

If you want to recover text in its original format, you might as well use the start and stop markers on the first and last tokens of the "text" sequence from the input stream (ANTLR 3 keeps the text input stream in memory).  At the same time, you are probably best off in going ahead and including an expression grammar in your parser to handle what you are calling "free text"--otherwise, you will have to deal with nested conditionals in your lexer, and that gets very messy.  You can repackage the "text" segments in the output AST as a replacement for the nodes matched by the expression grammar.

--Loring 



----- Original Message ----
From: victor panizza <victor.panizza at gmail.com>
To: antlr-interest at antlr.org
Sent: Tuesday, June 17, 2008 5:17:29 PM
Subject: [antlr-interest] Free Text mixed with rules


Hello All, 
 
I have the follow issue:
 
I'm trying to develop a Parse Grammar and a Tree Parse for read PL/SQL scripts and translate it to DB2 SQL PL. Now, there're many diferences beetwen thems and this diferences are defined by the rules and some literals, but there're many things in common like the IF LOGICAL_OPERATION THEN END IF;
 
My Idea is create a Node in the tree (using a token like FREE_TEXT) where I can put each segment of free text like the above and later walk the tree and get this free text in the same order on they were recongnized. For example: 
 
IF (LAST_DAY(PDATE)!=PDATE) THEN
  PDATEAUX := ADD_MONTHS(PDATE, PMONTHS);
  RETURN ADD_MONTHS(PDATE, PMONTHS);
END IF;
 
In text tree representation:
 
(ROOT
          (FREE_TEXT "IF (LAST_DAY(PDATE)!=PDATE) THEN")
          (ASSING "PDATEAUX := PDATE;")
          (FREE_TEXT "RETURN ADD_MONTHS(PDATE, PMONTHS);")
          (FREE_TEXT "END IF;);")
)
 
 
Any idea, thanks in advance!!!!
 
-- 
Saludos,
Victor Panizza 



-- 
Saludos,
Victor Panizza 


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080619/5434ab58/attachment-0001.html