[antlr-interest] v3 - How to deal with include Files?

Loring Craymer lgcraymer at yahoo.com
Mon Sep 4 12:33:10 PDT 2006


I don't think that "lex everything twice" as Ter suggests is a good solution.  It does work--that's what a preprocessor does--but additional overhead is added to cover mapping positions (file and line directives).

I think that the better approach is to recognize "import foo" (or whatever) as a token in the lexer.  That works reasonably well with ANTLR 2 but may not work in ANTLR 3 yet (due to synpred problems).  Your lexer rule would look something like:

TEXT
    :
    (IMPORT)=> IMPORT
    |
    <standard TEXT def'n>
    ;

IMPORT
    :
    'i' "mport" <recognize rest of text, execute import actions, mark token as "SKIP">
    ;

(Separating the 'i' in "import" effectively means that the synpred is only invoked when there is a leading 'i'.)

--Loring

Alexandre Porcelli <porcelli at uol.com.br> wrote: Now I understood your solution, but isn't it more complicate? Because
you will need to handle lexical instructions inside the input stream
(you can have several different kinds of "include" statements in
several languages - include, copy, -inc, ...) and we can have same
more sofisticates include statements (like an ASP file that is not
just a statement is a tag).
And the second problem that I can see with this solution is to control
file positions (line and column position) because you will need to add
one more internal thing : fileName (to get info about what is the
active file name).
I´d like to know your opinion about this.

Regards,
Alexandre Porcelli


> hi. for includes, I'd probably just make my input character stream
> look for and do the includes so the lexer sees a continuous stream of
> char ready to process.
>
> Ter
> On Sep 4, 2006, at 9:44 AM, Alexandre Porcelli wrote:
>
>> Terence, I couldn't understand how to do it... I already saw the
>> island grammar sample but I couldn't understand how to "translate" it
>> to solve the include problem.
>>
>> Alexandre Porcelli
>>
>> On 9/4/06, Alexandre Porcelli  wrote:
>>> Terence, I couldn't understand how to do it... I already saw the
>>> island grammar sample but I couldn't understand how to "translate" it
>>> to solve the include problem.
>>>
>>> Alexandre Porcelli
>>>
>>>
>>> On 8/23/06, Alexandre Porcelli 
 wrote:
>>> > Hi All,
>>> >
>>> >   Id like to know how to deal with include Files?
>>> >
>>> >    In v2 there is TokenStreamSelector, how can I solve a include
>>> file
>>> > problem using v3?
>>> >
>>> > Best Regards,
>>> > Alexandre Porcelli
>>> >
>>>



 		
---------------------------------
 All-new Yahoo! Mail - Fire up a more powerful email and get things done faster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20060904/d5c04410/attachment.html 


More information about the antlr-interest mailing list