[antlr-interest] tree parser syntax

[Safiye Celik]

Hi,
I created two grammars, one is a lexer-parser grammar including tokens and
the parser rules, and the other is the tree grammar which walks on the tree
the lexer-parser grammar creates. But the problem is that, the two grammar
files create two .tokens files (each creates one) and the token reference
numbers are different in those files for the same token. And in the .tokens
file, the corresponding strings to the tokens are represented because there
are lexical rules in my lexer-parser grammar, whereas in the .tokens file of
the tree grammar, there are only the string names and their token numbers.
So, the two grammars do not know they are related. How should I make them
know this?Should I import lexer-parser grammar in the tree grammar or sth
like that?

Also, I have below two rules in my lexer-parser grammar:

a: b (x^ c)*;
b. d | y! a y!

and I want to parse the tree those rules create in the tree grammar file
with below rules:

a: b (x^ c)*;
b. d | a           // Here in the tree grammar I do not write "y"s which do
not exist in the tree

But I get left recursion error (for "a"s) in the second, tree grammar one.
Why? The tree has already been created, and I will parse it now, so there
should not be any left recursion problem here.

The questions I am asking may be very trivial, sorry for that. I am new to
ANTLR v3, and the document on ANTLR site telling about the differences
between the two versions is not enough for me to migrate to v3 easily.

Thanks in advance.

2009/8/19 Sam Barnett-Cormack <s.barnett-cormack at lancaster.ac.uk>

> Safiye Celik wrote:
>
>> Thanks for the answer. Now I wonder what is the way of putting that
>> INTEGER : ('0'..'9')+ to the tokens {} part of my tree grammar. I want this,
>> because I get no error for the tokens defined in tokens {} block in my tree
>> grammar. But I cannot do this directly as
>> tokens {
>> ...
>> INTEGER = '0'..'9'+
>> ...
>> }
>> I get an error of "Expecting SEMI, found '..'".
>> How can get rid of this error and put INTEGER token to tokens block?
>> Thanks in advance..
>>
>
> You need to create a separate lexer grammar first, defining your lexical
> rules. Before a tree parser is useful, you also need a conventional parser
> to build the tree the tree parser parses. You end up with three separate
> recognisers that feed into one another.
>
> Lexer produces token stream which is consumed by
> Parser produces AST which is consumed by
> TreeParser
>
> Sam
>
>  2009/8/19 Gavin Lambert <antlr at mirality.co.nz <mailto:
>> antlr at mirality.co.nz>>
>>
>>    At 00:59 20/08/2009, Safiye Celik wrote:
>>
>>        I get an error saying "lexer rule INTEGER not allowed in
>>        parser". How would I handle this issue? If I first write
>>        "grammar MyGrammar;" to my .g file and after listing rules and
>>        tokens write "tree grammar MyGrammar;" in order to parsing the
>>        tree, I get a syntax error. In ANTLR v2, we were able to declare
>>        lexer, parser, and then tree walker in the same .g file. What is
>>        the way of it in v3? Or how would I declare tokens such as
>>        INTEGER above to the tree grammar?
>>
>>
>>    In v3, each grammar must be in its own separate file.  (Though as a
>>    special case, a "combined" grammar can have both lexer and parser
>>    rules [not tree parser rules, though].)
>>
>>    The options section is used to tell the tree grammar where it will
>>    be getting its token definitions from.
>>
>>    Probably your best bet is to have a look at the examples (in the
>>    separate examples zipfile).
>>
>>
>>
>>
>> --
>> -safiye
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe:
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>
>
> --
> Sam Barnett-Cormack
>



-- 
-safiye
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090820/a2393fcb/attachment.html