[antlr-interest] Confusing, hopefully-final, problems

[Sam Barnett-Cormack]

Jim Idle wrote:
> Sam Barnett-Cormack wrote:
>> Sam Barnett-Cormack wrote:
>>   
>>> Sam Barnett-Cormack wrote:
>>>     
>>>
>>> type : ((typeWithConstraint)=>typeWithConstraint | builtinType | 
>>> referencedType) constraint* ;
>>>
>>> And a load of parser warnings and errors that it wasn't bothering to 
>>> generate before... *sigh*
>>>     
>>
>> Or even *lexer* warnings and errors

(turns out those were the result of weird behaviour I've been seeing 
that I can only assume is a bug; I'll report on that separately)

> I think that you need to start again and go a lot more slowly in 
> building up your grammar. There is no point asking about each and every 
> error you are getting, as you won't learn anything. Do this:

Well, I think I've been learning a lot... nowhere near every error I've 
got has come here, only the ones I've failed to trace on my own. After 
getting a pointer on a few, I've managed to fix most of the rest myself. 
I'm currently wrestling with something odd that I can't see how to 
resolve, but I'm trying to understand it myself before I ask for help. I 
apologise if I'm currently too incompetent to be worth anyone helping.

> 1) Do not use 'xxxx' type things in the parser rules;

I'm aware of the issues - they lead to a more readable grammar, more 
natural-feeling, and I'm pretty damn sure that they aren't causing any 
of my problems.

> 2) Start with a blank lexer and parser;

Well, I certainly did this time.

> 3) Build the parser rules for expressions, by starting with the primary 
> elements and gradually inserting rules in the expression tree one at a 
> time - compile and test after each rule, fix issue as soon as you see 
> them, do not leave them to accumulate as you will get nowhere;
> 4) Now add one parser rule at a time (you will sometimes need a few at 
> once of course) and compile and test after each change - compile and 
> test after each rule, fix issue as soon as you see them, do not leave 
> them to accumulate as you will get nowhere;

Now, I'll admit I did get a bit tied up, and starting like that 
originally would have been good (and was what I was trying to do). Once 
it seemed to be going okay, I got frustrated looking between screen and 
standard so often, and let myself go too far. However, I doubt very much 
that starting again now would help. I'd just run into the remaining 
problem(s) from a different angle.

> The things you are seeing are standard things that happen when you start 
> out by typing in a grammar from the normative grammar of some language 
> spec. Such language specs are not really written with writing a parser 
> spec in mind (most say this) and even if they are they are usually 
> written with yacc/bison in mind. Consequently they are full of 
> ambiguities that you must left factor as you go.

Well, I didn't just blindly copy them out, although I'll admit I got 
tied up in left-recursion issues. The grammar actually seems to have 
some ambiguities that would be ambiguous anywhere, actually, which is a 
pain in the neck, by c'est la vie.

I also didn't want to refactor it anymore than necessary because there 
is a relationship between the structure of rules in the standard and the 
semantic structure, which should help at later stages.

> Unless you have built a lot of grammars, that is why you need to do this 
> in small steps - otherwise you will end up turning on backtracking and 
> adorning the grammar with predicates that include entire rules such as 
> (typeWithConstraint)=> when the real issue is that you have not left 
> factored the grammar correctly.

Oop, thanks for that one, I'd missed that when I was removing 
unnecessary predicates that were exploratory.

> Get a copy of the book if you can, it will help a lot, and read through 
> as much of the Wiki as you can before your fingers itch to get back at 
> the keyboard.

Got the book. Read it, in fact. Also have a good mark on languages & 
compilers from my undergrad days, but that doesn't really mean anything.
Read a lot of the wiki, which seems to largely have a more judgemental 
tone than the book, but read a lot of it anyway.

In case you're interested, and by way of demonstrating that I am making 
progress, the current state of the grammar is attached. Only one error 
stopping it compiling, and one non-breaking warning that I will want to 
fix before using this for anything.

Sam

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ASN_1.g
Url: http://www.antlr.org/pipermail/antlr-interest/attachments/20090302/1cfae2ce/attachment.pl