[antlr-interest] Backtracking and labels
Eric
researcher0x00 at gmail.com
Sun Mar 18 07:24:32 PDT 2012
Hi Kieran,
I looked at the github files and see that you are working with the C
target. I have never worked with the C target, that is Jim Idle's world.
I also noticed that you mention the grammar works with ANTLR 3.3 but that
you want to use ANTLR 3.4. If it were me and the grammar worked with ANTLR
3.3, I would skip the path of trying to fix ANTLR 3.4; you will learn a lot
by trying to fix ANTLR 3.4 but will you actually be productive on your
ultimate goal? I have spent time modifying ANTLR to extend for parsing
binary files; learned a lot but all of the changes went to bit heaven.
If there is something in the grammar forcing you to use 3.4, I would take
the easier route and see if I could modify the grammar to work with 3.4 as
is or 3.3.
One of the ways I view a grammar is as a high level language for parsing.
If I can't define what I need through the grammar, I use actions. Think of
a grammar as a high level language and actions as assembly language. It is
amazing how much you can do with actions. Remember all parts of a grammar,
including predicates are translated into the target language. Instead of
trying to patch ANTLR to use a predicate, just recreate the predicate using
actions; it should be easier than fixing ANTLR for someone who doesn't
understand all of ANTLRs internals.I don't even understand all of ANTLRs
internals.
Hope that helps,
Eric
On Sun, Mar 18, 2012 at 4:21 AM, Kieran Simpson <kierans777 at gmail.com>wrote:
> Eric (and list)
>
> I had a shot at porting Sam's fixes to the C target. While it does indeed
> fix the label generation issue (yay!), it unfortunately introduces a number
> of other changes which results in uncompilable code :'( :'(
>
> I've forked the antlr3 repo at https://github.com/kierans/**antlr3<https://github.com/kierans/antlr3>and pushed up my current changes. If anyone has any suggestions (or wishes
> to fork and add some patches) they're more than welcome. With some more
> tweaking hopefully this bug will go away :D
>
> Cheers,
>
>
> On 17/03/12 3:48 PM, Eric wrote:
>
>> Hi Kieran,
>> Did one of my crazy suggestions actually pan out?
>> I took a quick 30 second look at the code and it looks doable. When you
>> mentioned Sam I thought it would be C# code, but is actually a java,
>> grammar, and string template patch. I'm not sure if all three patches in
>> the update have to be applied. If I wasn't going to sleep right now I
>> would take a shot at it.
>> It might be as simple as applying the patch to the files and building.
>> Don't let the unknown file types scare you, just treat it like any patch
>> that you would do by hand, looking to make sure that the before and
>> after parts match up and there is a good change it will work.
>> I should be able to help you tomorrow with it. Drop a message if you
>> need help and I will jump in when I see it.
>> Good luck and great job on finding it, that fix is wanted by at least
>> several people. If you get it to work you might become someones hero.
>> Eric.
>>
>>
>> On Sat, Mar 17, 2012 at 12:17 AM, Kieran Simpson <kierans777 at gmail.com
>> <mailto:kierans777 at gmail.com>> wrote:
>>
>> Further to the related discussion of the Java grammar not compiling
>> (see
>> http://antlr.markmail.org/**search/?q=java+parser+not+**
>> compiling#query:java%20parser%**20not%20compiling+page:1+mid:**
>> vyevxqph4xniwl6q+state:results<http://antlr.markmail.org/search/?q=java+parser+not+compiling#query:java%20parser%20not%20compiling+page:1+mid:vyevxqph4xniwl6q+state:results>
>> ) I've found Sam's fix for the labels not being generated properly.
>>
>> It's
>> https://github.com/antlr/**antlr3/commit/**
>> db6adaefb12a1901606f81c9c8931a**a07d12949f<https://github.com/antlr/antlr3/commit/db6adaefb12a1901606f81c9c8931aa07d12949f>
>> and building against the current head of the antlr3 repo, the fix still
>> holds for the Java target, however the C target still has the bug.
>>
>> I'm not that familiar with Antlr's internal workings, nor String
>> Template. If anyone can help it would be appreciated (not just by my
>> I'm sure).
>>
>> Cheers,
>>
>>
>> On 23/07/64 5:59 AM, Kieran Simpson wrote:
>> > As a follow up, I checked out the antlr3 repo from Git Hub and
>> built the
>> > current master branch. Still have the same problem
>> >
>> > [java] '-classpath'
>> > [java]
>> >
>> './lib/ST4-4.0.4.jar:./lib/**antlr-2.7.7.jar:./lib/antlr-3.**
>> 4.1-SNAPSHOT.jar:./lib/antlr-**runtime-3.4.1-SNAPSHOT.jar:./**
>> lib/stringtemplate-3.2.1.jar'
>> >
>> >
>> > Cheers,
>> >
>> > On 11/03/12 5:43 PM, Kieran Simpson wrote:
>> >> @Bart - yes you're right, I forgot to add the leading ^ for the
>> >> resulting AST in my example. Typo :(
>> >>
>> >> @Ivan - I ran my build with Antlr 3.3 and the resulting code is OK
>> >>
>> >> static void
>> synpred9_CoreConfiguration_**fragment(**pCoreConfigurationParser
>> >> ctx )
>> >> {
>> >> pANTLR3_COMMON_TOKEN value;
>> >>
>> >> value = NULL;
>> >>
>> >> However I want to use 3.4 ;)
>> >>
>> >> I was thinking it might be a classpath issue, however I'm not
>> certain.
>> >> The classpaths are (from my Ant build):
>> >>
>> >> [java] '-classpath'
>> >> [java]
>> >>
>> './lib/ST4-4.0.4.jar:./lib/**antlr-2.7.7.jar:./lib/antlr-3.**
>> 4.jar:./lib/antlr-runtime-3.4.**jar:./lib/stringtemplate-3.2.**1.jar'
>> >>
>> >>
>> >>
>> >> [java] '-classpath'
>> >> [java]
>> >>
>> './lib/antlr-2.7.7.jar:./lib/**antlr-3.3.jar:./lib/antlr-**
>> runtime-3.3.jar:./lib/**stringtemplate-3.2.1.jar'
>> >>
>> >>
>> >>
>> >> What is really bothering my is that the the code generated from
>> >> Antlrworks (when I Debug the grammar) is OK
>> >>
>> >> public final void synpred9_CoreConfiguration_**fragment() throws
>> >> RecognitionException {
>> >> Token value=null;
>> >>
>> >> If I can get some pointers about where to look/what to look for
>> in the
>> >> code generators (eg: template fragments) I might be able to get
>> more
>> >> info on the code that's playing up. It could be a bad JAR that's
>> being
>> >> retrieved by my build system (Apache Ivy).
>> >>
>> >> Cheers,
>> >>
>> >> On 23/07/64 5:59 AM, Ivan Brezina wrote:
>> >>> Hi
>> >>> maybe you're hitting already discovered bug. This was
>> introduced in the
>> >>> version 3.4. What does the output look like when you use ANTLR
>> version
>> >>> 3.3?
>> >>>
>> >>> Ivan
>> >>>
>> >>> Quoting Kieran Simpson <kierans777 at gmail.com
>> <mailto:kierans777 at gmail.com>>**:
>>
>> >>>
>> >>>> For my rule:
>> >>>>
>> >>>> rule
>> >>>> : OPTION1
>> >>>> | value=TION2 -> ($value)
>> >>>> ;
>> >>>>
>> >>>> with backtracking on (and memoize not that I think that's
>> relevant) the
>> >>>> synpredicate code generated has errors, both for the Java and
>> C targets
>> >>>> where the variable name is missing in the declaration
>> >>>>
>> >>>> eg:
>> >>>>
>> >>>> public final void synpred9_grammar_fragment() throws
>> >>>> RecognitionException {
>> >>>> Token =ll;
>> >>>>
>> >>>> and
>> >>>>
>> >>>> static void synpred9_grammar_fragment(**pCoreConfigurationParser
>> ctx )
>> >>>> {
>> >>>> pANTLR3_COMMON_TOKEN ;
>> >>>>
>> >>>> =ULL;
>> >>>>
>> >>>> Interestingly this grammar runs in debug mode when used with
>> Antlrworks
>> >>>> even though I'm using version 3.4 of Antlr with my project and
>> 3.4 is
>> >>>> included with Antlrworks (1.4.3).
>> >>>>
>> >>>> Just in case it's due to an out of date version, here are some
>> md5sums
>> >>>>
>> >>>> 9638f9b7b1b823fcaba977f06f0dd7**61 ./antlr/3.4/antlrworks-1.4.3.
>> **jar
>> >>>>
>> >>>> 465472089d78cd80f926f5825a29ec**7f ./lib/antlr-3.4.jar
>> >>>> 0e0318be407e51fdf7ba6777eabfdf**73 ./lib/antlr-runtime-3.4.jar
>> >>>>
>> >>>> Is something out of date here or is this a bug with the
>> backtracking
>> >>>> generation?
>> >>>>
>> >>>> Cheers,
>>
>
More information about the antlr-interest
mailing list