[antlr-interest] python-lang parser to python target

Aaron Maxwell amax at redsymbol.net
Fri Jul 18 14:55:52 PDT 2008


Hi all,

I'm using antlr 3.1b2 to generate a python-language parser, using Frank 
Wierzbicki's python 2.5 grammar [0].  I'm generating it to the Python target.  
The lexer has some java code interspersed - for example, the PythonLexer 
constructor sports this at the end:
{{{
    /** Handles context-sensitive lexing of implicit line joining such as
     *  the case where newline is ignored in cases like this:
     *  a = [3,
     *       4]
     */
    int implicitLineJoiningLevel = 0;
    int startPos=-1;
}}}

This snippet is identical to what is in the corresponding spot in the java 
target lexer.  There are other small places where one or two lines of java 
code exist, i.e. it will say "implicitLineJoiningLevel++;" instead 
of "self.implicitLineJoiningLevel += 1".  In MOST of these, the java code is 
wrapped by Python comment lines saying "#action start" and "#action stop".

What can I do about this?  I'm happy to contribute code, and would love to 
decipher the bug and contribute a fix.  I'm not fluent in Java and very new 
to Antlr, though, so I'm not sure how likely that is.  Any pointers 
appreciated.

As for what I'm using this for, I have a semi-immediate need for it to "just 
work", and as a fallback I'm planning to just hand-convert the inline java 
code to python.  This looks pretty straightforward, except for the emit() 
function - e.g.:
{{{
                #action start
                emit(new ClassicToken(NEWLINE,nl.getText()));
                #action end
}}}

What would the Python equivalent of emit() be?

Thanks,
Aaron

[0] http://www.antlr.org/grammar/1200715779785/Python.g

-- 
Aaron Maxwell
http://redsymbol.net


More information about the antlr-interest mailing list