[antlr-interest] Creating a Python-like language using Antlr 3.2

Luca Belluccini lucabelluccini at gmail.com
Sat May 29 11:28:34 PDT 2010


I'm trying to develop a Python-like language for academic purposes.
This language will be turned into Twincat language (a language similar
to PASCAL used to control BECKHOFF PLCs).
We're not implementing classes or so on, but it will meet the
following requirements:
- INDENT/DEDENT code blocks
- data types will be handled as virtual classes (e.g. INTEGER in
Twincat is mapped using a "fake" out.integer object in case of output
data, in.integer in case of input data; out.time for TIME variables
and so on...)
I was trying to edit correctly the Python grammars written by Terence
Parr and Loring Craymer.
I understood mostly the whole grammar (except some options such as
'greedy'). I bought the book "The Definitive ANTLR Reference" and I
read some stuff about parsing Python and multiple token emission.
The implementations I found, 2.5 and 2.3.3 are using a PythonTokenStream.java.
Is it possible to use the following code instead of the specific
PythonTokenStream Class (and add the INDENT/DEDENT rule into the Antlr
Grammar file, as in the hint at page 110) ?

@lexer::members {
List tokens = new ArrayList();
public void emit(Token token) {
        state.token = token;
    	tokens.add(token);
}
public Token nextToken() {
    	super.nextToken();
        if ( tokens.size()==0 ) {
            return Token.EOF_TOKEN;
        }
        return (Token)tokens.remove(0);
}
}

Thanks in advance, I hope to read some comments.


More information about the antlr-interest mailing list