[antlr-interest] Reading from sys.stdin with ANTLR-Python (for an REPL)

Thomas Brandon tbrandonau at gmail.com
Mon Feb 18 01:40:14 PST 2008


On Feb 18, 2008 4:52 PM, Aaron Lebo <aaron.m.lebo at gmail.com> wrote:
> Hi. I'm obviously using the Python runtime of ANTLR.
>
> Reading from a file and parsing it is trivial. The code looks a bit like:
>
> import antlr3
> import LangLexer
> import LangParser
>
> input = antlr3.ANTLRFileStream('filename...')
>  lexer = LangLexer.LangLexer(input)
> tokens = antlr3.CommonTokenStream(lexer)
> parser = LangParser.LangParser(tokens)
>
> So I'm having no issue with that. What I am wanting to do is an REPL and I
> need to interactively parse the data coming from sys.stdin. The problem is
> that all of the 'Stream' classes from the antlr3 package read all input on
> creation, so passing sys.stdin or sys.stdin.read() causes the REPL to hang,
> as something like antlr3.ANTLRInputStream(sys.stdin) reads infinitely.
>
> Am I making sense? What solution is there?
>
You could provide your own input stream implementation.
CommonTokenStream (in Java anyway) also reads all tokens on first
access so you would need to provide your own implementation there as
well.
I believe such an implementation has been posted on the list. For Java
but would give you something to convert.
Or you could lex\parse a line at a time. You could reuse the lexer and
parser. Either create a modifiable input stream or pass the lexer a
new one for each line. Either create a new token stream each line or
implement your own. This may be easier than supporting all the
possible cases in a truly incremental input design.

Tom.


More information about the antlr-interest mailing list