[antlr-interest] antlr python unit testing
Laurie Harper
laurie at holoweb.net
Thu May 3 22:51:43 PDT 2007
Hmm, I tried this but it only works if I re-assign stdin before creating
the lexer, which makes sense of course, but isn't really what I want.
I'm building an expression parser as part of a page templating system.
It will be used to parse an arbitrary number of expressions within the
template, each representing a dynamic reference to external data. So, I
need to be able to invoke the parser repeatedly on varying inputs.
That's the same behaviour I was looking for in order to unit test the
parser/lexer, hence this thread. The solution I came up with looks
something like:
class MockReader(object):
def setInput(self, text):
self.text = text
self.index = 0
def read(count):
start = self.index
end = self.index + count
self.index = end
return self.text[start, end]
class Test(unittest.TestCase):
def setUp(self):
self.L = Lexer()
self.P = Parser(self.L)
def test_first(self):
src = MockReader()
src.setInput('test expr 1')
self.L.setInput(src)
self.P.someRule()
src.setInput('test expr 2')
self.L.setInput(src)
self.P.someRule()
...
This seemed to be working, until I tried calling different rules in the
same test case. It looks like calling setInput() on the lexer leaves the
lexer and/or parser in an inconsistent state :-(
So, what is the general solution to re-using a parser instance with
different string inputs? Any suggestions would be greatly appreciated.
L.
Kaleb Pederson wrote:
> Hi Laurie,
>
> In python, stdin is just a file descriptor that you can change. So, if you
> wanted to manipulate it to read from a file, or some other object that had
> the same interface as a file descriptor, you could do so.
>
> For example:
>
> import sys
> sys.stdin = open('myfile','r');
>
> Then, if you needed to get the regular stdin back, sys.stdin = sys.__stdin__
>
> So, take advantage of the StringIO class:
>
> import StringIO
> import sys
>
> sys.stdin = StringIO.StringIO("My text to be parsed")
>
> Now you can run your unit tests using whatever text you want.
>
> Hope that helps.
>
> --Kaleb
>
> On Thursday 03 May 2007, Laurie Harper wrote:
>> I'm using ANTLR for Python and have my lexer and parser partially
>> working. I would like to start writing unit tests to identify exactly
>> what is and isn't right in what I have so far. (Call it test-first
>> debugging ;-)
>>
>> This is a total newbie question but, how do I pass input into my lexer
>> or parser from a Python script, rather than having it come from stdin?
>> Ideally, I'd like to create a lexer instance and a parser instance and
>> then write unit tests that exercise specific rules with varying input,
>> al la:
>>
>> import unittest
>> class LexerTests(unittest.TestCase):
>> def setUp(self):
>> self.L = lexer()
>> self.P = parser(self.L)
>>
>> def test_lexer(self):
>> input = "specimen input"
>>
>> # *****
>> self.L.INJECT_INPUT_CHARACTERS(input)
>> # *****
>>
>> self.failUnlessEqual(some_result, self.L.nextToken())
>>
>> def test_parser(self):
>> input = "specimen input"
>>
>> # *****
>> self.L.INJECT_INPUT_CHARACTERS(input)
>> # *****
>>
>> self.failUnlessEqual(some_result, self.P.someRule()
>>
>> How do I achieve the INJECT_INPUT_CHARACTERS(input) part?
>>
>> L.
>
>
>
More information about the antlr-interest
mailing list