Tuesday, July 1, 2008

Test environment pollution

For some wonderful reason, running doctests along with the unit tests pollutes the test environment and causes tests to fail (these same tests pass when doctests are not run). So figuring out this tangled mess of execution context looks to be a significant part of what I'll be doing the rest of the week. Random thoughts and information related to these issues follow. :)

I found that I could specify a "sanitized" globals dictionary for the doctests (in the loadTestsFromModule method that's been monkeypatched onto nose.plugins.doctests.Doctest), which, as far as I can tell, has the desired effect of simulating running the doctests in a Python that's been started up and had "import numpy as np" executed. This helps prevent doctests from depending on things that might have been imported into the module where they live, locally defined functions, etc.

Some of the tests make changes that affect other tests, like the doctests in lib/scimath.py that use numpy.set_printoptions to change the display precision in order to make test output predictable. I can monkeypatch an afterContext method onto nose.plugins.doctests.Doctest to restore the original state, but this seems like a fragile way to do things, since I have to code restoration of everything that might get changed. (At least it works, though.)

Somehow, when doctests are run, the memmap module replaces the memmap class in the execution context of the test_memmap.py tests, and so they fail. I really don't know how that's happening yet.