I've discovered that my seat-of-the-pants install of ATLAS a week or so ago caused a from-scratch build of NumPy to be broken. :/ So I'm now burning up some time doing a correct install of ATLAS/LAPACK, and maybe I'll actually be able to "import numpy" by the end of the day. (Yes, my machine is that slow).

The monkeypatching madness is now gone from numpy.testing, as are the decorators. There have been some minor additions/changes to support SciPy tests (like moving print_assert_equal to NumPy so there don't have to be 3 separate but identical implementations in SciPy) and make the output more informative (show NumPy/SciPy/Python versions and say which tests are being run).

I submitted patches for the __mul__ and __mod__ problems of chararray, since the fixes would actually change the current behavior of the class, and I figure that's best looked at by others. This presumes that somebody's actually using the class.... ;) The 1.1.x development should be done soon, and then there should be plenty of eyes on the 1.2 defects, so the fix should still fit into the GSoC schedule.

Jarrod also asked if I could standardize the NumPy imports (as "import numpy as np") whenever I run across them, so I've been doing that a little this week in SciPy and NumPy. As soon as I can get my ATLAS/LAPACK issue sorted out, and I can be sure I haven't broken anything, I'll be checking some more of those changes in.

I'm also cleaning up the SciPy test output. Currently it's littered with deprecation warnings and (apparently) debugging output from the libraries.

Finally, since the deadline is in sight, I figured I should spend some time on the final documentation. I've got some scripts hacked together to automatically generate HTML tables (for overly detailed reports) and diff files from svn + numpy/scipy-svn mailing lists, which should take care of the bulk of the tedious work (which, for some reason, I did manually last year...).

SciPy has been switched over to use numpy.testing, scipy.testing has been removed, and it seems like everything is ok with the unit tests. Unlike NumPy, SciPy doesn't import all its subpackages on "import scipy", so the doctests need some slightly different execution context rules. Right now it looks like tests will have the module in which they are declared made available to them. So, for example, a doctest for scipy.linalg.qr will have linalg available in its context (an implicit "from scipy import linalg"). Of course this means there's a ton of SciPy doctests that need to be updated so that they'll run again.

The monkeypatching of the nose doctest plugin in numpy.testing.nosetester, which was originally a method or two, has gotten a little out of hand (as pointed out by others on the numpy-discussion list), so I'm in the process of changing over to use a new plugin (subclassed from the nose doctest plugin, numpy.plugins.doctest.Doctest). I'm also going to replace the decorators used to write the NoseTester method docstrings; since there's no other decorators used in NumPy, I'm not going to be the one that adds them. ;) Hopefully all that should be done tomorrow.

The __mul__ issue for defchararray was interesting; it appears if you pass a NumPy integer type to ndarray.__new__ as the itemsize argument, it uses the itemsize of the integer object rather than the value held by it, so (on my 32-bit machine) this meant that the return array always used an itemsize of 4 bytes. I'm not sure if this behavior is intentional.

Alright, so I didn't spend the entire week looking into the NumPy test environment pollution that occurs when doctests are run. I wanted to write some new tests to improve coverage, so I had to finish updating all the docstrings to run under the sanitized namespace, and get a more useful indication of coverage. That was a little tedious, but not difficult. I had to write a short script to generate annotated NumPy source (I didn't see any easy way to make nose do it), since the default coverage output--a list of uncovered line numbers--is impossible to use. For me, anyway.

Since nobody had suggested any specific bits of NumPy that they thought deserved more coverage, I just started slogging through the annotated source and looked for big blocks of uncovered code. There weren't any tests for chararray, so I started there.

The __mul__ operator doesn't seem to act right; string results that should be longer than 4 characters seem to get truncated (the resulting array dtype is S4 for some reason). The __mod__ operator doesn't seem to act like I'd expect either, but at the moment I'm willing to chalk it up to not knowing enough about how NumPy does things. So I'm off to go read more...

Test environment pollution

For some wonderful reason, running doctests along with the unit tests pollutes the test environment and causes tests to fail (these same tests pass when doctests are not run). So figuring out this tangled mess of execution context looks to be a significant part of what I'll be doing the rest of the week. Random thoughts and information related to these issues follow. :)

I found that I could specify a "sanitized" globals dictionary for the doctests (in the loadTestsFromModule method that's been monkeypatched onto nose.plugins.doctests.Doctest), which, as far as I can tell, has the desired effect of simulating running the doctests in a Python that's been started up and had "import numpy as np" executed. This helps prevent doctests from depending on things that might have been imported into the module where they live, locally defined functions, etc.

Some of the tests make changes that affect other tests, like the doctests in lib/scimath.py that use numpy.set_printoptions to change the display precision in order to make test output predictable. I can monkeypatch an afterContext method onto nose.plugins.doctests.Doctest to restore the original state, but this seems like a fragile way to do things, since I have to code restoration of everything that might get changed. (At least it works, though.)

Somehow, when doctests are run, the memmap module replaces the memmap class in the execution context of the test_memmap.py tests, and so they fail. I really don't know how that's happening yet.

NumPy doctest customizations committed

Since nobody objected to my warning post, the following features are now available for all NumPy 1.2 doctests:

1. All doctests in NumPy will have the numpy module available in their execution context as "np".
2. The normalized whitespace option is enabled for all doctests.
3. Adding "#random" to expected output will now cause that output to be ignored, but the associated example command will still be executed. So you can now do this to provide a descriptive example but not have it fail:
   

   >>> random.random()
   0.1234567890 #random: output may differ on your system
   

Later today I'll commit some changes to existing doctests that take advantage of these changes. Apparently some of the doctests haven't been run in a long time; tensorsolve, for example, referenced an undefined "wrap" function, and there were no unit tests to turn up this problem. :/ We're going to add a flag to the buildbot test command to run the doctests as well from now on, so that might help improve coverage a bit.

By the way, many thanks to whomever installed nose on the NumPy buildbots! :)

Oops, nevermind about <BLANKLINE>

One should not post first thing in the morning, it seems. Avoiding use of <BLANKLINE> in doctests has nothing to do with whitespace normalization. :)

Doctest tweaking

Based on a long discussion thread, I'm in the midst of hacking up the NumPy test framework to make the following changes:

1. All doctests in NumPy will have numpy in their execution context as "np" (i.e., there will be an implicit "import numpy as np" statement for all doctests, which makes the sample code shorter).
   
2. The "normalize whitespace" option is enabled for all doctests. This one didn't come up in the discussion, but I ran across a few existing examples that fail because of insignificant whitespace differences (like a space after a calculation result), and figured it would save a lot of pain on the part of docstring editors. Especially since the default failure message doesn't show that whitespace is causing the test to fail. (I think this should also avoid having to use the ugly <BLANKLINE> in docstrings, have to check).
   
3. Commands that have certain stuff in them will have their output ignored (but they will still be executed). At present the list of triggers is: ['plt.', 'plot.', '#random']. This avoids failures caused by output differences that aren't relevant: plotting function/method calls that return objects which (for some reason) include an address in their repr, examples that generate inherently random results, etc.
   

I may also turn on the detailed diff output for doctests as well; in some of the more complicated outputs, it's not always easy to spot some tiny difference in output. Come to think of it, the unittest fixture failure output could benefit from this, too, for the same reason.

All these changes will get run by the numpy-discussion list before they're actually committed, of course.