Hi Anthony.<div><br></div><div>I completely agree that we should target the level of functions actually performing the projection rather than yt's organization. The mock frontend suggestion was just a hack to get there. I don't know if there's a way around it though...</div>
<div><br></div><div>Here's an example of what I sorted through to get to projections:</div><div>- Load a test plotfile, check pf.h.proj to find it's source.</div><div>- Read through data_objects/hierarchy.py and utilities/parallel_tools/parallel_analysis_interface.py to find where proj is attached, can't find it.</div>
<div>- The proj docstring says it is a reference to AMRQuadProj. Can't find a class by that name.</div><div>- Search data_objects sources for "proj", find AMRProjBase.</div><div><br></div><div>So it looks like the functionality is wrapped up in the __project_level and _project_grid methods. I can't think of a way to test those without creating an AMRProjBase, and that requires a staticoutput object.</div>
<div><br></div><div>So unfortunately, I think it would still come down to having a fake frontend. It's not ideal, but it seems like any more isolation would require big rewrites to yt.</div><div><br></div><div>Of course, I could be missing something. Matt, can you think of a better way?</div>
<div><br></div><div>- Casey</div>
<div><br></div><div><br><div class="gmail_quote">On Mon, Sep 24, 2012 at 11:02 AM, Anthony Scopatz <span dir="ltr"><<a href="mailto:scopatz@gmail.com" target="_blank">scopatz@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Helo Casey, <div><br></div><div>Sorry for taking the whole weekend to respond. </div><div><br><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>I would like to help with this, but it's difficult to figure out where to start.</div>
</blockquote></div></div></div></blockquote><div><br></div></div><div>Not to worry. I think that any of the items listed at the bottom of Matt's original email</div><div>would be a great place to start.</div><div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br></div><div>Say I want to test projections. I make a fake 3D density field, maybe something as simple as np.arange(4**3).reshape((4, 4, 4)). I write down the answer to the x-projection. Now all I need to do is call assert_allclose(yt_result, answer, rtol=1e-15), but I don't know what pieces of low-level yt stuff to call to get to `yt_result`. Hopefully that's clear...</div>
<div><br></div><div>Maybe this comes down to creating a fake frontend we can attach fields to?</div></blockquote></div></div></div></blockquote><div><br></div></div><div>Actually, I disagree with this strategy, as I told Matt when we spoke last week. </div>
<div><b>What is important is that we test the science and math parts of the code </b></div><div><b>before, </b><b>if ever, dealing with the software architecture that surrounds them. </b></div><div><b><br></b></div><div>
Let's taking your example of projections. What we need to test is the actual function</div>
<div>or method which actually slogs through the projection calculation. In many cases in</div><div>yt these functions are not directly attached to the front end but live in analysis, visualization</div><div>or utilities subpackages. It is these such packages that we should worry about testing.</div>
<div>We can easily create routines to feed them sample data. </div><div><br></div><div>On the other hand, testing or mocking things like frontends should be a very low priority. </div><div>At the end of the day what you are testing here is pulling in data from disk or other </div>
<div>sources. Effectively, this is just re-testing functionality present in h5py, etc. That is not </div><div>really our job. Yes, in a perfect world, front ends would be tested too. But I think that the</div><div>priority should be placed on things like the KDTree. </div>
<div><br></div><div>Be Well</div><span><font color="#888888"><div>Anthony</div></font></span><div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br></div><div>- Casey</div><div><br></div><div><br><div class="gmail_quote">On Fri, Sep 21, 2012 at 2:42 PM, Matthew Turk <span dir="ltr"><<a href="mailto:matthewturk@gmail.com" target="_blank">matthewturk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
As some of you have seen (at least Stephen), I filed a ticket this<br>
morning about increasing testing coverage. The other night Anthony<br>
and I met up in NYC and he had something of an "intervention" about<br>
the sufficiency of answer testing for yt; it didn't take too much work<br>
on his part to convince me that we should be testing not just against<br>
a gold standard, but also performing unit tests. In the past I had<br>
eschewed unit testing simply because the task of mocking data was<br>
quite tricky, and by adding tests that use smaller bits we could cover<br>
unit testable areas with answer testing.<br>
<br>
But, this isn't really a good strategy. Let's move to having both.<br>
The testing infrastructure he recommends is the nearly-omnipresent<br>
nose:<br>
<br>
<a href="http://nose.readthedocs.org/en/latest/" target="_blank">http://nose.readthedocs.org/en/latest/</a><br>
<br>
The ticket to track this is here:<br>
<br>
<a href="https://bitbucket.org/yt_analysis/yt/issue/426/increase-unit-test-coverage" target="_blank">https://bitbucket.org/yt_analysis/yt/issue/426/increase-unit-test-coverage</a><br>
<br>
There are a couple sub-items here:<br>
<br>
1) NumPy's nose test plugins provide a lot of necessary functionality<br>
that we have reimplemented in the answer testing utilities. I'd like<br>
to start using the numpy plugins, which include things like<br>
conditional test execution, array comparisons, "slow" tests, etc etc.<br>
2) We can evaluate, using conditional test execution, moving to nose<br>
for answer testing. But that's not on the agenda now.<br>
3) Writing tests for nose is super easy, and running them is too. Just do:<br>
<br>
nosetest -w yt/<br>
<br>
when in your source directory.<br>
<br>
4) I've written a simple sample here:<br>
<br>
<a href="https://bitbucket.org/yt_analysis/yt-3.0/src/da10ffc17f6d/yt/utilities/tests/test_interpolators.py" target="_blank">https://bitbucket.org/yt_analysis/yt-3.0/src/da10ffc17f6d/yt/utilities/tests/test_interpolators.py</a><br>
<br>
5) I'll handle writing up some mock data that doesn't require shipping<br>
lots of binary files, which can then be used for checking things that<br>
absolutely require hierarchies.<br>
<br>
--<br>
<br>
The way to organize tests is easy. Inside each directory with<br>
testable items create a new directory called "tests", and in here toss<br>
some scripts. You can stick a bunch of functions in those scripts.<br>
<br>
Anyway, I'm going to start writing more of these (in the main yt repo,<br>
and this change will be grafted there as well) and I'll write back<br>
once the data mocking is ready. I'd like it if we started encouraging<br>
or even mandating simple tests (and/or answer tests) for functionality<br>
that gets added, but that's a discussion that should be held<br>
separately.<br>
<br>
The items on the ticket:<br>
<br>
* kD-tree for nearest neighbor<br>
* Geometric selection routines<br>
* Profiles<br>
* Projections -- underlying quadtree<br>
* Data object selection of data containers<br>
* Data object selection of points<br>
* Orientation class<br>
* Pixelization<br>
* Color maps<br>
* PNG writing<br>
<br>
Is anyone willing to claim any additional items that they will help<br>
write unit tests for?<br>
<br>
-Matt<br>
_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
</blockquote></div><br></div>
<br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div>
</div></div></blockquote></div></div></div><br></div>
<br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div>