[yt-svn] commit/yt: ngoldbaum: Merged in xarthisius/yt (pull request #2058)
commits-noreply at bitbucket.org
commits-noreply at bitbucket.org
Wed Mar 23 13:53:38 PDT 2016
1 new commit in yt:
http://link.bitbucket.org/wf/click?upn=8USRlNyft-2BCzk2l4Ywl6gDx2lD2xxoS9E7MwXb2SMR-2BXcTD42YocdnOFkyGBVHOU6E6PJ1U3Xk7rLdXLrUtxz03bcGaDU9wCLVfXZlT-2F-2BV0-3D_ll4ctv0L-2ByeRZFC1LslHcg6aJmnQ70VruLbmeLQr27BJwZjKog4Ujn-2Bi-2B9HmWCKg-2F8dA1r-2BeaKRlnKUM6YpD0yHiIrknJU5jKhkWPemmctGoJoghweiFdjUfdUOD-2Bvnk91PA-2FmVjCtvnNMK8BgwPJZH6OPNtUsemAElj9pf5WKVY-2F2LLH6NRWpxudQcLrBMv2UOLMXB0f9keuwKq4lo2-2Bovai6NipuGQ74tDgJ-2FqhaQ-3D
Changeset: ee78f27e30fb
Branch: yt
User: ngoldbaum
Date: 2016-03-23 20:53:30+00:00
Summary: Merged in xarthisius/yt (pull request #2058)
Update answer testing docs
Affected #: 1 file
diff -r 7538e9cb964c701a9bef8dcb3a788119997b97a8 -r ee78f27e30fbc234af0d10107fc4022f5d42d1d9 doc/source/developing/testing.rst
--- a/doc/source/developing/testing.rst
+++ b/doc/source/developing/testing.rst
@@ -19,7 +19,7 @@
checking in any code that breaks existing functionality. To further this goal,
an automatic buildbot runs the test suite after each code commit to confirm
that yt hasn't broken recently. To supplement this effort, we also maintain a
-`continuous integration server <http://link.bitbucket.org/wf/click?upn=RhcBcfh3o3OexLylRqBSSeffThKl5wGEWpBk35gYMt-2F1i3i7Cdjz-2BA93imiBaXY4_ll4ctv0L-2ByeRZFC1LslHcg6aJmnQ70VruLbmeLQr27BJwZjKog4Ujn-2Bi-2B9HmWCKg-2F8dA1r-2BeaKRlnKUM6YpD04nLVVwcjvW46oUYHqk6QJq5KR72TRe3DVNDsdnPxRvyDIEnGW-2F0Ab5-2Bd9M6hTXj0zRDcX-2F0xj-2Bd0LvzEeRm92LhRFk39JR0tUo40UTL2k0ld8uQ1iOy0eX70xDFN7JwyMn1obAj-2FbfLo56R9LrBYH4-3D>`_ that runs the
+`continuous integration server <http://link.bitbucket.org/wf/click?upn=8USRlNyft-2BCzk2l4Ywl6gEifL6SlWCUOo2iin058pvnChi95ByN0SZ4ANMKlVDue_ll4ctv0L-2ByeRZFC1LslHcg6aJmnQ70VruLbmeLQr27BJwZjKog4Ujn-2Bi-2B9HmWCKg-2F8dA1r-2BeaKRlnKUM6YpD0-2FHxm9rTPoWbYXVJNe4urPORt4qLpIfc3s9y8gjwb5O0D8U5q9c1LvefCJHAvRRtnfy6VR5WIAESfQmlPwpzqIiK-2FBfO4r3eF4wZ-2Fy6PtXMvG3O7yQplwfkoib2rMwOe3-2FHaya-2ByXz-2F-2FRS6eZ0d2qXA-3D>`_ that runs the
tests with each commit to the yt version control repository.
.. _unit_testing:
@@ -476,3 +476,82 @@
test is more useful if you are finding yourself writing a ton of boilerplate
code to get your image comparison test working. The ``GenericImageTest`` is
more useful if you only need to do a one-off image comparison test.
+
+Enabling Answer Tests on Jenkins
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Before any code is added to or modified in the yt codebase, each incoming
+changeset is run against all available unit and answer tests on our `continuous
+integration server <http://link.bitbucket.org/wf/click?upn=RhcBcfh3o3OexLylRqBSSeffThKl5wGEWpBk35gYMt-2F1i3i7Cdjz-2BA93imiBaXY4_ll4ctv0L-2ByeRZFC1LslHcg6aJmnQ70VruLbmeLQr27BJwZjKog4Ujn-2Bi-2B9HmWCKg-2F8dA1r-2BeaKRlnKUM6YpD09ktgv06kEb8hQHjZUPxpBNYLSsVoo8ATlppaqj5p0flHCZQjkOGvsUNyEGkTRuiAVNyNa3XzrzfEUQbVPhpaA9YU-2BRapWNYPfKa7b55SfkuD0zoMHIJa1JPIDB-2FbmsYlRWWtb5sVzMDGbjkw7HclJ4-3D>`_. While unit tests are
+autodiscovered by `nose <http://link.bitbucket.org/wf/click?upn=-2F-2B5EjSCIdHksviJD1NH3VT33nG2WRIz72zZynHbnfBV84blL2pz26ULzJCzH4d2r_ll4ctv0L-2ByeRZFC1LslHcg6aJmnQ70VruLbmeLQr27BJwZjKog4Ujn-2Bi-2B9HmWCKg-2F8dA1r-2BeaKRlnKUM6YpD032PJtUxo1ak8Ei9Eei8a40mNWpsV5fQaXjJijWRY-2BhgxX8LH42TJlR3JzcdFwdzdEvwzQ-2BWUIQ58LLF1xWkfHP4ZEWKuRgY3hozFIA05W7CBvj3qpu4Lxlz-2FK0-2FzmP87GoW9tfEtoEEEQNDrYbqkSI-3D>`_ itself,
+answer tests require definition of which set of tests constitute to a given
+answer. Configuration for the integration server is stored in
+*tests/tests_2.7.yaml* in the main yt repository:
+
+.. code-block:: yaml
+
+ answer_tests:
+ local_artio_270:
+ - yt/frontends/artio/tests/test_outputs.py
+ # ...
+ other_tests:
+ unittests:
+ - '-v'
+ - '-s'
+
+Each element under *answer_tests* defines answer name (*local_artio_270* in above
+snippet) and specifies a list of files/classes/methods that will be validated
+(*yt/frontends/artio/tests/test_outputs.py* in above snippet). On the testing
+server it is translated to:
+
+.. code-block:: bash
+
+ $ nosetests --with-answer-testing --local --local-dir ... --answer-big-data \
+ --answer-name=local_artio_270 \
+ yt/frontends/artio/tests/test_outputs.py
+
+If the answer doesn't exist on the server yet, ``nosetests`` is run twice and
+during first pass ``--answer-store`` is added to the commandline.
+
+Updating Answers
+~~~~~~~~~~~~~~~~
+
+In order to regenerate answers for a particular set of tests it is sufficient to
+change the answer name in *tests/tests_2.7.yaml* e.g.:
+
+.. code-block:: diff
+
+ --- a/tests/tests_2.7.yaml
+ +++ b/tests/tests_2.7.yaml
+ @@ -25,7 +25,7 @@
+ - yt/analysis_modules/halo_finding/tests/test_rockstar.py
+ - yt/frontends/owls_subfind/tests/test_outputs.py
+
+ - local_owls_270:
+ + local_owls_271:
+ - yt/frontends/owls/tests/test_outputs.py
+
+ local_pw_270:
+
+would regenerate answers for OWLS frontend.
+
+Adding New Answer Tests
+~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to add a new set of answer tests, it is sufficient to extend the
+*answer_tests* list in *tests/tests_2.7.yaml* e.g.:
+
+.. code-block:: diff
+
+ --- a/tests/tests_2.7.yaml
+ +++ b/tests/tests_2.7.yaml
+ @@ -60,6 +60,10 @@
+ - yt/analysis_modules/absorption_spectrum/tests/test_absorption_spectrum.py:test_absorption_spectrum_non_cosmo
+ - yt/analysis_modules/absorption_spectrum/tests/test_absorption_spectrum.py:test_absorption_spectrum_cosmo
+
+ + local_gdf_270:
+ + - yt/frontends/gdf/tests/test_outputs.py
+ +
+ +
+ other_tests:
+ unittests:
+
Repository URL: http://link.bitbucket.org/wf/click?upn=8USRlNyft-2BCzk2l4Ywl6gDx2lD2xxoS9E7MwXb2SMR-2BI0v8SbQq-2B8-2FZaaHaJT85r_ll4ctv0L-2ByeRZFC1LslHcg6aJmnQ70VruLbmeLQr27BJwZjKog4Ujn-2Bi-2B9HmWCKg-2F8dA1r-2BeaKRlnKUM6YpD0yFEkrcmkaA09T0t8C6ekQaVLmdm7ccatgSZaeQbVYEOu2PWvAmv5SlR6YP115LQ-2BOwiOfCa0A06baRaDlCqwnvXPfhxIyoRV2ZbY-2FkqkVTkME7vHDJIicaBd9YM0mvTq1-2FDxBRTjRbeEFO9vITtnKA-3D
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-svn-spacepope.org/attachments/20160323/bd105689/attachment-0001.htm>
More information about the yt-svn
mailing list