[yt-svn] commit/yt: 3 new changesets
commits-noreply at bitbucket.org
commits-noreply at bitbucket.org
Wed Aug 3 11:09:43 PDT 2016
3 new commits in yt:
https://bitbucket.org/yt_analysis/yt/commits/2ae7e0f42c8e/
Changeset: 2ae7e0f42c8e
Branch: yt
User: jwise77
Date: 2014-12-27 20:56:15+00:00
Summary: Adding rockstar halo properties from rockstar-galaxies fork.
Affected #: 1 file
diff -r b3c23ff06040ef967b7c9ac041d5ad3dc4edb79a -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa yt/frontends/rockstar/definitions.py
--- a/yt/frontends/rockstar/definitions.py
+++ b/yt/frontends/rockstar/definitions.py
@@ -38,7 +38,7 @@
# Note the final field here, which is a field for min/max format revision in
# which the field appears.
-KNOWN_REVISIONS=[0, 1]
+KNOWN_REVISIONS=[0, 1, 2]
halo_dt = [
('particle_identifier', np.int64),
@@ -101,6 +101,12 @@
('min_pos_err', np.float32),
('min_vel_err', np.float32),
('min_bulkvel_err', np.float32),
+ ('type', np.int32, (2, 100)),
+ ('sm', np.float32, (2, 100)),
+ ('gas', np.float32, (2, 100)),
+ ('bh', np.float32, (2, 100)),
+ ('peak_density', np.float32, (2, 100)),
+ ('av_density', np.float32, (2, 100)),
]
halo_dts = {}
https://bitbucket.org/yt_analysis/yt/commits/a0f0edfe2bdf/
Changeset: a0f0edfe2bdf
Branch: yt
User: jwise77
Date: 2016-06-14 13:00:38+00:00
Summary: Merging
Affected #: 71 files
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 .hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -45,9 +45,11 @@
yt/utilities/lib/mesh_intersection.cpp
yt/utilities/lib/mesh_samplers.cpp
yt/utilities/lib/mesh_traversal.cpp
+yt/utilities/lib/mesh_triangulation.c
yt/utilities/lib/mesh_utilities.c
yt/utilities/lib/misc_utilities.c
yt/utilities/lib/particle_mesh_operations.c
+yt/utilities/lib/primitives.c
yt/utilities/lib/origami.c
yt/utilities/lib/particle_mesh_operations.c
yt/utilities/lib/pixelization_routines.c
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 CONTRIBUTING.rst
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -651,7 +651,7 @@
.. _multiple-PRs:
Working with Multiple BitBucket Pull Requests
-+++++++++++++++++++++++++++++++++++++++++++++
+---------------------------------------------
Once you become active developing for yt, you may be working on
various aspects of the code or bugfixes at the same time. Currently,
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 appveyor.yml
--- /dev/null
+++ b/appveyor.yml
@@ -0,0 +1,38 @@
+# AppVeyor.com is a Continuous Integration service to build and run tests under
+# Windows
+
+environment:
+
+ global:
+ PYTHON: "C:\\Miniconda-x64"
+
+ matrix:
+
+ - PYTHON_VERSION: "2.7"
+
+ - PYTHON_VERSION: "3.5"
+
+
+platform:
+ -x64
+
+install:
+ - "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%"
+
+ # Install the build and runtime dependencies of the project.
+ # Create a conda environment
+ - "conda create -q --yes -n test python=%PYTHON_VERSION%"
+ - "activate test"
+
+ # Check that we have the expected version of Python
+ - "python --version"
+
+ # Install specified version of numpy and dependencies
+ - "conda install -q --yes numpy nose setuptools ipython Cython sympy h5py matplotlib"
+ - "python setup.py develop"
+
+# Not a .NET project
+build: false
+
+test_script:
+ - "nosetests -e test_all_fields ."
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/analyzing/fields.rst
--- a/doc/source/analyzing/fields.rst
+++ b/doc/source/analyzing/fields.rst
@@ -14,14 +14,126 @@
out of favor, as these discrete fields can be any type of sparsely populated
data.
+What are fields?
+----------------
+
+Fields in yt are denoted by a two-element tuple, of the form ``(field_type,
+field_name)``. The first element, the "field type" is a category for a
+field. Possible field types used in yt include *gas* (for fluid mesh fields
+defined on a mesh) or *io* (for fields defined at particle locations). Field
+types can also correspond to distinct particle of fluid types in a single
+simulation. For example, a plasma physics simulation using the Particle in Cell
+method might have particle types corresponding to *electrons* and *ions*. See
+:ref:`known-field-types` below for more info about field types in yt.
+
+The second element of field tuples, the "field name", denotes the specific field
+to select, given the field type. Possible field names include *density*,
+*velocity_x* or *pressure* --- these three fields are examples of field names
+that might be used for a fluid defined on a mesh. Examples of particle fields
+include *particle_mass*, *particle_position*, or *particle_velocity_x*. In
+general, particle field names are prefixed by "particle\_", which makes it easy
+to distinguish between a particle field or a mesh field when no field type is
+provided.
+
+What fields are available?
+--------------------------
+
+We provide a full list of fields that yt recognizes by default at
+:ref:`field-list`. If you want to create additional custom derived fields,
+see :ref:`creating-derived-fields`.
+
+Every dataset has an attribute, ``ds.fields``. This attribute possesses
+attributes itself, each of which is a "field type," and each field type has as
+its attributes the fields themselves. When one of these is printed, it returns
+information about the field and things like units and so on. You can use this
+for tab-completing as well as easier access to information.
+
+As an example, you might browse the available fields like so:
+
+.. code-block:: python
+
+ print(dir(ds.fields))
+ print(dir(ds.fields.gas))
+ print(ds.fields.gas.density)
+
+On an Enzo dataset, the result from the final command would look something like
+this:::
+
+ Alias Field for "('enzo', 'Density')" (gas, density): (units: g/cm**3)
+
+You can use this to easily explore available fields, particularly through
+tab-completion in Jupyter/IPython.
+
+For a more programmatic method of accessing fields, you can utilize the
+``ds.field_list``, ``ds.derived_field_list`` and some accessor methods to gain
+information about fields. The full list of fields available for a dataset can
+be found as the attribute ``field_list`` for native, on-disk fields and
+``derived_field_list`` for derived fields (``derived_field_list`` is a superset
+of ``field_list``). You can view these lists by examining a dataset like this:
+
+.. code-block:: python
+
+ ds = yt.load("my_data")
+ print(ds.field_list)
+ print(ds.derived_field_list)
+
+By using the ``field_info()`` class, one can access information about a given
+field, like its default units or the source code for it.
+
+.. code-block:: python
+
+ ds = yt.load("my_data")
+ ds.index
+ print(ds.field_info["gas", "pressure"].get_units())
+ print(ds.field_info["gas", "pressure"].get_source())
+
+Using fields to access data
+---------------------------
+
+The primary *use* of fields in yt is to access data from a dataset. For example,
+if I want to use a data object (see :ref:`Data-objects` for more detail about
+data objects) to access the ``('gas', 'density')`` field, one can do any of the
+following:
+
+.. code-block:: python
+
+ ad = ds.all_data()
+
+ # just a field name
+ density = ad['density']
+
+ # field tuple with no parentheses
+ density = ad['gas', 'density']
+
+ # full field tuple
+ density = ad[('gas', 'density')]
+
+ # through the ds.fields object
+ density = ad[ds.fields.gas.density]
+
+The first data access example is the simplest. In that example, the field type
+is inferred from the name of the field. The next two examples use the field type
+explicitly, this might be necessary if there is more than one field type with a
+"density" field defined in the same dataset. The third example is slightly more
+verbose but is syntactically identical to the second example due to the way
+indexing works in the Python language.
+
+The final example uses the ``ds.fields`` object described above. This way of
+accessing fields lends itself to interactive use, especially if you make heavy
+use of IPython's tab completion features. Any of these ways of denoting the
+``('gas', 'density')`` field can be used when supplying a field name to a yt
+data object, analysis routines, or plotting and visualization function.
+
+Accessing Fields without a Field Type
+-------------------------------------
+
In previous versions of yt, there was a single mechanism of accessing fields on
-a data container -- by their name, which was mandated to be a single string,
-and which often varied between different code frontends. yt 3.0 allows
-for datasets containing multiple different types of fluid fields, mesh fields,
-particles (with overlapping or disjoint lists of fields). To enable accessing
-these fields in a meaningful, simple way, the mechanism for accessing them has
-changed to take an optional *field type* in addition to the *field name* of
-the form ('*field type*', '*field name*').
+a data container -- by their name, which was mandated to be a single string, and
+which often varied between different code frontends. yt 3.0 allows for datasets
+containing multiple different types of fluid fields, mesh fields, particles
+(with overlapping or disjoint lists of fields). However, to preserve backward
+compatibility and make interactive use simpler, yt will still accept field names
+given as a string and will try to infer the field type given a field name.
As an example, we may be in a situation where have multiple types of particles
which possess the ``particle_position`` field. In the case where a data
@@ -30,9 +142,9 @@
.. code-block:: python
- print(ad["humans", "particle_position"])
- print(ad["dogs", "particle_position"])
- print(ad["dinosaurs", "particle_position"])
+ print(ad["dark_matter", "particle_position"])
+ print(ad["stars", "particle_position"])
+ print(ad["black_holes", "particle_position"])
Each of these three fields may have different sizes. In order to enable
falling back on asking only for a field by the name, yt will use the most
@@ -45,7 +157,8 @@
print(ad["particle_velocity"])
-it would select ``dinosaurs`` as the field type.
+it would select ``black_holes`` as the field type, since the last field accessed
+used that field type.
The same operations work for fluid and mesh fields. As an example, in some
cosmology simulations, we may want to examine the mass of particles in a region
@@ -189,58 +302,6 @@
* Species fields, such as for chemistry species (yt can recognize the entire
periodic table in field names and construct ionization fields as need be)
-What fields are available?
---------------------------
-
-We provide a full list of fields that yt recognizes by default at
-:ref:`field-list`. If you want to create additional custom derived fields,
-see :ref:`creating-derived-fields`.
-
-Every dataset has an attribute, ``ds.fields``. This attribute possesses
-attributes itself, each of which is a "field type," and each field type has as
-its attributes the fields themselves. When one of these is printed, it returns
-information about the field and things like units and so on. You can use this
-for tab-completing as well as easier access to information.
-
-As an example, you might browse the available fields like so:
-
-.. code-block:: python
-
- print(dir(ds.fields))
- print(dir(ds.fields.gas))
- print(ds.fields.gas.density)
-
-On an Enzo dataset, the result from the final command would look something like
-this:::
-
- Alias Field for "('enzo', 'Density')" (gas, density): (units: g/cm**3)
-
-You can use this to easily explore available fields, particularly through
-tab-completion in Jupyter/IPython.
-
-For a more programmatic method of accessing fields, you can utilize the
-``ds.field_list``, ``ds.derived_field_list`` and some accessor methods to gain
-information about fields. The full list of fields available for a dataset can
-be found as the attribute ``field_list`` for native, on-disk fields and
-``derived_field_list`` for derived fields (``derived_field_list`` is a superset
-of ``field_list``). You can view these lists by examining a dataset like this:
-
-.. code-block:: python
-
- ds = yt.load("my_data")
- print(ds.field_list)
- print(ds.derived_field_list)
-
-By using the ``field_info()`` class, one can access information about a given
-field, like its default units or the source code for it.
-
-.. code-block:: python
-
- ds = yt.load("my_data")
- ds.index
- print(ds.field_info["gas", "pressure"].get_units())
- print(ds.field_info["gas", "pressure"].get_source())
-
.. _bfields:
Magnetic Fields
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/cookbook/various_lens.py
--- a/doc/source/cookbook/various_lens.py
+++ b/doc/source/cookbook/various_lens.py
@@ -85,6 +85,7 @@
cam = sc.add_camera(ds, lens_type='spherical')
# Set the size ratio of the final projection to be 2:1, since spherical lens
# will generate the final image with length of 2*pi and height of pi.
+# Recommended resolution for YouTube 360-degree videos is [3840, 2160]
cam.resolution = [500, 250]
# Standing at (x=0.4, y=0.5, z=0.5), we look in all the radial directions
# from this point in spherical coordinate.
@@ -99,9 +100,11 @@
# Stereo-spherical lens
cam = sc.add_camera(ds, lens_type='stereo-spherical')
-# Set the size ratio of the final projection to be 4:1, since spherical-perspective lens
-# will generate the final image with both left-eye and right-eye ones jointed together.
-cam.resolution = [1000, 250]
+# Set the size ratio of the final projection to be 1:1, since spherical-perspective lens
+# will generate the final image with both left-eye and right-eye ones jointed together,
+# with left-eye image on top and right-eye image on bottom.
+# Recommended resolution for YouTube virtual reality videos is [3840, 2160]
+cam.resolution = [500, 500]
cam.position = ds.arr([0.4, 0.5, 0.5], 'code_length')
cam.switch_orientation(normal_vector=normal_vector,
north_vector=north_vector)
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/developing/testing.rst
--- a/doc/source/developing/testing.rst
+++ b/doc/source/developing/testing.rst
@@ -95,17 +95,17 @@
To create new unit tests:
#. Create a new ``tests/`` directory next to the file containing the
- functionality you want to test. Be sure to add this new directory as a
- subpackage in the setup.py script located in the directory you're adding a
- new ``tests/`` folder to. This ensures that the tests will be deployed in
- yt source and binary distributions.
+ functionality you want to test and add an empty ``__init__.py`` file to
+ it.
#. Inside that directory, create a new python file prefixed with ``test_`` and
including the name of the functionality.
#. Inside that file, create one or more routines prefixed with ``test_`` that
- accept no arguments. These should ``yield`` a tuple of the form
- ``function``, ``argument_one``, ``argument_two``, etc. For example
- ``yield assert_equal, 1.0, 1.0`` would be captured by nose as a test that
- asserts that 1.0 is equal to 1.0.
+ accept no arguments. The test function should do some work that tests some
+ functionality and should also verify that the results are correct using
+ assert statements or functions.
+# Tests can ``yield`` a tuple of the form ``function``, ``argument_one``,
+ ``argument_two``, etc. For example ``yield assert_equal, 1.0, 1.0`` would be
+ captured by nose as a test that asserts that 1.0 is equal to 1.0.
#. Use ``fake_random_ds`` to test on datasets, and be sure to test for
several combinations of ``nproc``, so that domain decomposition can be
tested as well.
@@ -470,9 +470,9 @@
test.prefix = "my_unique_name"
# this ensures a nice test name in nose's output
- test_my_ds.__description__ = test.description
+ test_my_ds.__name__ = test.description
- yield test_my_ds
+ yield test
Another good example of an image comparison test is the
``PlotWindowAttributeTest`` defined in the answer testing framework and used in
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/examining/loading_data.rst
--- a/doc/source/examining/loading_data.rst
+++ b/doc/source/examining/loading_data.rst
@@ -1330,8 +1330,6 @@
grid structure as spatial fields as described in :ref:`loading-gadget-data`.
To load Gizmo datasets using the standard HDF5 output format::
-.. code-block:: python
-
import yt
ds = yt.load("snapshot_600.hdf5")
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/examining/low_level_inspection.rst
--- a/doc/source/examining/low_level_inspection.rst
+++ b/doc/source/examining/low_level_inspection.rst
@@ -232,3 +232,28 @@
whatever interface they wish for displaying and saving their image data.
You can use the :class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer`
to accomplish this as described in :ref:`fixed-resolution-buffers`.
+
+High-level Information about Particles
+--------------------------------------
+
+There are a number of high-level helpers attached to ``Dataset`` objects to find
+out information about the particles in an output file. First, one can check if
+there are any particles in a dataset at all by examining
+``ds.particles_exist``. This will be ``True`` for datasets the include particles
+and ``False`` otherwise.
+
+One can also see which particle types are available in a dataset. Particle types
+that are available in the dataset's on-disk output are known as "raw" particle
+types, and they will appear in ``ds.particle_types_raw``. Particle types that
+are dynamically defined via a particle filter of a particle union will also
+appear in the ``ds.particle_types`` list. If the simulation only has one
+particle type on-disk, its name will by ``'io'``. If there is more than one
+particle type, the names of the particle types will be inferred from the output
+file. For example, Gadget HDF5 files have particle type names like ``PartType0``
+and ``PartType1``, while Enzo data, which usually only has one particle type,
+will only have a particle named ``io``.
+
+Finally, one can see the number of each particle type by inspecting
+``ds.particle_type_counts``. This will be a dictionary mappying the names of
+particle types in ``ds.particle_types_raw`` to the number of each particle type
+in a simulation output.
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/quickstart/2)_Data_Inspection.ipynb
--- a/doc/source/quickstart/2)_Data_Inspection.ipynb
+++ b/doc/source/quickstart/2)_Data_Inspection.ipynb
@@ -154,6 +154,35 @@
"cell_type": "markdown",
"metadata": {},
"source": [
+ "Finally, we can get basic information about the particle types and number of particles in a simulation:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "print (ds.particle_types)\n",
+ "print (ds.particle_types_raw)\n",
+ "print (ds.particle_type_counts)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For this dataset, we see that there are two particle types defined, (`io` and `all`), but that only one of these particle types in in `ds.particle_types_raw`. The `ds.particle_types` list contains *all* particle types in the simulation, including ones that are dynamically defined like particle unions. The `ds.particle_types_raw` list includes only particle types that are in the output file we loaded the dataset from.\n",
+ "\n",
+ "We can also see that there are a bit more than 1.1 million particles in this simulation. Only particle types in `ds.particle_types_raw` will appear in the `ds.particle_type_counts` dictionary."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
"# Mesh Structure\n",
"\n",
"If you're using a simulation type that has grids (for instance, here we're using an Enzo simulation) you can examine the structure of the mesh. For the most part, you probably won't have to use this unless you're debugging a simulation or examining in detail what is going on."
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/visualizing/callbacks.rst
--- a/doc/source/visualizing/callbacks.rst
+++ b/doc/source/visualizing/callbacks.rst
@@ -692,6 +692,28 @@
s.annotate_triangle_facets(points, plot_args={"colors": 'black'})
s.save()
+.. _annotate-mesh-lines:
+
+Annotate Mesh Lines Callback
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. function:: annotate_mesh_lines(plot_args=None)
+
+ (This is a proxy for
+ :class:`~yt.visualization.plot_modifications.MeshLinesCallback`.)
+
+ This draws the mesh line boundaries over a plot using a Matplotlib
+ line collection. This callback is only useful for unstructured or
+ semi-structured mesh datasets.
+
+.. python-script::
+
+ import yt
+ ds = yt.load('MOOSE_sample_data/out.e')
+ sl = yt.SlicePlot(ds, 2, ('connect1', 'nodal_aux'))
+ sl.annotate_mesh_lines(plot_args={'color':'black'})
+ sl.save()
+
.. _annotate-ray:
Overplot the Path of a Ray
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/visualizing/plots.rst
--- a/doc/source/visualizing/plots.rst
+++ b/doc/source/visualizing/plots.rst
@@ -406,16 +406,14 @@
import yt
ds = yt.load('MOOSE_sample_data/out.e-s010')
sl = yt.SlicePlot(ds, 'z', ('connect1', 'diffused'))
- sl.annotate_mesh_lines(thresh=0.1)
+ sl.annotate_mesh_lines(plot_args={'color':'black'})
sl.zoom(0.75)
sl.save()
-This annotation is performed by marking the pixels where the mapped coordinate is close
-to the element boundary. What counts as 'close' (in the mapped coordinate system) is
-determined by the ``thresh`` parameter, which can be varied to make the lines thicker or
-thinner.
+The ``plot_args`` parameter is a dictionary of keyword arguments that will be passed
+to matplotlib. It can be used to control the mesh line color, thickness, etc...
-The above example all involve 8-node hexahedral mesh elements. Here is another example from
+The above examples all involve 8-node hexahedral mesh elements. Here is another example from
a dataset that uses 6-node wedge elements:
.. python-script::
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 doc/source/visualizing/volume_rendering.rst
--- a/doc/source/visualizing/volume_rendering.rst
+++ b/doc/source/visualizing/volume_rendering.rst
@@ -31,8 +31,8 @@
as grid or continent lines, and then to render a production-quality
visualization. By changing the "lens" used, a single camera path can output
images suitable for planetarium domes, immersive and head tracking systems
-(such as the Oculus Rift or recent "spherical" movie viewers such as the
-mobile YouTube app), as well as standard screens.
+(such as the Oculus Rift or recent 360-degree/virtual reality movie viewers
+such as the mobile YouTube app), as well as standard screens.
.. image:: _images/scene_diagram.svg
:width: 50%
@@ -327,13 +327,19 @@
The :class:`~yt.visualization.volume_rendering.lens.SphericalLens` produces
a cylindrical-spherical projection. Movies rendered in this way can be
-displayed in head-tracking devices (e.g. Oculus Rift) or in YouTube 360 view
-(for more information see `the YouTube help
-<https://support.google.com/youtube/answer/6178631?hl=en>`, but it's a
-simple matter of running a script on an encoded movie file.)
+displayed as YouTube 360-degree videos (for more information see
+`the YouTube help: Upload 360-degree videos
+<https://support.google.com/youtube/answer/6178631?hl=en>`_).
:class:`~yt.visualization.volume_rendering.lens.StereoSphericalLens`
is identical to :class:`~yt.visualization.volume_rendering.lens.SphericalLens`
-but it produces two images from nearby camera positions for use in 3D viewing.
+but it produces two images from nearby camera positions for virtual reality
+movies, which can be displayed in head-tracking devices (e.g. Oculus Rift)
+or in mobile YouTube app with Google Cardboard (for more information
+see `the YouTube help: Upload virtual reality videos
+<https://support.google.com/youtube/answer/6316263?hl=en>`_).
+`This virtual reality video
+<https://youtu.be/ZYWY53X7UQE>`_ on YouTube is an example produced with
+:class:`~yt.visualization.volume_rendering.lens.StereoSphericalLens`.
.. _annotated-vr-example:
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 setup.py
--- a/setup.py
+++ b/setup.py
@@ -83,127 +83,82 @@
Extension("yt.geometry.grid_visitors",
["yt/geometry/grid_visitors.pyx"],
include_dirs=["yt/utilities/lib"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/grid_visitors.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.grid_container",
["yt/geometry/grid_container.pyx"],
include_dirs=["yt/utilities/lib/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/grid_container.pxd",
- "yt/geometry/grid_visitors.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.oct_container",
["yt/geometry/oct_container.pyx",
"yt/utilities/lib/tsearch.c"],
include_dirs=["yt/utilities/lib"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/selection_routines.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.oct_visitors",
["yt/geometry/oct_visitors.pyx"],
include_dirs=["yt/utilities/lib/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/selection_routines.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.particle_oct_container",
["yt/geometry/particle_oct_container.pyx"],
include_dirs=["yt/utilities/lib/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/selection_routines.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.selection_routines",
["yt/geometry/selection_routines.pyx"],
include_dirs=["yt/utilities/lib/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/utilities/lib/grid_traversal.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/oct_visitors.pxd",
- "yt/geometry/grid_container.pxd",
- "yt/geometry/grid_visitors.pxd",
- "yt/geometry/selection_routines.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.particle_deposit",
["yt/geometry/particle_deposit.pyx"],
include_dirs=["yt/utilities/lib/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/selection_routines.pxd",
- "yt/geometry/particle_deposit.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.particle_smooth",
["yt/geometry/particle_smooth.pyx"],
include_dirs=["yt/utilities/lib/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/selection_routines.pxd",
- "yt/geometry/particle_deposit.pxd",
- "yt/geometry/particle_smooth.pxd"]),
+ libraries=std_libs),
Extension("yt.geometry.fake_octree",
["yt/geometry/fake_octree.pyx"],
include_dirs=["yt/utilities/lib/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/selection_routines.pxd"]),
+ libraries=std_libs),
Extension("yt.utilities.spatial.ckdtree",
["yt/utilities/spatial/ckdtree.pyx"],
include_dirs=["yt/utilities/lib/"],
libraries=std_libs),
Extension("yt.utilities.lib.bitarray",
["yt/utilities/lib/bitarray.pyx"],
- libraries=std_libs, depends=["yt/utilities/lib/bitarray.pxd"]),
- Extension("yt.utilities.lib.primitives",
- ["yt/utilities/lib/primitives.pyx"],
- libraries=std_libs,
- depends=["yt/utilities/lib/primitives.pxd",
- "yt/utilities/lib/vec3_ops.pxd",
- "yt/utilities/lib/bounding_volume_hierarchy.pxd"]),
+ libraries=std_libs),
Extension("yt.utilities.lib.bounding_volume_hierarchy",
["yt/utilities/lib/bounding_volume_hierarchy.pyx"],
include_dirs=["yt/utilities/lib/"],
extra_compile_args=omp_args,
extra_link_args=omp_args,
libraries=std_libs,
- depends=["yt/utilities/lib/element_mappings.pxd",
- "yt/utilities/lib/vec3_ops.pxd",
- "yt/utilities/lib/primitives.pxd"]),
+ depends=["yt/utilities/lib/mesh_triangulation.h"]),
Extension("yt.utilities.lib.contour_finding",
["yt/utilities/lib/contour_finding.pyx"],
include_dirs=["yt/utilities/lib/",
"yt/geometry/"],
- libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/utilities/lib/amr_kdtools.pxd",
- "yt/utilities/lib/grid_traversal.pxd",
- "yt/utilities/lib/contour_finding.pxd",
- "yt/geometry/oct_container.pxd"]),
+ libraries=std_libs),
Extension("yt.utilities.lib.geometry_utils",
["yt/utilities/lib/geometry_utils.pyx"],
extra_compile_args=omp_args,
extra_link_args=omp_args,
- libraries=std_libs, depends=["yt/utilities/lib/fp_utils.pxd"]),
+ libraries=std_libs),
Extension("yt.utilities.lib.marching_cubes",
["yt/utilities/lib/marching_cubes.pyx",
"yt/utilities/lib/fixed_interpolator.c"],
include_dirs=["yt/utilities/lib/"],
libraries=std_libs,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/utilities/lib/fixed_interpolator.pxd",
- "yt/utilities/lib/fixed_interpolator.h",
- ]),
+ depends=["yt/utilities/lib/fixed_interpolator.h"]),
+ Extension("yt.utilities.lib.mesh_triangulation",
+ ["yt/utilities/lib/mesh_triangulation.pyx"],
+ depends=["yt/utilities/lib/mesh_triangulation.h"]),
Extension("yt.utilities.lib.pixelization_routines",
["yt/utilities/lib/pixelization_routines.pyx",
"yt/utilities/lib/pixelization_constants.c"],
include_dirs=["yt/utilities/lib/"],
- language="c++",
- libraries=std_libs, depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/utilities/lib/pixelization_constants.h",
- "yt/utilities/lib/element_mappings.pxd"]),
+ libraries=std_libs,
+ depends=["yt/utilities/lib/pixelization_constants.h"]),
+ Extension("yt.utilities.lib.primitives",
+ ["yt/utilities/lib/primitives.pyx"],
+ libraries=std_libs),
Extension("yt.utilities.lib.origami",
["yt/utilities/lib/origami.pyx",
"yt/utilities/lib/origami_tags.c"],
@@ -217,15 +172,11 @@
libraries=std_libs,
extra_compile_args=omp_args,
extra_link_args=omp_args,
- depends=["yt/utilities/lib/fp_utils.pxd",
- "yt/utilities/lib/kdtree.h",
- "yt/utilities/lib/fixed_interpolator.h",
- "yt/utilities/lib/fixed_interpolator.pxd",
- "yt/utilities/lib/field_interpolation_tables.pxd",
- "yt/utilities/lib/vec3_ops.pxd"]),
+ depends=["yt/utilities/lib/kdtree.h",
+ "yt/utilities/lib/fixed_interpolator.h"]),
Extension("yt.utilities.lib.element_mappings",
["yt/utilities/lib/element_mappings.pyx"],
- libraries=std_libs, depends=["yt/utilities/lib/element_mappings.pxd"]),
+ libraries=std_libs),
Extension("yt.utilities.lib.alt_ray_tracers",
["yt/utilities/lib/alt_ray_tracers.pyx"],
libraries=std_libs),
@@ -238,10 +189,14 @@
"amr_kdtools"
]
for ext_name in lib_exts:
+ lib_deps = []
+ if ext_name == 'misc_utilities':
+ lib_deps.append('yt/geometry/selection_routines.pxd')
cython_extensions.append(
Extension("yt.utilities.lib.{}".format(ext_name),
["yt/utilities/lib/{}.pyx".format(ext_name)],
- libraries=std_libs, depends=["yt/utilities/lib/fp_utils.pxd"]))
+ libraries=std_libs,
+ depends=lib_deps))
lib_exts = ["write_array", "ragged_arrays", "line_integral_convolution"]
for ext_name in lib_exts:
@@ -262,11 +217,7 @@
include_dirs=["yt/frontends/artio/artio_headers/",
"yt/geometry/",
"yt/utilities/lib/"],
- depends=glob.glob("yt/frontends/artio/artio_headers/*.c") +
- ["yt/utilities/lib/fp_utils.pxd",
- "yt/geometry/oct_container.pxd",
- "yt/geometry/selection_routines.pxd",
- "yt/geometry/particle_deposit.pxd"]),
+ depends=glob.glob("yt/frontends/artio/artio_headers/*.c")),
Extension("yt.utilities.spatial._distance_wrap",
glob.glob("yt/utilities/spatial/src/*.c")),
Extension("yt.visualization._MPL",
@@ -282,30 +233,13 @@
embree_extensions = [
Extension("yt.utilities.lib.mesh_construction",
["yt/utilities/lib/mesh_construction.pyx"],
- depends=["yt/utilities/lib/mesh_construction.pxd",
- "yt/utilities/lib/mesh_intersection.pxd",
- "yt/utlilites/lib/mesh_samplers.pxd",
- "yt/utlilites/lib/mesh_traversal.pxd"]),
+ depends=["yt/utilities/lib/mesh_triangulation.h"]),
Extension("yt.utilities.lib.mesh_traversal",
- ["yt/utilities/lib/mesh_traversal.pyx"],
- depends=["yt/utilities/lib/mesh_traversal.pxd",
- "yt/utilities/lib/grid_traversal.pxd",
- "yt/utilities/lib/bounding_volume_hierarchy.pxd"]),
+ ["yt/utilities/lib/mesh_traversal.pyx"]),
Extension("yt.utilities.lib.mesh_samplers",
- ["yt/utilities/lib/mesh_samplers.pyx"],
- depends=["yt/utilities/lib/mesh_samplers.pxd",
- "yt/utilities/lib/element_mappings.pxd",
- "yt/utilities/lib/mesh_construction.pxd",
- "yt/utilities/lib/bounding_volume_hierarchy.pxd",
- "yt/utilities/lib/primitives.pxd"]),
+ ["yt/utilities/lib/mesh_samplers.pyx"]),
Extension("yt.utilities.lib.mesh_intersection",
- ["yt/utilities/lib/mesh_intersection.pyx"],
- depends=["yt/utilities/lib/mesh_intersection.pxd",
- "yt/utilities/lib/mesh_construction.pxd",
- "yt/utilities/lib/bounding_volume_hierarchy.pxd",
- "yt/utilities/lib/mesh_samplers.pxd",
- "yt/utilities/lib/primitives.pxd",
- "yt/utilities/lib/vec3_ops.pxd"]),
+ ["yt/utilities/lib/mesh_intersection.pyx"]),
]
embree_prefix = os.path.abspath(read_embree_location())
@@ -381,9 +315,12 @@
_build_py.run(self)
class build_ext(_build_ext):
- # subclass setuptools extension builder to avoid importing numpy
+ # subclass setuptools extension builder to avoid importing cython and numpy
# at top level in setup.py. See http://stackoverflow.com/a/21621689/1382869
def finalize_options(self):
+ from Cython.Build import cythonize
+ self.distribution.ext_modules[:] = cythonize(
+ self.distribution.ext_modules)
_build_ext.finalize_options(self)
# Prevent numpy from thinking it is still in its setup process
# see http://stackoverflow.com/a/21621493/1382869
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/data_objects/data_containers.py
--- a/yt/data_objects/data_containers.py
+++ b/yt/data_objects/data_containers.py
@@ -24,6 +24,8 @@
from contextlib import contextmanager
from yt.data_objects.particle_io import particle_handler_registry
+from yt.fields.derived_field import \
+ DerivedField
from yt.frontends.ytdata.utilities import \
save_as_dataset
from yt.funcs import \
@@ -348,6 +350,7 @@
for i, chunk in enumerate(chunks):
with self._chunked_read(chunk):
gz = self._current_chunk.objs[0]
+ gz.field_parameters = self.field_parameters
wogz = gz._base_grid
ind += wogz.select(
self.selector,
@@ -982,6 +985,9 @@
raise YTFieldNotParseable(field)
ftype, fname = field
finfo = self.ds._get_field_info(ftype, fname)
+ elif isinstance(field, DerivedField):
+ ftype, fname = field.name
+ finfo = field
else:
fname = field
finfo = self.ds._get_field_info("unknown", fname)
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/data_objects/profiles.py
--- a/yt/data_objects/profiles.py
+++ b/yt/data_objects/profiles.py
@@ -15,6 +15,7 @@
import numpy as np
+from yt.fields.derived_field import DerivedField
from yt.frontends.ytdata.utilities import \
save_as_dataset
from yt.funcs import \
@@ -249,8 +250,11 @@
def __getitem__(self, field):
fname = self.field_map.get(field, None)
- if fname is None and isinstance(field, tuple):
- fname = self.field_map.get(field[1], None)
+ if fname is None:
+ if isinstance(field, tuple):
+ fname = self.field_map.get(field[1], None)
+ elif isinstance(field, DerivedField):
+ fname = self.field_map.get(field.name[1], None)
if fname is None:
raise KeyError(field)
else:
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/data_objects/static_output.py
--- a/yt/data_objects/static_output.py
+++ b/yt/data_objects/static_output.py
@@ -15,6 +15,7 @@
#-----------------------------------------------------------------------------
import functools
+import itertools
import numpy as np
import os
import time
@@ -24,6 +25,8 @@
from yt.extern.six import add_metaclass, string_types
from yt.config import ytcfg
+from yt.fields.derived_field import \
+ DerivedField
from yt.funcs import \
mylog, \
set_intersection, \
@@ -62,7 +65,6 @@
from yt.units.unit_systems import create_code_unit_system
from yt.data_objects.region_expression import \
RegionExpression
-
from yt.geometry.coordinates.api import \
CoordinateHandler, \
CartesianCoordinateHandler, \
@@ -171,6 +173,7 @@
derived_field_list = requires_index("derived_field_list")
fields = requires_index("fields")
_instantiated = False
+ _particle_type_counts = None
def __new__(cls, filename=None, *args, **kwargs):
if not isinstance(filename, string_types):
@@ -590,7 +593,10 @@
def _get_field_info(self, ftype, fname = None):
self.index
if fname is None:
- ftype, fname = "unknown", ftype
+ if isinstance(ftype, DerivedField):
+ ftype, fname = ftype.name
+ else:
+ ftype, fname = "unknown", ftype
guessing_type = False
if ftype == "unknown":
guessing_type = True
@@ -763,6 +769,27 @@
return fields
@property
+ def particles_exist(self):
+ for pt, f in itertools.product(self.particle_types_raw, self.field_list):
+ if pt == f[0]:
+ return True
+ return False
+
+ @property
+ def particle_type_counts(self):
+ self.index
+ if self.particles_exist is False:
+ return {}
+
+ # frontends or index implementation can populate this dict while
+ # creating the index if they know particle counts at that time
+ if self._particle_type_counts is not None:
+ return self._particle_type_counts
+
+ self._particle_type_counts = self.index._get_particle_type_counts()
+ return self._particle_type_counts
+
+ @property
def ires_factor(self):
o2 = np.log2(self.refine_by)
if o2 != int(o2):
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/data_objects/tests/test_dataset_access.py
--- a/yt/data_objects/tests/test_dataset_access.py
+++ b/yt/data_objects/tests/test_dataset_access.py
@@ -1,4 +1,8 @@
-from yt.testing import fake_amr_ds, assert_equal
+from yt.testing import \
+ assert_equal, \
+ fake_amr_ds, \
+ fake_particle_ds, \
+ fake_random_ds
# This will test the "dataset access" method.
@@ -37,3 +41,10 @@
rho *= 2.0
yield assert_equal, dd["density"]*2.0, ds.r["density"]
yield assert_equal, dd["gas", "density"]*2.0, ds.r["gas", "density"]
+
+def test_particle_counts():
+ ds = fake_random_ds(16, particles=100)
+ assert ds.particle_type_counts == {'io': 100}
+
+ pds = fake_particle_ds(npart=128)
+ assert pds.particle_type_counts == {'io': 128}
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/fields/astro_fields.py
--- a/yt/fields/astro_fields.py
+++ b/yt/fields/astro_fields.py
@@ -136,7 +136,8 @@
registry.add_field((ftype, "sz_kinetic"),
function=_sz_kinetic,
units=unit_system["length"]**-1,
- validators=[ValidateParameter("axis")])
+ validators=[
+ ValidateParameter("axis", {'axis': [0, 1, 2]})])
def _szy(field, data):
scale = 0.88 / mh * kboltz / (me * clight*clight) * sigma_thompson
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/fields/derived_field.py
--- a/yt/fields/derived_field.py
+++ b/yt/fields/derived_field.py
@@ -23,6 +23,7 @@
NeedsDataField, \
NeedsProperty, \
NeedsParameter, \
+ NeedsParameterValue, \
FieldUnitsError
from .field_detector import \
FieldDetector
@@ -256,14 +257,21 @@
pass
class ValidateParameter(FieldValidator):
- def __init__(self, parameters):
+ def __init__(self, parameters, parameter_values=None):
"""
This validator ensures that the dataset has a given parameter.
+
+ If *parameter_values* is supplied, this will also ensure that the field
+ is available for all permutations of the field parameter.
"""
FieldValidator.__init__(self)
self.parameters = ensure_list(parameters)
+ self.parameter_values = parameter_values
def __call__(self, data):
doesnt_have = []
+ if self.parameter_values is not None:
+ if isinstance(data, FieldDetector):
+ raise NeedsParameterValue(self.parameter_values)
for p in self.parameters:
if not data.has_field_parameter(p):
doesnt_have.append(p)
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/fields/field_detector.py
--- a/yt/fields/field_detector.py
+++ b/yt/fields/field_detector.py
@@ -17,7 +17,8 @@
from collections import defaultdict
from yt.units.yt_array import YTArray
from .field_exceptions import \
- NeedsGridType
+ NeedsGridType, \
+ NeedsParameterValue
class FieldDetector(defaultdict):
Level = 1
@@ -26,7 +27,7 @@
_id_offset = 0
domain_id = 0
- def __init__(self, nd = 16, ds = None, flat = False):
+ def __init__(self, nd = 16, ds = None, flat = False, field_parameters=None):
self.nd = nd
self.flat = flat
self._spatial = not flat
@@ -36,6 +37,7 @@
self.LeftEdge = [0.0, 0.0, 0.0]
self.RightEdge = [1.0, 1.0, 1.0]
self.dds = np.ones(3, "float64")
+ self.field_parameters = field_parameters
class fake_dataset(defaultdict):
pass
@@ -106,6 +108,32 @@
for i in nfd.requested_parameters:
if i not in self.requested_parameters:
self.requested_parameters.append(i)
+ except NeedsParameterValue as npv:
+ # redo field detection with a new FieldDetector, ensuring
+ # all needed field parameter values are set
+ for param in npv.parameter_values:
+ # temporarily remove any ValidateParameter instances for
+ # this field to avoid infinitely re-raising
+ # NeedsParameterValue exceptions
+ saved_validators = []
+ for i, validator in enumerate(finfo.validators):
+ params = getattr(validator, 'parameters', [])
+ if param in params:
+ saved_validators.append(validator)
+ del finfo.validators[i]
+
+ for pv in npv.parameter_values[param]:
+ nfd = FieldDetector(self.nd, ds=self.ds,
+ field_parameters={param: pv})
+ vv = finfo(nfd)
+ for i in nfd.requested:
+ if i not in self.requested:
+ self.requested.append(i)
+ for i in nfd.requested_parameters:
+ if i not in self.requested_parameters:
+ self.requested_parameters.append(i)
+
+ finfo.validators.extend(saved_validators)
if vv is not None:
if not self.flat: self[item] = vv
else: self[item] = vv.ravel()
@@ -176,6 +204,8 @@
}
def get_field_parameter(self, param, default = 0.0):
+ if self.field_parameters and param in self.field_parameters:
+ return self.field_parameters[param]
self.requested_parameters.append(param)
if param in ['bulk_velocity', 'center', 'normal']:
return self.ds.arr(np.random.random(3) * 1e-2, self.fp_units[param])
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/fields/field_exceptions.py
--- a/yt/fields/field_exceptions.py
+++ b/yt/fields/field_exceptions.py
@@ -46,6 +46,10 @@
def __str__(self):
return "(%s)" % (self.missing_parameters)
+class NeedsParameterValue(ValidationException):
+ def __init__(self, parameter_values):
+ self.parameter_values = parameter_values
+
class NeedsConfiguration(ValidationException):
def __init__(self, parameter, value):
self.parameter = parameter
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/fields/field_info_container.py
--- a/yt/fields/field_info_container.py
+++ b/yt/fields/field_info_container.py
@@ -253,12 +253,17 @@
self[name] = DerivedField(name, f, **kwargs)
return f
return create_function
- ptype = kwargs.get("particle_type", False)
- if ptype:
+
+ if isinstance(name, tuple):
+ self[name] = DerivedField(name, function, **kwargs)
+ return
+
+ if kwargs.get("particle_type", False):
ftype = 'all'
else:
ftype = self.ds.default_fluid_type
- if not isinstance(name, tuple) and (ftype, name) not in self:
+
+ if (ftype, name) not in self:
tuple_name = (ftype, name)
self[tuple_name] = DerivedField(tuple_name, function, **kwargs)
self.alias(name, tuple_name)
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/fields/tests/test_field_access.py
--- /dev/null
+++ b/yt/fields/tests/test_field_access.py
@@ -0,0 +1,40 @@
+from yt.testing import fake_random_ds, assert_equal
+from yt.data_objects.profiles import create_profile
+from yt.visualization.plot_window import \
+ SlicePlot, \
+ ProjectionPlot, \
+ OffAxisProjectionPlot
+from yt.visualization.profile_plotter import \
+ ProfilePlot, \
+ PhasePlot
+
+def test_field_access():
+ ds = fake_random_ds(16)
+
+ ad = ds.all_data()
+ sp = ds.sphere(ds.domain_center, 0.25)
+ cg = ds.covering_grid(0, ds.domain_left_edge, ds.domain_dimensions)
+ scg = ds.smoothed_covering_grid(0, ds.domain_left_edge, ds.domain_dimensions)
+ sl = ds.slice(0, ds.domain_center[0])
+ proj = ds.proj('density', 0)
+ prof = create_profile(ad, 'radius', 'density')
+
+ for data_object in [ad, sp, cg, scg, sl, proj, prof]:
+ assert_equal(
+ data_object['gas', 'density'],
+ data_object[ds.fields.gas.density]
+ )
+
+ for field in [('gas', 'density'), ds.fields.gas.density]:
+ ad = ds.all_data()
+ prof = ProfilePlot(ad, 'radius', field)
+ phase = PhasePlot(ad, 'radius', field, 'cell_mass')
+ s = SlicePlot(ds, 2, field)
+ oas = SlicePlot(ds, [1, 1, 1], field)
+ p = ProjectionPlot(ds, 2, field)
+ oap = OffAxisProjectionPlot(ds, [1, 1, 1], field)
+
+ for plot_object in [s, oas, p, oap, prof, phase]:
+ plot_object._setup_plots()
+ if hasattr(plot_object, '_frb'):
+ plot_object._frb[field]
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/art/api.py
--- a/yt/frontends/art/api.py
+++ b/yt/frontends/art/api.py
@@ -17,7 +17,8 @@
ARTDomainFile,\
ARTDomainSubset,\
ARTIndex,\
- ARTDataset
+ ARTDataset, \
+ DarkMatterARTDataset
from .fields import \
ARTFieldInfo
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/art/data_structures.py
--- a/yt/frontends/art/data_structures.py
+++ b/yt/frontends/art/data_structures.py
@@ -337,6 +337,8 @@
mylog.info("Discovered %i species of particles", len(ls_nonzero))
mylog.info("Particle populations: "+'%9i '*len(ls_nonzero),
*ls_nonzero)
+ self._particle_type_counts = dict(
+ zip(self.particle_types_raw, ls_nonzero))
for k, v in particle_header_vals.items():
if k in self.parameters.keys():
if not self.parameters[k] == v:
@@ -631,6 +633,10 @@
return False
if not f.endswith(suffix):
return False
+ if "s0" not in f:
+ # ATOMIC.DAT, for instance, passes the other tests, but then dies
+ # during _find_files because it can't be split.
+ return False
with open(f, 'rb') as fh:
try:
amr_prefix, amr_suffix = filename_pattern['amr']
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/art/tests/test_outputs.py
--- a/yt/frontends/art/tests/test_outputs.py
+++ b/yt/frontends/art/tests/test_outputs.py
@@ -69,6 +69,14 @@
ad[('specie2', 'particle_type')].size +
ad[('specie3', 'particle_type')].size), AnaNDM
+ for spnum in range(5):
+ npart_read = ad['specie%s' % spnum, 'particle_type'].size
+ npart_header = ds.particle_type_counts['specie%s' % spnum]
+ if spnum == 3:
+ # see issue 814
+ npart_read += 1
+ assert_equal(npart_read, npart_header)
+
AnaBoxSize = YTQuantity(7.1442196564, 'Mpc')
AnaVolume = YTQuantity(364.640074656, 'Mpc**3')
Volume = 1
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/artio/data_structures.py
--- a/yt/frontends/artio/data_structures.py
+++ b/yt/frontends/artio/data_structures.py
@@ -182,6 +182,17 @@
return (self.dataset.domain_width /
(self.dataset.domain_dimensions * 2**(self.max_level))).min()
+ def _get_particle_type_counts(self):
+ # this could be done in the artio C interface without creating temporary
+ # arrays but I don't want to touch that code
+ # if a future brave soul wants to try, take a look at
+ # `read_sfc_particles` in _artio_caller.pyx
+ result = {}
+ ad = self.ds.all_data()
+ for ptype in self.ds.particle_types_raw:
+ result[ptype] = ad[ptype, 'PID'].size
+ return result
+
def convert(self, unit):
return self.dataset.conversion_factors[unit]
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/artio/tests/test_outputs.py
--- a/yt/frontends/artio/tests/test_outputs.py
+++ b/yt/frontends/artio/tests/test_outputs.py
@@ -48,6 +48,7 @@
s1 = dobj["ones"].sum()
s2 = sum(mask.sum() for block, mask in dobj.blocks)
yield assert_equal, s1, s2
+ assert_equal(ds.particle_type_counts, {'N-BODY': 100000, 'STAR': 110650})
@requires_file(sizmbhloz)
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/enzo/data_structures.py
--- a/yt/frontends/enzo/data_structures.py
+++ b/yt/frontends/enzo/data_structures.py
@@ -483,6 +483,15 @@
random_sample = np.mgrid[0:max(len(self.grids),1)].astype("int32")
return self.grids[(random_sample,)]
+ def _get_particle_type_counts(self):
+ try:
+ ret = {}
+ for ptype in self.grid_active_particle_count:
+ ret[ptype] = self.grid_active_particle_count[ptype].sum()
+ return ret
+ except AttributeError:
+ return super(EnzoHierarchy, self)._get_particle_type_counts()
+
def find_particles_by_type(self, ptype, max_num=None, additional_fields=None):
"""
Returns a structure of arrays with all of the particles'
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/enzo/tests/test_outputs.py
--- a/yt/frontends/enzo/tests/test_outputs.py
+++ b/yt/frontends/enzo/tests/test_outputs.py
@@ -83,6 +83,7 @@
for test in big_patch_amr(ds, _fields):
test_galaxy0030.__name__ = test.description
yield test
+ assert_equal(ds.particle_type_counts, {'io': 1124453})
@requires_ds(enzotiny)
def test_simulated_halo_mass_function():
@@ -152,3 +153,6 @@
[f for f in apcos.field_list if f[0] == 'CenOstriker'])
assert_equal(apcos_fields, real_apcos_fields)
+
+ assert_equal(apcos.particle_type_counts,
+ {'CenOstriker': 899755, 'DarkMatter': 32768})
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/gizmo/fields.py
--- a/yt/frontends/gizmo/fields.py
+++ b/yt/frontends/gizmo/fields.py
@@ -99,6 +99,15 @@
data[ptype, "%s_metallicity" % species]
num_neighbors = 64
+ for species in ['H', 'H_p0', 'H_p1']:
+ for suf in ["_density", "_number_density"]:
+ field = "%s%s" % (species, suf)
+ fn = add_volume_weighted_smoothed_field(
+ ptype, "particle_position", "particle_mass",
+ "smoothing_length", "density", field,
+ self, num_neighbors)
+ self.alias(("gas", field), fn[0])
+
for species in self.nuclei_names:
self.add_field(
(ptype, "%s_nuclei_mass_density" % species),
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/ramses/data_structures.py
--- a/yt/frontends/ramses/data_structures.py
+++ b/yt/frontends/ramses/data_structures.py
@@ -474,6 +474,13 @@
for level in range(self.max_level+1):
self.level_stats[level+self.dataset.min_level+1]['numcells'] = levels[level]
+ def _get_particle_type_counts(self):
+ npart = 0
+ for dom in self.domains:
+ npart += dom.local_particle_count
+
+ return {'io': npart}
+
def print_stats(self):
# This function prints information based on the fluid on the grids,
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/frontends/ramses/tests/test_outputs.py
--- a/yt/frontends/ramses/tests/test_outputs.py
+++ b/yt/frontends/ramses/tests/test_outputs.py
@@ -47,7 +47,7 @@
s1 = dobj["ones"].sum()
s2 = sum(mask.sum() for block, mask in dobj.blocks)
yield assert_equal, s1, s2
-
+ assert_equal(ds.particle_type_counts, {'io': 1090895})
@requires_file(output_00080)
def test_RAMSESDataset():
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/geometry/coordinates/cartesian_coordinates.py
--- a/yt/geometry/coordinates/cartesian_coordinates.py
+++ b/yt/geometry/coordinates/cartesian_coordinates.py
@@ -76,10 +76,15 @@
field_data = ad[field]
buff_size = size[0:dimension] + (1,) + size[dimension:]
+ ax = data_source.axis
+ xax = self.x_axis[ax]
+ yax = self.y_axis[ax]
c = np.float64(data_source.center[dimension].d)
- bounds.insert(2*dimension, c)
- bounds.insert(2*dimension, c)
- bounds = np.reshape(bounds, (3, 2))
+
+ extents = np.zeros((3, 2))
+ extents[ax] = np.array([c, c])
+ extents[xax] = bounds[0:2]
+ extents[yax] = bounds[2:4]
# if this is an element field, promote to 2D here
if len(field_data.shape) == 1:
@@ -101,13 +106,10 @@
img = pixelize_element_mesh(coords,
indices,
- buff_size, field_data, bounds,
+ buff_size, field_data, extents,
index_offset=offset)
# re-order the array and squeeze out the dummy dim
- ax = data_source.axis
- xax = self.x_axis[ax]
- yax = self.y_axis[ax]
return np.squeeze(np.transpose(img, (yax, xax, ax)))
elif dimension < 3:
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/geometry/geometry_handler.py
--- a/yt/geometry/geometry_handler.py
+++ b/yt/geometry/geometry_handler.py
@@ -57,10 +57,6 @@
mylog.debug("Detecting fields.")
self._detect_output_fields()
- def __del__(self):
- if self._data_file is not None:
- self._data_file.close()
-
def _initialize_state_variables(self):
self._parallel_locking = False
self._data_file = None
@@ -195,6 +191,10 @@
except TypeError:
return self._data_file[full_name]
+ def _get_particle_type_counts(self):
+ # this is implemented by subclasses
+ raise NotImplementedError
+
def _close_data_file(self):
if self._data_file:
self._data_file.close()
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/geometry/grid_geometry_handler.py
--- a/yt/geometry/grid_geometry_handler.py
+++ b/yt/geometry/grid_geometry_handler.py
@@ -119,6 +119,9 @@
"""
return self.select_grids(self.grid_levels.max())[0].dds[:].min()
+ def _get_particle_type_counts(self):
+ return {self.ds.particle_types_raw[0]: self.grid_particle_count.sum()}
+
def _initialize_level_stats(self):
# Now some statistics:
# 0 = number of grids
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/geometry/grid_visitors.pxd
--- a/yt/geometry/grid_visitors.pxd
+++ b/yt/geometry/grid_visitors.pxd
@@ -17,36 +17,36 @@
cimport numpy as np
cdef struct GridTreeNode:
- int num_children
- int level
+ np.int32_t num_children
+ np.int32_t level
np.int64_t index
np.float64_t left_edge[3]
np.float64_t right_edge[3]
GridTreeNode **children
np.int64_t start_index[3]
- int dims[3]
+ np.int32_t dims[3]
np.float64_t dds[3]
cdef struct GridTreeNodePadded:
- int num_children
- int level
- long int index
- double left_edge_x
- double left_edge_y
- double left_edge_z
- double right_edge_x
- double right_edge_y
- double right_edge_z
- long int children_pointers
- long int start_index_x
- long int start_index_y
- long int start_index_z
- int dims_x
- int dims_y
- int dims_z
- double dds_x
- double dds_y
- double dds_z
+ np.int32_t num_children
+ np.int32_t level
+ np.int64_t index
+ np.float64_t left_edge_x
+ np.float64_t left_edge_y
+ np.float64_t left_edge_z
+ np.float64_t right_edge_x
+ np.float64_t right_edge_y
+ np.float64_t right_edge_z
+ np.int64_t children_pointers
+ np.int64_t start_index_x
+ np.int64_t start_index_y
+ np.int64_t start_index_z
+ np.int32_t dims_x
+ np.int32_t dims_y
+ np.int32_t dims_z
+ np.float64_t dds_x
+ np.float64_t dds_y
+ np.float64_t dds_z
cdef struct GridVisitorData:
GridTreeNode *grid
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/geometry/particle_geometry_handler.py
--- a/yt/geometry/particle_geometry_handler.py
+++ b/yt/geometry/particle_geometry_handler.py
@@ -14,6 +14,7 @@
# The full license is in the file COPYING.txt, distributed with this software.
#-----------------------------------------------------------------------------
+import collections
import numpy as np
import os
import weakref
@@ -51,6 +52,13 @@
self.dataset.domain_left_edge)
return dx.min()
+ def _get_particle_type_counts(self):
+ result = collections.defaultdict(lambda: 0)
+ for df in self.data_files:
+ for k in df.total_particles.keys():
+ result[k] += df.total_particles[k]
+ return dict(result)
+
def convert(self, unit):
return self.dataset.conversion_factors[unit]
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/units/unit_lookup_table.py
--- a/yt/units/unit_lookup_table.py
+++ b/yt/units/unit_lookup_table.py
@@ -23,7 +23,7 @@
planck_charge_esu, planck_energy_erg, planck_mass_grams, \
planck_temperature_K, planck_time_s, mass_hydrogen_grams, \
grams_per_pound, standard_gravity_cm_per_s2, pascal_per_atm, \
- newton_cgs
+ newton_cgs, cm_per_rearth, cm_per_rjup
import numpy as np
# Lookup a unit symbol with the symbol string, and provide a tuple with the
@@ -87,6 +87,8 @@
"msun": (mass_sun_grams, dimensions.mass, 0.0, r"M_\odot"),
"Rsun": (cm_per_rsun, dimensions.length, 0.0, r"R_\odot"),
"rsun": (cm_per_rsun, dimensions.length, 0.0, r"R_\odot"),
+ "R_sun": (cm_per_rsun, dimensions.length, 0.0, r"R_\odot"),
+ "r_sun": (cm_per_rsun, dimensions.length, 0.0, r"R_\odot"),
"Lsun": (luminosity_sun_ergs_per_sec, dimensions.power, 0.0, r"L_\odot"),
"Tsun": (temp_sun_kelvin, dimensions.temperature, 0.0, r"T_\odot"),
"Zsun": (metallicity_sun, dimensions.dimensionless, 0.0, r"Z_\odot"),
@@ -151,6 +153,12 @@
"m_geom": (mass_sun_grams, dimensions.mass, 0.0, r"M_\odot"),
"l_geom": (newton_cgs*mass_sun_grams/speed_of_light_cm_per_s**2, dimensions.length, 0.0, r"M_\odot"),
"t_geom": (newton_cgs*mass_sun_grams/speed_of_light_cm_per_s**3, dimensions.time, 0.0, r"M_\odot"),
+
+ # Some Solar System units
+ "R_earth": (cm_per_rearth, dimensions.length, 0.0, r"R_\oplus"),
+ "r_earth": (cm_per_rearth, dimensions.length, 0.0, r"R_\oplus"),
+ "R_jup": (cm_per_rjup, dimensions.length, 0.0, r"R_\mathrm{Jup}"),
+ "r_jup": (cm_per_rjup, dimensions.length, 0.0, r"R_\mathrm{Jup}"),
}
# This dictionary formatting from magnitude package, credit to Juan Reyero.
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/units/unit_symbols.py
--- a/yt/units/unit_symbols.py
+++ b/yt/units/unit_symbols.py
@@ -116,11 +116,11 @@
Msun = solar_mass = quan(1.0, "Msun")
msun = quan(1.0, "msun")
-Rsun = solar_radius = quan(1.0, "Rsun")
-rsun = quan(1.0, "rsun")
-Lsun = lsun = solar_luminosity = quan(1.0, "Lsun")
-Tsun = solar_temperature = quan(1.0, "Tsun")
-Zsun = solar_metallicity = quan(1.0, "Zsun")
+Rsun = R_sun = solar_radius = quan(1.0, "Rsun")
+rsun = r_sun = quan(1.0, "rsun")
+Lsun = lsun = l_sun = solar_luminosity = quan(1.0, "Lsun")
+Tsun = T_sun = solar_temperature = quan(1.0, "Tsun")
+Zsun = Z_sun = solar_metallicity = quan(1.0, "Zsun")
#
# Misc Astronomical units
@@ -129,6 +129,10 @@
AU = astronomical_unit = quan(1.0, "AU")
au = quan(1.0, "au")
ly = light_year = quan(1.0, "ly")
+Rearth = R_earth = earth_radius = quan(1.0, 'R_earth')
+rearth = r_earth = quan(1.0, 'r_earth')
+Rjup = R_jup = jupiter_radius = quan(1.0, 'R_jup')
+rjup = r_jup = quan(1.0, 'r_jup')
#
# Physical units
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/utilities/amr_kdtree/amr_kdtree.py
--- a/yt/utilities/amr_kdtree/amr_kdtree.py
+++ b/yt/utilities/amr_kdtree/amr_kdtree.py
@@ -25,7 +25,7 @@
scatter_image
from yt.utilities.lib.amr_kdtools import \
Node, \
- add_pygrids, \
+ add_grids, \
find_node, \
kd_is_leaf, \
set_dirty, \
@@ -95,7 +95,7 @@
gles = np.array([g.LeftEdge for g in grids])
gres = np.array([g.RightEdge for g in grids])
gids = np.array([g.id for g in grids], dtype="int64")
- add_pygrids(self.trunk, gids.size, gles, gres, gids,
+ add_grids(self.trunk, gids.size, gles, gres, gids,
self.comm_rank, self.comm_size)
del gles, gres, gids, grids
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/utilities/file_handler.py
--- a/yt/utilities/file_handler.py
+++ b/yt/utilities/file_handler.py
@@ -14,7 +14,6 @@
#-----------------------------------------------------------------------------
from yt.utilities.on_demand_imports import _h5py as h5py
-from distutils.version import LooseVersion
class HDF5FileHandler(object):
handle = None
@@ -22,15 +21,6 @@
def __init__(self, filename):
self.handle = h5py.File(filename, 'r')
- def __del__(self):
- if self.handle is not None:
- # In h5py 2.4 and newer file handles are closed automatically.
- # so only close the handle on old versions. This works around an
- # issue in h5py when run under python 3.4.
- # See https://github.com/h5py/h5py/issues/534
- if LooseVersion(h5py.__version__) < '2.4.0':
- self.handle.close()
-
def __getitem__(self, key):
return self.handle[key]
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/utilities/lib/alt_ray_tracers.pyx
--- a/yt/utilities/lib/alt_ray_tracers.pyx
+++ b/yt/utilities/lib/alt_ray_tracers.pyx
@@ -101,7 +101,7 @@
rleft, rright, zleft, zright, \
cleft, cright, thetaleft, thetaright, \
tmleft, tpleft, tmright, tpright, tsect
- cdef np.ndarray[np.intp_t, ndim=1] inds, tinds, sinds
+ cdef np.ndarray[np.int64_t, ndim=1, cast=True] inds, tinds, sinds
cdef np.ndarray[np.float64_t, ndim=2] xyz, rztheta, ptemp, b1, b2, dsect
# set up points
diff -r 2ae7e0f42c8ee896a81fc2b3b981cd1a994fadaa -r a0f0edfe2bdf9ad49dba95c5cf8c1edb6820feb8 yt/utilities/lib/amr_kdtools.pxd
--- a/yt/utilities/lib/amr_kdtools.pxd
+++ b/yt/utilities/lib/amr_kdtools.pxd
@@ -35,6 +35,6 @@
cdef Split * split
cdef int level
-cdef int point_in_node(Node node, np.ndarray[np.float64_t, ndim=1] point)
-cdef Node _find_node(Node node, np.float64_t *point)
+cdef int point_in_node(Node node, np.float64_t[:] point)
+cdef Node _find_node(Node node, np.float64_t[:] point)
cdef int _kd_is_leaf(Node node)
This diff is so big that we needed to truncate the remainder.
https://bitbucket.org/yt_analysis/yt/commits/dd9b133f7742/
Changeset: dd9b133f7742
Branch: yt
User: ngoldbaum
Date: 2016-08-03 18:09:09+00:00
Summary: Merged in jwise77/yt (pull request #2306)
Adding new rockstar halo properties from rockstar-galaxies fork
Affected #: 1 file
diff -r 69a338edb13e0241936fec3c26a25982de325653 -r dd9b133f77428e809ffca42829f822ba1dd9bfe2 yt/frontends/rockstar/definitions.py
--- a/yt/frontends/rockstar/definitions.py
+++ b/yt/frontends/rockstar/definitions.py
@@ -38,7 +38,7 @@
# Note the final field here, which is a field for min/max format revision in
# which the field appears.
-KNOWN_REVISIONS=[0, 1]
+KNOWN_REVISIONS=[0, 1, 2]
halo_dt = [
('particle_identifier', np.int64),
@@ -101,6 +101,12 @@
('min_pos_err', np.float32),
('min_vel_err', np.float32),
('min_bulkvel_err', np.float32),
+ ('type', np.int32, (2, 100)),
+ ('sm', np.float32, (2, 100)),
+ ('gas', np.float32, (2, 100)),
+ ('bh', np.float32, (2, 100)),
+ ('peak_density', np.float32, (2, 100)),
+ ('av_density', np.float32, (2, 100)),
]
halo_dts = {}
Repository URL: https://bitbucket.org/yt_analysis/yt/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the yt-svn
mailing list