[Yt-svn] commit/yt-doc: MatthewTurk: Fixing a couple broken links. Extended the section on calling external codes
Bitbucket
commits-noreply at bitbucket.org
Fri Mar 4 00:07:32 PST 2011
1 new changeset in yt-doc:
http://bitbucket.org/yt_analysis/yt-doc/changeset/0a54e8bcce15/
changeset: r31:0a54e8bcce15
user: MatthewTurk
date: 2011-03-04 09:07:24
summary: Fixing a couple broken links. Extended the section on calling external codes
and writing Cython wrappers.
affected #: 6 files (7.7 KB)
--- a/source/advanced/external_analysis.rst Thu Mar 03 21:03:47 2011 -0500
+++ b/source/advanced/external_analysis.rst Fri Mar 04 03:07:24 2011 -0500
@@ -194,5 +194,209 @@
Writing and Calling our Wrapper
+++++++++++++++++++++++++++++++
+Now we begin the tricky part, of writing our wrapper code. We've already
+figured out how to build it, which is halfway to being able to test that it
+works, and we now need to start writing Cython code.
+
+For a more detailed introduction to Cython, see the Cython documentation at
+http://docs.cython.org/ . We'll cover a few of the basics for wrapping code
+however.
+
+To start out with, we need to open up and edit our file,
+``axes_calculator.pyx``. Open this in your favorite version of vi (mine is
+vim) and we will get started by declaring the struct we need to pass in. But
+first, we need to include some header information:
+
+.. code-block:: cython
+
+ import numpy as np
+ cimport numpy as np
+ cimport cython
+ from stdlib import malloc, free
+
+These lines simply import and "Cython import" some common routines. For more
+information about what is already available, see the Cython documentation. For
+now, we need to start translating our data.
+
+To do so, we tell Cython both where the struct should come from, and then we
+describe the struct itself. One fun thing to note is that if you don't need to
+set or access all the values in a struct, and it just needs to be passed around
+opaquely, you don't have to include them in the definition. For an example of
+this, see the ``png_writer.pyx`` file in the yt repository. Here's the syntax
+for pulling in (from a file called ``axes_calculator.h``) a struct like the one
+described above:
+
+.. code-block:: cython
+
+ cdef extern from "axes_calculator.h":
+ ctypedef struct ParticleCollection:
+ long npart
+ double *xpos
+ double *ypos
+ double *zpos
+
+So far, pretty easy! We've basically just translated the declaration from the
+``.h`` file. Now that we have done so, any other Cython code can create and
+manipulate these ``ParticleCollection`` structs -- which we'll do shortly.
+Next up, we need to declare the function we're going to call, which looks
+nearly exactly like the one in the ``.h`` file. (One common problem is that
+Cython doesn't know what ``const`` means, so just remove it wherever you see
+it.) Declare it like so:
+
+.. code-block:: cython
+
+ void calculate_axes(ParticleCollection *part,
+ double *ax1, double *ax2, double *ax3)
+
+Note that this is indented one level, to indicate that it, too, comes from
+``axes_calculator.h``. The next step is to create a function that accepts
+arrays and converts them to the format the struct likes. We declare our
+function just like we would a normal Python function, using ``def``. You can
+also use ``cdef`` if you only want to call a function from within Cython. We
+want to call it from Python, too, so we just use ``def``. Note that we don't
+here specify types for the various arguments. In a moment we'll refine this to
+have better argument types.
+
+.. code-block:: cython
+
+ def examine_axes(xpos, ypos, zpos):
+ cdef double ax1[3], ax2[3], ax3[3]
+ cdef ParticleCollection particles
+ cdef int i
+
+ particles.npart = len(xpos)
+ particles.xpos = <double *> malloc(particles.npart * sizeof(double))
+ particles.ypos = <double *> malloc(particles.npart * sizeof(double))
+ particles.zpos = <double *> malloc(particles.npart * sizeof(double))
+
+ for i in range(particles.npart):
+ particles.xpos[i] = xpos[i]
+ particles.ypos[i] = ypos[i]
+ particles.zpos[i] = zpos[i]
+
+ calculate_axes(particles, ax1, ax2, ax3)
+
+ free(particles.xpos)
+ free(particles.ypos)
+ free(particles.zpos)
+
+ return ( (ax1[0], ax1[1], ax1[2]),
+ (ax2[0], ax2[1], ax2[2]),
+ (ax3[0], ax3[1], ax3[2]) )
+
+This does the rest. Note that we've weaved in C-type declarations (ax1, ax2,
+ax3) and Python access to the variables fed in. This function will probably be
+quite slow -- because it doesn't know anything about the variables xpos, ypos,
+zpos, it won't be able to speed up access to them. Now we will see what we can
+do by declaring them to be of array-type before we start handling them at all.
+We can do that by annotating in the function argument list. But first, let's
+test that it works. From the directory in which you placed these files, run:
+
+.. code-block:: bash
+
+ $ python2.6 setup.py build_ext -i
+
+Now, create a sample file that feeds in the particles:
+
+.. code-block:: python
+
+ import axes_calculator
+ axes_calculator.examine_axes(xpos, ypos, zpos)
+
+Most of the time in that function is spent in converting the data. So now we
+can go back and we'll try again, rewriting our converter function to believe
+that its being fed arrays from NumPy:
+
+.. code-block:: cython
+
+ def examine_axes(np.ndarray[np.float64_t, dim=1] xpos,
+ np.ndarray[np.float64_t, dim=1] ypos,
+ np.ndarray[np.float64_t, dim=1] zpos):
+ cdef double ax1[3], ax2[3], ax3[3]
+ cdef ParticleCollection particles
+ cdef int i
+
+ particles.npart = len(xpos)
+ particles.xpos = <double *> malloc(particles.npart * sizeof(double))
+ particles.ypos = <double *> malloc(particles.npart * sizeof(double))
+ particles.zpos = <double *> malloc(particles.npart * sizeof(double))
+
+ for i in range(particles.npart):
+ particles.xpos[i] = xpos[i]
+ particles.ypos[i] = ypos[i]
+ particles.zpos[i] = zpos[i]
+
+ calculate_axes(particles, ax1, ax2, ax3)
+
+ free(particles.xpos)
+ free(particles.ypos)
+ free(particles.zpos)
+
+ return ( (ax1[0], ax1[1], ax1[2]),
+ (ax2[0], ax2[1], ax2[2]),
+ (ax3[0], ax3[1], ax3[2]) )
+
+This should be substantially faster, assuming you feed it arrays.
+
+Now, there's one last thing we can try. If we know our function won't modify
+our arrays, and they are C-Contiguous, we can simply grab pointers to the data:
+
+.. code-block:: cython
+
+ def examine_axes(np.ndarray[np.float64_t, dim=1] xpos,
+ np.ndarray[np.float64_t, dim=1] ypos,
+ np.ndarray[np.float64_t, dim=1] zpos):
+ cdef double ax1[3], ax2[3], ax3[3]
+ cdef ParticleCollection particles
+ cdef int i
+
+ particles.npart = len(xpos)
+ particles.xpos = <double *> xpos.data
+ particles.ypos = <double *> ypos.data
+ particles.zpos = <double *> zpos.data
+
+ for i in range(particles.npart):
+ particles.xpos[i] = xpos[i]
+ particles.ypos[i] = ypos[i]
+ particles.zpos[i] = zpos[i]
+
+ calculate_axes(particles, ax1, ax2, ax3)
+
+ return ( (ax1[0], ax1[1], ax1[2]),
+ (ax2[0], ax2[1], ax2[2]),
+ (ax3[0], ax3[1], ax3[2]) )
+
+But note! This will break or do weird things if you feed it arrays that are
+non-contiguous.
+
+At this point, you should have a mostly working piece of wrapper code. And it
+was pretty easy! Let us know if you run into any problems, or if you are
+interested in distributing your code with yt.
+
Exporting Data from yt
----------------------
+
+yt is installed alongside h5py. If you need to export your data from yt, to
+share it with people or to use it inside another code, h5py is a good way to do
+so. You can write out complete datasets with just a few commands. You have to
+import, and then save things out into a file.
+
+.. code-block:: python
+
+ import h5py
+ f = h5py.File("some_file.h5")
+ f.create_dataset("/data", data=some_data)
+
+This will create ``some_file.h5`` if necessary and add a new dataset
+(``/data``) to it. Writing out in ASCII should be relatively straightforward.
+For instance:
+
+.. code-block:: python
+
+ f = open("my_file.txt", "w")
+ for halo in halos:
+ x, y, z = halo.center_of_mass()
+ f.write("%0.2f %0.2f %0.2f\n", x, y, z)
+ f.close()
+
+This example could be extended to work with any data object's fields, as well.
--- a/source/advanced/index.rst Thu Mar 03 21:03:47 2011 -0500
+++ b/source/advanced/index.rst Fri Mar 04 03:07:24 2011 -0500
@@ -15,3 +15,4 @@
creating_datatypes
debugdrive
developing
+ external_analysis
--- a/source/reference/api/callback_list.rst Thu Mar 03 21:03:47 2011 -0500
+++ b/source/reference/api/callback_list.rst Fri Mar 04 03:07:24 2011 -0500
@@ -1,3 +1,5 @@
+.. _callback-api:
+
Callback List
=============
--- a/source/reference/api/extension_types.rst Thu Mar 03 21:03:47 2011 -0500
+++ b/source/reference/api/extension_types.rst Fri Mar 04 03:07:24 2011 -0500
@@ -57,6 +57,8 @@
~yt.visualization.volume_rendering.transfer_functions.TransferFunction
~yt.visualization.volume_rendering.software_sampler.VolumeRendering
+.. _image_writer:
+
Image Writing
-------------
--- a/source/reference/changelog.rst Thu Mar 03 21:03:47 2011 -0500
+++ b/source/reference/changelog.rst Fri Mar 04 03:07:24 2011 -0500
@@ -23,7 +23,7 @@
* Direct writing of PNGs (see :ref:`image_writer`)
* Multi-band image writing (see :ref:`image_writer`)
* Parallel halo merger tree (see :ref:`merger_tree`)
- * Parallel structure function generator (see :ref:`struct_fcn_gen`)
+ * Parallel structure function generator (see :ref:`two_point_functions`)
* Image pan and zoom object and display widget (see :ref:`image-panner`)
* Parallel volume rendering (see :ref:`volume_rendering`)
* Multivariate volume rendering, allowing for multiple forms of emission and
--- a/source/visualizing/plots.rst Thu Mar 03 21:03:47 2011 -0500
+++ b/source/visualizing/plots.rst Fri Mar 04 03:07:24 2011 -0500
@@ -69,8 +69,8 @@
collection. It's initially filled with the "Density" values (but that can be
changed with
:meth:`~yt.visualization.plot_collection.PlotCollection.switch_field`) and is
-set to the width of the full domain. Below, in :ref:`controlling-images`, we
-talk about how to change the viewport.
+set to the width of the full domain. Below, in
+:ref:`how-to-control-image-plots`, we talk about how to change the viewport.
.. _how-to-make-projections:
@@ -104,7 +104,7 @@
pc.add_projection("Temperature", 1, "Density")
Because we're taking an average, we are given back a Temperature, not an
-integration of Temperature. Below, in :ref:`controlling-images`, we
+integration of Temperature. Below, in :ref:`how-to-control-image-plots`, we
talk about how to change the viewport.
.. _how-to-make-oblique-slices:
@@ -129,7 +129,7 @@
Note that we can use any vector here, which means we could (for instance)
calculate the angular momentum vector of some region and supply that. Below,
-in :ref:`controlling-images`, we talk about how to change the viewport.
+in :ref:`how-to-control-image-plots`, we talk about how to change the viewport.
.. _how-to-control-image-plots:
Repository URL: https://bitbucket.org/yt_analysis/yt-doc/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the yt-svn
mailing list