[yt-svn] commit/yt: MatthewTurk: Merged in jzuhone/yt (pull request #1692)
commits-noreply at bitbucket.org
commits-noreply at bitbucket.org
Thu Aug 13 09:10:54 PDT 2015
1 new commit in yt:
https://bitbucket.org/yt_analysis/yt/commits/ef1a228a21cc/
Changeset: ef1a228a21cc
Branch: yt
User: MatthewTurk
Date: 2015-08-13 16:10:44+00:00
Summary: Merged in jzuhone/yt (pull request #1692)
Adding loadtxt and savetxt convenience functions.
Affected #: 5 files
diff -r 3e6809ef522e9b61ac781688fa3dd3d4930792a9 -r ef1a228a21cc2a8736606107e929806ca6d0c60d doc/source/analyzing/units/1)_Symbolic_Units.ipynb
--- a/doc/source/analyzing/units/1)_Symbolic_Units.ipynb
+++ b/doc/source/analyzing/units/1)_Symbolic_Units.ipynb
@@ -1,7 +1,7 @@
{
"metadata": {
"name": "",
- "signature": "sha256:5d881061b9e82bd9df5d3598983c8ddc5fbec35e3bf7ae4524430dc558e27489"
+ "signature": "sha256:0dbaef644354e4d0191367f8f90e6dfd0d3d527925ef0331e1ef381c9099a8cd"
},
"nbformat": 3,
"nbformat_minor": 0,
@@ -516,9 +516,147 @@
"language": "python",
"metadata": {},
"outputs": []
+ },
+ {
+ "cell_type": "heading",
+ "level": 3,
+ "metadata": {},
+ "source": [
+ "Saving and Loading `YTArray`s to/from disk"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`YTArray`s can be written to disk, to be loaded again to be used in yt or in a different context later. There are two formats that can be written to/read from: HDF5 and ASCII. \n",
+ "\n",
+ "To write to HDF5, use `write_hdf5`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "my_dens = YTArray(np.random.random(10), 'Msun/kpc**3')\n",
+ "my_temp = YTArray(np.random.random(10), 'K')\n",
+ "my_dens.write_hdf5(\"my_data.h5\", dataset_name=\"density\")\n",
+ "my_temp.write_hdf5(\"my_data.h5\", dataset_name=\"temperature\")"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Where we used the `dataset_name` keyword argument to create a separate dataset for each array in the same file.\n",
+ "\n",
+ "We can use the `from_hdf5` classmethod to read the data back in:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "read_dens = YTArray.from_hdf5(\"my_data.h5\", dataset_name=\"density\")\n",
+ "print read_dens\n",
+ "print my_dens"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can use the `info` keyword argument to `write_hdf5` to write some additional data to the file, which will be stored as attributes of the dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "my_vels = YTArray(np.random.normal(10), 'km/s')\n",
+ "info = {\"source\":\"galaxy cluster\",\"user\":\"jzuhone\"}\n",
+ "my_vels.write_hdf5(\"my_data.h5\", dataset_name=\"velocity\", info=info)"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To write one or more `YTArray`s to an ASCII text file, use `yt.savetxt`, which works a lot like NumPy's `savetxt`, except with units:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "import yt\n",
+ "a = YTArray(np.random.random(size=10), \"cm\")\n",
+ "b = YTArray(np.random.random(size=10), \"g\")\n",
+ "c = YTArray(np.random.random(size=10), \"s\")\n",
+ "yt.savetxt(\"my_data.dat\", [a,b,c], header='My cool data', footer='Data is over', delimiter=\"\\t\")"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The file we wrote can then be easily used in other contexts, such as plotting in Gnuplot, or loading into a spreadsheet, or just for causal examination. We can quickly check it here:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "%%bash \n",
+ "more my_data.dat"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can see that the header comes first, and then right before the data we have a subheader marking the units of each column. The footer comes after the data. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`yt.loadtxt` can be used to read the same data with units back in, or read data that has been generated from some other source. Just make sure it's in the format above. `loadtxt` can also selectively read from particular columns in the file with the `usecols` keyword argument:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "bb, cc = yt.loadtxt(\"my_data.dat\", usecols=(1,2), delimiter=\"\\t\")\n",
+ "print bb\n",
+ "print b\n",
+ "print\n",
+ "print cc\n",
+ "print c"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
}
],
"metadata": {}
}
]
-}
+}
\ No newline at end of file
diff -r 3e6809ef522e9b61ac781688fa3dd3d4930792a9 -r ef1a228a21cc2a8736606107e929806ca6d0c60d yt/__init__.py
--- a/yt/__init__.py
+++ b/yt/__init__.py
@@ -103,7 +103,9 @@
YTQuantity, \
uconcatenate, \
uintersect1d, \
- uunion1d
+ uunion1d, \
+ loadtxt, \
+ savetxt
from yt.fields.api import \
field_plugins, \
@@ -169,4 +171,3 @@
# Import some helpful math utilities
from yt.utilities.math_utils import \
ortho_find, quartiles, periodic_position
-
diff -r 3e6809ef522e9b61ac781688fa3dd3d4930792a9 -r ef1a228a21cc2a8736606107e929806ca6d0c60d yt/frontends/fits/data_structures.py
--- a/yt/frontends/fits/data_structures.py
+++ b/yt/frontends/fits/data_structures.py
@@ -111,6 +111,8 @@
field_units = field_units.replace(unit, known_units[unit])
n += 1
if n != len(units): field_units = "dimensionless"
+ if field_units[0] == "/":
+ field_units = "1%s" % field_units
return field_units
except KeyError:
return "dimensionless"
diff -r 3e6809ef522e9b61ac781688fa3dd3d4930792a9 -r ef1a228a21cc2a8736606107e929806ca6d0c60d yt/units/tests/test_ytarray.py
--- a/yt/units/tests/test_ytarray.py
+++ b/yt/units/tests/test_ytarray.py
@@ -35,7 +35,7 @@
YTArray, YTQuantity, \
unary_operators, binary_operators, \
uconcatenate, uintersect1d, \
- uunion1d
+ uunion1d, loadtxt, savetxt
from yt.utilities.exceptions import \
YTUnitOperationError, YTUfuncUnitError
from yt.testing import fake_random_ds, requires_module
@@ -1061,3 +1061,22 @@
yield assert_true, ret == 0.5
yield assert_true, ret.units.is_dimensionless
yield assert_true, ret.units.base_value == 1.0
+
+def test_load_and_save():
+ tmpdir = tempfile.mkdtemp()
+ curdir = os.getcwd()
+ os.chdir(tmpdir)
+
+ a = YTArray(np.random.random(10), "kpc")
+ b = YTArray(np.random.random(10), "Msun")
+ c = YTArray(np.random.random(10), "km/s")
+
+ savetxt("arrays.dat", [a,b,c], delimiter=",")
+
+ d, e = loadtxt("arrays.dat", usecols=(1,2), delimiter=",")
+
+ yield assert_array_equal, b, d
+ yield assert_array_equal, c, e
+
+ os.chdir(curdir)
+ shutil.rmtree(tmpdir)
diff -r 3e6809ef522e9b61ac781688fa3dd3d4930792a9 -r ef1a228a21cc2a8736606107e929806ca6d0c60d yt/units/yt_array.py
--- a/yt/units/yt_array.py
+++ b/yt/units/yt_array.py
@@ -40,6 +40,7 @@
from sympy import Rational
from yt.units.unit_lookup_table import unit_prefixes, prefixable_units
from yt.units.equivalencies import equivalence_registry
+from yt.utilities.logger import ytLogger as mylog
NULL_UNIT = Unit()
@@ -1229,7 +1230,7 @@
def ucross(arr1,arr2, registry=None):
"""Applies the cross product to two YT arrays.
- This wrapper around numpy.cross preserves units.
+ This wrapper around numpy.cross preserves units.
See the documentation of numpy.cross for full
details.
"""
@@ -1310,3 +1311,113 @@
else:
raise RuntimeError("Undefined operation for a YTArray subclass. "
"Received operand types (%s) and (%s)" % (cls1, cls2))
+
+def loadtxt(fname, dtype='float', delimiter='\t', usecols=None, comments='#'):
+ r"""
+ Load YTArrays with unit information from a text file. Each row in the
+ text file must have the same number of values.
+
+ Parameters
+ ----------
+ fname : str
+ Filename to read.
+ dtype : data-type, optional
+ Data-type of the resulting array; default: float.
+ delimiter : str, optional
+ The string used to separate values. By default, this is any
+ whitespace.
+ usecols : sequence, optional
+ Which columns to read, with 0 being the first. For example,
+ ``usecols = (1,4,5)`` will extract the 2nd, 5th and 6th columns.
+ The default, None, results in all columns being read.
+ comments : str, optional
+ The character used to indicate the start of a comment;
+ default: '#'.
+
+ Examples
+ --------
+ >>> temp, velx = yt.loadtxt("sphere.dat", usecols=(1,2), delimiter="\t")
+ """
+ f = open(fname, 'r')
+ next_one = False
+ units = []
+ num_cols = -1
+ for line in f.readlines():
+ words = line.strip().split()
+ if len(words) == 0:
+ continue
+ if line[0] == comments:
+ if next_one:
+ units = words[1:]
+ if len(words) == 2 and words[1] == "Units":
+ next_one = True
+ else:
+ # Here we catch the first line of numbers
+ try:
+ col_words = line.strip().split(delimiter)
+ for word in col_words:
+ float(word)
+ num_cols = len(col_words)
+ break
+ except ValueError:
+ mylog.warning("Unrecognized character at beginning of line: \"%s\"." % line[0])
+ f.close()
+ if len(units) != num_cols:
+ mylog.warning("Malformed or incomplete units header. Arrays will be "
+ "dimensionless!")
+ units = ["dimensionless"]*num_cols
+ arrays = np.loadtxt(fname, dtype=dtype, comments=comments,
+ delimiter=delimiter, converters=None,
+ unpack=True, usecols=usecols, ndmin=0)
+ if usecols is not None:
+ units = [units[col] for col in usecols]
+ mylog.info("Array units: %s" % ", ".join(units))
+ return tuple([YTArray(arr, unit) for arr, unit in zip(arrays, units)])
+
+def savetxt(fname, arrays, fmt='%.18e', delimiter='\t', header='',
+ footer='', comments='#'):
+ r"""
+ Write YTArrays with unit information to a text file.
+
+ Parameters
+ ----------
+ fname : str
+ The file to write the YTArrays to.
+ arrays : list of YTArrays or single YTArray
+ The array(s) to write to the file.
+ fmt : str or sequence of strs, optional
+ A single format (%10.5f), or a sequence of formats.
+ delimiter : str, optional
+ String or character separating columns.
+ header : str, optional
+ String that will be written at the beginning of the file, before the
+ unit header.
+ footer : str, optional
+ String that will be written at the end of the file.
+ comments : str, optional
+ String that will be prepended to the ``header`` and ``footer`` strings,
+ to mark them as comments. Default: '# ', as expected by e.g.
+ ``yt.loadtxt``.
+
+ Examples
+ --------
+ >>> sp = ds.sphere("c", (100,"kpc"))
+ >>> a = sphere["density"]
+ >>> b = sphere["temperature"]
+ >>> c = sphere["velocity_x"]
+ >>> yt.savetxt("sphere.dat", [a,b,c], header='My sphere stuff', delimiter="\t")
+ """
+ if not isinstance(arrays, list):
+ arrays = [arrays]
+ units = []
+ for array in arrays:
+ if hasattr(array, "units"):
+ units.append(str(array.units))
+ else:
+ units.append("dimensionless")
+ if header != '':
+ header += '\n'
+ header += " Units\n " + '\t'.join(units)
+ np.savetxt(fname, np.transpose(arrays), header=header,
+ fmt=fmt, delimiter=delimiter, footer=footer,
+ newline='\n', comments=comments)
Repository URL: https://bitbucket.org/yt_analysis/yt/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the yt-svn
mailing list