[yt-svn] commit/yt: 2 new changesets

[commits-noreply at bitbucket.org]

2 new commits in yt:

https://bitbucket.org/yt_analysis/yt/commits/d2084505cfbe/
Changeset:   d2084505cfbe
User:        ngoldbaum
Date:        2017-09-05 16:08:28+00:00
Summary:     Hanlde invalid tipsy aux files

Right now if a file matching the tipsy aux file glob pattern is present in the
output directory, we attempt to parse the file as a valid tipsy aux file
die with a ValueError. Instead, let's detect this case and remove the field
from the list of detected tipsy auxiliary fields.

This was found in the context of swithing between mainline and the demeshening
branch. In the demeshening we create sidecar files (in particular, one named
`galaxy.00300.kdtree`) that are detected by mailine yt as auxiliary
files. However, this would happen for users as well if they happen to have a
file with a name matching the aux file glob pattern we use in their output
directory.
Affected #:  1 file

diff -r e6ea9f86b3eeaf95f3ece1ccec8b35253eff4459 -r d2084505cfbe6ccfaf79bd1d2cf0edae93636a23 yt/frontends/tipsy/io.py
--- a/yt/frontends/tipsy/io.py
+++ b/yt/frontends/tipsy/io.py
@@ -325,8 +325,10 @@
         for f in glob.glob(data_file.filename + '.*'):
             afield = f.rsplit('.')[-1]
             filename = data_file.filename + '.' + afield
-            if not os.path.exists(filename): continue
+            if not os.path.exists(filename):
+                continue
             self._aux_fields.append(afield)
+        skip_afields = []
         for afield in self._aux_fields:
             filename = data_file.filename + '.' + afield
             # We need to do some fairly ugly detection to see what format the
@@ -338,6 +340,11 @@
                            count=1) != tot_parts:
                 with open(filename, 'rb') as f:
                     header_nparts = f.readline()
+                    try:
+                        header_nparts = int(header_nparts)
+                    except ValueError:
+                        skip_afields.append(afield)
+                        continue
                     if int(header_nparts) != tot_parts:
                         raise RuntimeError
                 self._aux_pdtypes[afield] = "ascii"
@@ -349,8 +356,9 @@
                 else:
                     self._aux_pdtypes[afield] = np.dtype([('aux', endian + 'f')])
             else:
-                raise RuntimeError
-
+                skip_afields.append(afield)
+        for afield in skip_afields:
+            self._aux_fields.remove(afield)
         # Add the auxiliary fields to each ptype we have
         for ptype in self._ptypes:
             if any([ptype == field[0] for field in self._field_list]):


https://bitbucket.org/yt_analysis/yt/commits/8bed49af0634/
Changeset:   8bed49af0634
User:        bwkeller
Date:        2017-09-07 15:56:53+00:00
Summary:     Merge pull request #1549 from ngoldbaum/tipsy-aux-fix

Handle invalid tipsy aux files
Affected #:  1 file

diff -r 566d0c5b179b7ad571a03c3723218340365c2bf2 -r 8bed49af0634477ff8ec58b5a7d266fc593193a5 yt/frontends/tipsy/io.py
--- a/yt/frontends/tipsy/io.py
+++ b/yt/frontends/tipsy/io.py
@@ -325,8 +325,10 @@
         for f in glob.glob(data_file.filename + '.*'):
             afield = f.rsplit('.')[-1]
             filename = data_file.filename + '.' + afield
-            if not os.path.exists(filename): continue
+            if not os.path.exists(filename):
+                continue
             self._aux_fields.append(afield)
+        skip_afields = []
         for afield in self._aux_fields:
             filename = data_file.filename + '.' + afield
             # We need to do some fairly ugly detection to see what format the
@@ -338,6 +340,11 @@
                            count=1) != tot_parts:
                 with open(filename, 'rb') as f:
                     header_nparts = f.readline()
+                    try:
+                        header_nparts = int(header_nparts)
+                    except ValueError:
+                        skip_afields.append(afield)
+                        continue
                     if int(header_nparts) != tot_parts:
                         raise RuntimeError
                 self._aux_pdtypes[afield] = "ascii"
@@ -349,8 +356,9 @@
                 else:
                     self._aux_pdtypes[afield] = np.dtype([('aux', endian + 'f')])
             else:
-                raise RuntimeError
-
+                skip_afields.append(afield)
+        for afield in skip_afields:
+            self._aux_fields.remove(afield)
         # Add the auxiliary fields to each ptype we have
         for ptype in self._ptypes:
             if any([ptype == field[0] for field in self._field_list]):

Repository URL: https://bitbucket.org/yt_analysis/yt/

--

This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.