[yt-users] strange errors

[Matthew Turk]

Hi Geoffrey,

If you're using a script that iterates with an individual python
instance (i.e., calling "python my_script.py") for each data dump,
then I don't think it's an OOM unless somehow multiple instances are
running simultaneously.

I'm not sure what to suggest other than talking to the help desk at
the particular supercomputer center.

-Matt

On Tue, Jul 3, 2012 at 5:32 AM, Geoffrey So <gsiisg at gmail.com> wrote:
> I am trying to make projections and slices in the same script and an
> iterating over several DD datasets, but after a while I get this error
> pasted below, it seems that there's some hdf5 error where h5py cannot open a
> file.  I've made sure the file exists and the script continues to run if I
> change the DD#### to continue where it failed.  I've already modified the
> script so that each DD is using a new python instace intead of iterating
> through the files with one python call.  I tried this in queue and
> interactive and get the same error, sometimes after many DD, but sometimes
> after only 1 or 2.  Is this indicative of memory issue?   If it is, I'm not
> getting the OOM messages even in interactive node.  Has anyone encountered
> this before?  I've stuck in import gc at the top and gc.collect() at the end
> of the script, too, but doesn't help.
>
> From
> G.S.
>
> Traceback (most recent call last):
>   File "moviesGrey.py", line 35, in <module>
>     proj = pf.h.proj(0, field, weight_field=field)
>   File
> "/home/ux455076/dev-yt/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/data_objects/data_containers.py",
> line 2001, in __init__
>     if self._okay_to_serialize and self.serialize:
> self._serialize(node_name=self._node_name)
>   File
> "/home/ux455076/dev-yt/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/data_objects/data_containers.py",
> line 970, in _serialize
>     self._store_fields(self._key_fields, node_name, force)
>   File
> "/home/ux455076/dev-yt/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/data_objects/data_containers.py",
> line 947, in _store_fields
>     passthrough = True)
>   File
> "/home/ux455076/dev-yt/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 241, in in_order
>     f2(*args, **kwargs)
>   File
> "/home/ux455076/dev-yt/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/data_objects/hierarchy.py",
> line 278, in _reload_data_file
>     self._data_file = h5py.File(self.__data_filename, self._data_mode)
>   File
> "/home/ux455076/dev-yt/lib/python2.7/site-packages/h5py/_hl/files.py", line
> 150, in __init__
>     fid = make_fid(name, mode, fapl)
>   File
> "/home/ux455076/dev-yt/lib/python2.7/site-packages/h5py/_hl/files.py", line
> 45, in make_fid
>     fid = h5f.open(name, h5f.ACC_RDONLY, fapl=plist)
>   File "h5f.pyx", line 70, in h5py.h5f.open (h5py/h5f.c:1618)
> IOError: unable to open file (File accessability: Unable to open file)
> Exception AttributeError: "'EnzoHierarchy' object has no attribute
> '_data_file'" in <bound method EnzoHierarchy.__del__ of
> <yt.frontends.enzo.data_structures.EnzoHierarchy object at 0x32a4d10>>
> ignored
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>