[yt-users] IOError in h5py

Nathan Goldbaum nathan12343 at gmail.com
Mon Jan 7 08:58:48 PST 2013


You can also create a my_plugins.py file in the .yt directory in your
home directory.  This can contain supplementary functions and field
definitions outside of yt source control.

Cheers,

Nathan

On Jan 7, 2013, at 6:42 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Patrick,
>
> On Mon, Jan 7, 2013 at 11:38 AM, Patrick Rieser
> <patrick.rieser at uibk.ac.at> wrote:
>> Thanks a lot, it seems to be working now!
>
> Awesome, I'm glad to hear it!
>
>>
>> By the way, I did only change the fields.py in frontends/flash to include
>> some more stuff.
>
> If you think these are of general use, please feel free to submit a
> pull request.  The "yt bootstrap_dev" command will get you bitbucket
> and hg usernames.  You can then do this by forking on BitBucket:
>
> https://bitbucket.org/yt_analysis/yt/fork
>
> then pushing to your new repository:
>
> cd /home/patrick/Programme/yt-x86_64/src/yt-hg
> hg commit
> hg push https://bitbucket.org/your_username/yt
>
> and issuing a pull request:
>
> https://bitbucket.org/your_username/yt/pull-request/new
>
> But if you'd rather not, of course that's okay too.  :)
>
> Best,
>
> Matt
>
>>
>>
>> Best wishes,
>> Patrick
>>
>>
>> Am 2013-01-07 17:19, schrieb Matthew Turk:
>>
>>> Hi Patrick,
>>>
>>> Ah, interesting -- this is before the fix to the GDF frontend.  A
>>> couple things -- it looks like you've made some changes to the source
>>> code (the + at the end of the hash indicates this.)  If this isn't a
>>> big deal, or you don't need those changes, I'd recommend you update to
>>> the latest version manually:
>>>
>>> cd /home/patrick/Programme/yt-x86_64/src/yt-hg
>>> hg pull
>>> hg up -C
>>> python2.7 setup.py build_ext -i
>>>
>>> If you want to retain those changes, you can be a bit safer about it:
>>>
>>> cd /home/patrick/Programme/yt-x86_64/src/yt-hg
>>> hg diff > backup.patch
>>> hg pull
>>> hg up
>>> python2.7 setup.py build_ext -i
>>>
>>> This will prompt you to merge your changes into the tip of the
>>> development repository.  If the merge fails, you can use the contents
>>> of "backup.patch" to restore your changes later.
>>>
>>> Can you try updating and see if that fixes the issue?
>>>
>>> -Matt
>>>
>>> On Mon, Jan 7, 2013 at 11:16 AM, Patrick Rieser
>>> <patrick.rieser at uibk.ac.at> wrote:
>>>>
>>>> Here you go:
>>>>
>>>> yt module located at:
>>>>     /home/patrick/Programme/yt-x86_64/src/yt-hg
>>>> The supplemental repositories are located at:
>>>>     /home/patrick/Programme/yt-x86_64/src/yt-supplemental
>>>>
>>>> The current version of the code is:
>>>>
>>>> ---
>>>> 9d5b945ee428+ (yt) tip
>>>> ---
>>>>
>>>> This installation CAN be automatically updated.
>>>> Updated successfully.
>>>>
>>>>
>>>> Best wishes,
>>>> Patrick
>>>>
>>>> Am 2013-01-07 17:04, schrieb Matthew Turk:
>>>>
>>>>> Hi Patrick,
>>>>>
>>>>> This could also be related to a critical bug fix identified not too
>>>>> long ago.  One tricky thing about HDF5 is that once in a while
>>>>> depending on the order of operations, errors can accumulate without
>>>>> being flushed.  What may have happened is that the file was left open
>>>>> at a different juncture and then never closed; this could result in a
>>>>> non-intuitive error message.  What we identified  was that the
>>>>> GridDataFormat frontend was in fact leaving the files open, which then
>>>>> left the handle around and caused non-intuitive errors later on during
>>>>> the process.  Could you send the output of:
>>>>>
>>>>> yt instinfo
>>>>>
>>>>> This will help us determine how the error is occurring.
>>>>>
>>>>> Best,
>>>>>
>>>>> Matt
>>>>>
>>>>> On Mon, Jan 7, 2013 at 11:02 AM, Patrick Rieser
>>>>> <patrick.rieser at uibk.ac.at> wrote:
>>>>>>
>>>>>> In iyt I can load a snapshot just fine.
>>>>>>
>>>>>> It could be a problem using the Sun Grid Engine then. Here is my python
>>>>>> submit script if you need it (it sources the yt environment, does some
>>>>>> things on the data set and then submits a yt job for a subset):
>>>>>> http://paste.yt-project.org/show/3045/
>>>>>>
>>>>>>
>>>>>> h5ls output seems ok:
>>>>>> bflags                   Dataset {2633, 1}
>>>>>> block\ size              Dataset {2633, 3}
>>>>>> bounding\ box            Dataset {2633, 3, 2}
>>>>>> coordinates              Dataset {2633, 3}
>>>>>> dens                     Dataset {2633, 16, 16, 16}
>>>>>> gid                      Dataset {2633, 15}
>>>>>> integer\ runtime\ parameters Dataset {46}
>>>>>> integer\ scalars         Dataset {14}
>>>>>> logical\ runtime\ parameters Dataset {53}
>>>>>> logical\ scalars         Dataset {2}
>>>>>> magp                     Dataset {2633, 16, 16, 16}
>>>>>> magx                     Dataset {2633, 16, 16, 16}
>>>>>> magy                     Dataset {2633, 16, 16, 16}
>>>>>> magz                     Dataset {2633, 16, 16, 16}
>>>>>> node\ type               Dataset {2633}
>>>>>> pden                     Dataset {2633, 16, 16, 16}
>>>>>> pres                     Dataset {2633, 16, 16, 16}
>>>>>> processor\ number        Dataset {2633}
>>>>>> real\ runtime\ parameters Dataset {76}
>>>>>> real\ scalars            Dataset {15}
>>>>>> refine\ level            Dataset {2633}
>>>>>> sim\ info                Dataset {1}
>>>>>> string\ runtime\ parameters Dataset {78}
>>>>>> string\ scalars          Dataset {1}
>>>>>> temp                     Dataset {2633, 16, 16, 16}
>>>>>> unknown\ names           Dataset {11, 1}
>>>>>> velx                     Dataset {2633, 16, 16, 16}
>>>>>> vely                     Dataset {2633, 16, 16, 16}
>>>>>> velz                     Dataset {2633, 16, 16, 16}
>>>>>> which\ child             Dataset {2633}
>>>>>>
>>>>>>
>>>>>> Could the problem be caused by a defunct snapshot that got mixed in?
>>>>>>
>>>>>>
>>>>>> Best wishes,
>>>>>> Patrick
>>>>>>
>>>>>>
>>>>>> Am 2013-01-07 16:54, schrieb j s oishi:
>>>>>>
>>>>>> Hi Patrick,
>>>>>>
>>>>>> What happens if you do
>>>>>>
>>>>>> $ h5ls <filename>
>>>>>>
>>>>>> where <filename> is the file you're trying to open? This looks like it
>>>>>> could
>>>>>> be an error outside of yt.
>>>>>>
>>>>>> Also, can you load() any other snapshots?
>>>>>>
>>>>>> Jeff
>>>>>>
>>>>>>
>>>>>> On Mon, Jan 7, 2013 at 10:49 AM, Patrick Rieser
>>>>>> <patrick.rieser at uibk.ac.at>
>>>>>> wrote:
>>>>>>>
>>>>>>> I stumbled over some error in yt while executing a script that used to
>>>>>>> work on an older version of yt. Here is the traceback:
>>>>>>>
>>>>>>>    File "start_SGE.py", line 28, in <module>
>>>>>>>    File "/home/patrick/Auswertung/yt-run/Find_Clumps/get_name.py",
>>>>>>> line
>>>>>>> 58,
>>>>>>> in get_snapshots
>>>>>>>    File "/home/patrick/Auswertung/yt-run/Find_Clumps/get_name.py",
>>>>>>> line
>>>>>>> 43,
>>>>>>> in get_redshift
>>>>>>>    File
>>>>>>> "/home/patrick/Programme/yt-x86_64/src/yt-hg/yt/convenience.py",
>>>>>>> line 71, in load
>>>>>>>    File
>>>>>>>
>>>>>>>
>>>>>>> "/home/patrick/Programme/yt-x86_64/src/yt-hg/yt/frontends/flash/data_structures.py",
>>>>>>> line 218, in __init__
>>>>>>>    File
>>>>>>>
>>>>>>>
>>>>>>> "/home/patrick/Programme/yt-x86_64/lib/python2.7/site-packages/h5py/_hl/files.py",
>>>>>>> line 165, in __init__
>>>>>>>    File
>>>>>>>
>>>>>>>
>>>>>>> "/home/patrick/Programme/yt-x86_64/lib/python2.7/site-packages/h5py/_hl/files.py",
>>>>>>> line 57, in make_fid
>>>>>>>    File "h5f.pyx", line 70, in h5py.h5f.open (h5py/h5f.c:1626)
>>>>>>> IOError: unable to open file (File accessability: Unable to open file)
>>>>>>> Exception AttributeError: "'NoneType' object has no attribute 'close'"
>>>>>>> in
>>>>>>> ignored
>>>>>>>
>>>>>>>
>>>>>>> I'd be very thankful if somebody got any idea how to fix this!
>>>>>>>
>>>>>>> Best wishes,
>>>>>>> Patrick
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



More information about the yt-users mailing list