[yt-users] Using time series data from FLASH with particle files.

Nathan Goldbaum nathan12343 at gmail.com
Thu May 14 09:34:51 PDT 2015


A smallish (few hundred megabytes at most) public test dataset would be
very helpful. Having it available on yt-project.org/data will make it much
easier for any dev who doesn't have access to data like yours to implement
the needed functionality.

Also don't be afraid to take a stab at implementing this functionality
yourself. A surprising number of yt features were initially implemented as
needed by new users.

On Thursday, May 14, 2015, Joshua Wall <joshua.e.wall at gmail.com> wrote:

> I would like to add that this feature would be *really* nice to have.
> Flash checkpoint files are huge, and I'm doing large (and going to be much
> larger) runs. The plot and particle files are nice and small, easy to store
> and move around. I cringe at the thought of analyzing a 5 M(cpu-hrs) run of
> checkpoint files for time series data. Let me know if I can help by
> providing test data files or anything! My OOP and python skills are
> probably not at the level you'd want for actual code development, but I'll
> pitch in where I can.
>
> Joshua Wall
> Drexel University
>
> On Mon, May 11, 2015 at 4:40 PM John ZuHone <jzuhone at gmail.com
> <javascript:_e(%7B%7D,'cvml','jzuhone at gmail.com');>> wrote:
>
>> I don't think we should expect this to work, actually. We never specified
>> that it should, to my knowledge. So we need to implement it.
>>
>> John ZuHone
>> Kavli Center for Astrophysics and Space Research
>> Massachusetts Institute of Technology
>> 77 Massachusetts Ave., 37-582G
>> Cambridge, MA 02139
>> (w) 617-253-2354
>> (m) 781-708-5004
>> jzuhone at space.mit.edu
>> <javascript:_e(%7B%7D,'cvml','jzuhone at space.mit.edu');>
>> jzuhone at gmail.com <javascript:_e(%7B%7D,'cvml','jzuhone at gmail.com');>
>> http://www.jzuhone.com
>>
>> On May 11, 2015, at 4:31 PM, Nathan Goldbaum <nathan12343 at gmail.com
>> <javascript:_e(%7B%7D,'cvml','nathan12343 at gmail.com');>> wrote:
>>
>>
>>
>> On Mon, May 11, 2015 at 1:24 PM, Nathan Goldbaum <nathan12343 at gmail.com
>> <javascript:_e(%7B%7D,'cvml','nathan12343 at gmail.com');>> wrote:
>>
>>>
>>>
>>> On Mon, May 11, 2015 at 1:19 PM, Joshua Wall <joshua.e.wall at gmail.com
>>> <javascript:_e(%7B%7D,'cvml','joshua.e.wall at gmail.com');>> wrote:
>>>
>>>> Dear users,
>>>>
>>>>     Since I'm already 0 for 1 today, I thought I'd also ask about this
>>>> error I've seen. I'm trying to get the time series of momentum density in
>>>> the x-dir from a simulation in FLASH where I have both the plot and
>>>> particle files with:
>>>>
>>>> ############################################################
>>>>
>>>> ts =
>>>> yt.load("/home/josh/amuse-svn/src/amuse/community/flash/SinkMomTest2/SMT_hdf5_plt_cnt_0??[!0]",
>>>>
>>>> particle_filename='/home/josh/amuse-svn/src/amuse/community/flash/SinkMomTest2/SMT_hdf5_part_0??[!0]')
>>>>
>>>> times = []
>>>> Mom_x = []
>>>> Mom_y = []
>>>>
>>>> for ds in ts:
>>>>
>>>>
>>>>     try:
>>>>         ds.add_field("total_px", function=_px, units="kg/s/m**2")
>>>>         ds.add_field("total_py", function=_py, units="kg/s/m**2")
>>>>     except:
>>>>         continue
>>>>
>>>>     dd = ds.all_data()
>>>>
>>>>     Mom_x.append(dd["total_px"])
>>>>     Mom_y.append(dd["total_py"])
>>>>     times.append(ds.current_time.in_units("Myr"))
>>>>
>>>> ###########################################################
>>>>
>>>> But this gives the error:
>>>>
>>>>
>>>> ---------------------------------------------------------------------------
>>>> IOError                                   Traceback (most recent call
>>>> last)
>>>> <ipython-input-3-4be7349ef199> in <module>()
>>>>       9 Mom_y = []
>>>>      10
>>>> ---> 11 for ds in ts:
>>>>      12
>>>>      13
>>>>
>>>> /usr/local/lib/python2.7/dist-packages/yt/data_objects/time_series.pyc
>>>> in __iter__(self)
>>>>     158         for o in self._pre_outputs:
>>>>     159             if isinstance(o, types.StringTypes):
>>>> --> 160                 ds = load(o, **self.kwargs)
>>>>     161                 self._setup_function(ds)
>>>>     162                 yield ds
>>>>
>>>> /usr/local/lib/python2.7/dist-packages/yt/convenience.pyc in
>>>> load(*args, **kwargs)
>>>>      78         if c._is_valid(*args, **kwargs): candidates.append(n)
>>>>      79     if len(candidates) == 1:
>>>> ---> 80         return output_type_registry[candidates[0]](*args,
>>>> **kwargs)
>>>>      81     if len(candidates) == 0:
>>>>      82         if ytcfg.get("yt", "enzo_db") != '' \
>>>>
>>>> /usr/local/lib/python2.7/dist-packages/yt/frontends/flash/data_structures.pyc
>>>> in __init__(self, filename, dataset_type, storage_filename,
>>>> particle_filename, units_override)
>>>>     206                 self._particle_handle =
>>>> HDF5FileHandler(self.particle_filename)
>>>>     207             except:
>>>> --> 208                 raise IOError(self.particle_filename)
>>>>     209         # These should be explicitly obtained from the file,
>>>> but for now that
>>>>     210         # will wait until a reorganization of the source tree
>>>> and better
>>>>
>>>> IOError:
>>>> /home/josh/amuse-svn/src/amuse/community/flash/SinkMomTest2/SMT_hdf5_part_0??[!0]
>>>>
>>>
>>> Unfortunately the FLASH frontend is gobbling the error h5py is raising
>>> here.
>>>
>>> A quick way to get the real error from h5py would be to replace "raise
>>> IOError..." on line 208 with just "raise" and then rerun the script.
>>>
>>
>> Ah, but now that I think about it, I don't think the glob pattern should
>> have made it all the way down to the FLASH I/O layer.  Maybe there's a bug
>> in the way we set up time series for FLASH datasets that include particle
>> files?
>>
>>
>>>
>>>
>>>>
>>>> #############################################################
>>>>
>>>> Any ideas?
>>>>
>>>> Joshua Wall
>>>> Drexel University
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> <javascript:_e(%7B%7D,'cvml','yt-users at lists.spacepope.org');>
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> <javascript:_e(%7B%7D,'cvml','yt-users at lists.spacepope.org');>
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> <javascript:_e(%7B%7D,'cvml','yt-users at lists.spacepope.org');>
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20150514/1acd1bc0/attachment.htm>
-------------- next part --------------
_______________________________________________
yt-users mailing list
yt-users at lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


More information about the yt-users mailing list