[yt-users] FLASH Time Series Analysis with Particle Files
Nathan Goldbaum
nathan12343 at gmail.com
Thu Dec 1 07:46:54 PST 2016
Hi Jason,
Thanks for those details.
The FLASH frontend has a capability to automatically detect the presence of
an accompanying particle file for a given plotfile. I have a test dataset
(that I will be uploading to yt-project.org/data shortly for future
questions about flash datasets that contain both particle and mesh fields)
that I've written the following test script for:
import yt
ts = yt.load('fiducial_1to1_b0_hdf5_plt_cnt_*')
for ds in ts:
print(ds.particle_type_counts)
This time series contains five outputs, with a plotfile and a particlefile
for each output:
$ ls
fiducial_1to1_b0_hdf5_part_0000 fiducial_1to1_b0_hdf5_plt_cnt_0001
fiducial_1to1_b0_hdf5_part_0001 fiducial_1to1_b0_hdf5_plt_cnt_0002
fiducial_1to1_b0_hdf5_part_0002 fiducial_1to1_b0_hdf5_plt_cnt_0003
fiducial_1to1_b0_hdf5_part_0003 fiducial_1to1_b0_hdf5_plt_cnt_0004
fiducial_1to1_b0_hdf5_part_0004 test.py
fiducial_1to1_b0_hdf5_plt_cnt_0000
When I run the script, it outputs:
{'io': 9999998}
{'io': 9999998}
{'io': 9999998}
{'io': 9999998}
{'io': 9999998}
As you can see, my script only explicitly loads the plotfile but yt still
recognizes that there are about 10 million particles in each ouput, since
the particle file for each plotfile is automatically associated with the
plotfile. You can see how this functionality is setup in the FLASH frontend
here:
https://bitbucket.org/yt_analysis/yt/src/yt/yt/frontends/flash/data_structures.py?#data_structures.py-202
This requires that the particle file associated with a plotfile has the
same output number. We used to strictly enforce that the particle file has
the same simulation time as the plotifle, but this ended up being too
strict for real-world use cases so now we only print a warning (see around
like 217 in the file I linked to above).
Is this sufficient for your use cases or do you need to explicitly specify
the name of the particle file every time you load the dataset? If so, one
hacky way around this would be to manually associate the particle file with
each plotfile using the `setup_function` keyword argument for the
DatasetSeries object. I could also look into making it so that if you pass
a glob pattern as the `particle_filename` keyword argument to the time
series object, it does the right thing, but I want to make sure that the
existing functionality will not work for you before trying that.
Hope that clears things up,
Nathan
On Wed, Nov 30, 2016 at 1:47 PM, Jason Galyardt <jason.galyardt at gmail.com>
wrote:
> Hi John & Nathan,
>
> Yes, I'm having trouble using the time series analysis machinery with my
> FLASH simulations. I've checked that the plot files and particle files all
> have the same current_time variable, so there's no issue there. I'm using a
> development version of yt with a changeset ID of 5cf30893afdf. This
> changeset includes Nathan's changes supporting cut_region functionality for
> FLASH plot files. Here's what I see when I try to do a time series
> analysis:
>
> ###########################
>
> In [9]: ts = yt.load("mysim_hdf5_plt_cnt_*",
> particle_filename="mysim_hdf5_part_*")
>
> In [10]: for ds in ts:
> print ds.current_time
> ....:
> ------------------------------------------------------------
> ---------------
> IOError Traceback (most recent call
> last)
> <ipython-input-10-fccd01bb5065> in <module>()
> ----> 1 for ds in ts:
> 2 print ds.current_time
> 3
>
> /home/emrys/yt/yt/data_objects/time_series.pyc in __iter__(self)
> 179 for o in self._pre_outputs:
> 180 if isinstance(o, string_types):
> --> 181 ds = self._load(o, **self.kwargs)
> 182 self._setup_function(ds)
> 183 yield ds
>
> /home/emrys/yt/yt/data_objects/time_series.pyc in _load(self, output_fn,
> **kwargs)
> 396 elif self._mixed_dataset_types:
> 397 return load(output_fn, **kwargs)
> --> 398 ds = load(output_fn, **kwargs)
> 399 self._dataset_cls = ds.__class__
> 400 return ds
>
> /home/emrys/yt/yt/convenience.pyc in load(*args, **kwargs)
> 84 candidates = find_lowest_subclasses(candidates)
> 85 if len(candidates) == 1:
> ---> 86 return candidates[0](*args, **kwargs)
> 87 if len(candidates) == 0:
> 88 if ytcfg.get("yt", "enzo_db") != '' \
>
> /home/emrys/yt/yt/frontends/flash/data_structures.pyc in __init__(self,
> filename, dataset_type, storage_filename, particle_filename,
> units_override, unit_system)
> 209 self._particle_handle =
> HDF5FileHandler(self.particle_filename)
> 210 except:
> --> 211 raise IOError(self.particle_filename)
> 212 # Check if the particle file has the same time
> 213 if self._particle_handle != self._handle:
>
> IOError: mysim_hdf5_part_*
>
> ###########################
>
> It looks to me like HDF5FileHandler() just does not like the wildcard.
> I've also tried, unsuccessfully, to feed a list of particle files to
> yt.load(). I thought perhaps that I should be using the DatasetSeries
> framework directly, but I'm not clear on how to pass in the list of
> particle files, as DatasetSeries does not understand the particle_filename
> keyword argument.
>
> Any ideas?
>
> Thanks,
> Jason
>
>
> On Wed, Nov 30, 2016 at 1:57 PM, <yt-users-request at lists.spacepope.org>
> wrote:
>
>> Send yt-users mailing list submissions to
>> yt-users at lists.spacepope.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> or, via email, send a message with subject or body 'help' to
>> yt-users-request at lists.spacepope.org
>>
>> You can reach the person managing the list at
>> yt-users-owner at lists.spacepope.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of yt-users digest..."
>>
>> Today's Topics:
>>
>> 1. FLASH Time Series Analysis with Particle Files (Jason Galyardt)
>> 2. Re: FLASH Time Series Analysis with Particle Files
>> (Nathan Goldbaum)
>> 3. Re: FLASH Time Series Analysis with Particle Files (John Zuhone)
>>
>>
>> ---------- Forwarded message ----------
>> From: Jason Galyardt <jason.galyardt at gmail.com>
>> To: Discussion of the yt analysis package <yt-users at lists.spacepope.org>
>> Cc:
>> Date: Wed, 30 Nov 2016 10:16:44 -0500
>> Subject: [yt-users] FLASH Time Series Analysis with Particle Files
>> Dear yt-users,
>>
>> In May 2015, there was some traffic on this list concerning errors in
>> time series analysis on FLASH simulations that include both plot files and
>> particle files. Does anyone have an update on the status of this issue?
>>
>> Thanks,
>> Jason
>>
>> ------
>> Jason Galyardt
>> University of Georgia
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Nathan Goldbaum <nathan12343 at gmail.com>
>> To: Discussion of the yt analysis package <yt-users at lists.spacepope.org>
>> Cc:
>> Date: Wed, 30 Nov 2016 09:32:37 -0600
>> Subject: Re: [yt-users] FLASH Time Series Analysis with Particle Files
>> Hi Jason,
>>
>> As far as I'm aware this should be working now. Are you having trouble?
>>
>> -Nathan
>>
>> On Wed, Nov 30, 2016 at 9:16 AM, Jason Galyardt <jason.galyardt at gmail.com
>> > wrote:
>>
>>> Dear yt-users,
>>>
>>> In May 2015, there was some traffic on this list concerning errors in
>>> time series analysis on FLASH simulations that include both plot files and
>>> particle files. Does anyone have an update on the status of this issue?
>>>
>>> Thanks,
>>> Jason
>>>
>>> ------
>>> Jason Galyardt
>>> University of Georgia
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>>
>> ---------- Forwarded message ----------
>> From: John Zuhone <jzuhone at gmail.com>
>> To: Discussion of the yt analysis package <yt-users at lists.spacepope.org>
>> Cc:
>> Date: Wed, 30 Nov 2016 11:32:09 -0500
>> Subject: Re: [yt-users] FLASH Time Series Analysis with Particle Files
>> Yes, my understanding is that this is working, provided that your files
>> have the same basename and the times of the files are the same.
>>
>> On Nov 30, 2016, at 10:32 AM, Nathan Goldbaum <nathan12343 at gmail.com>
>> wrote:
>>
>> Hi Jason,
>>
>> As far as I'm aware this should be working now. Are you having trouble?
>>
>> -Nathan
>>
>> On Wed, Nov 30, 2016 at 9:16 AM, Jason Galyardt <jason.galyardt at gmail.com
>> > wrote:
>>
>>> Dear yt-users,
>>>
>>> In May 2015, there was some traffic on this list concerning errors in
>>> time series analysis on FLASH simulations that include both plot files and
>>> particle files. Does anyone have an update on the status of this issue?
>>>
>>> Thanks,
>>> Jason
>>>
>>> ------
>>> Jason Galyardt
>>> University of Georgia
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20161201/607a2b6e/attachment-0001.htm>
More information about the yt-users
mailing list