[yt-users] Flash Particle files and DatasetSeries

Mon Mar 7 07:03:54 PST 2016

Hi all,

Just wanted to jump in here—as yt’s FLASH guy, I’m definitely following this discussion, but I’ve been on call this past week so I’ve not been able to really weigh in. I’ll be looking over this PR and implementation later today.

Best,

John

> On Mar 7, 2016, at 9:35 AM, Joshua Wall <joshua.e.wall at gmail.com> wrote:
> 
> Alex and Nathan,
> 
>      Alright, I also put this link on the pull request, but here's my idea for checking to see if both files are from the same timestep in Flash (I added it to Alex's paste code):
> 
> http://paste.yt-project.org/show/6298/ <http://paste.yt-project.org/show/6298/>
> 
>      I did a quick test just to see if it properly reads the times from a single set of plot and particle files on my end and it worked, but I haven't tested it in DataSeries yet. Just thought I'd post it real quick in case Alex wanted to drop it in his test. Also not sure that two if statements is the fastest, there might be a more elegant solution with the same information from part_file.handle.get('real scalars')[0] (which is where sim time lives in the particle file structure). If no one else gets any further I should have some time later today or tomorrow to test it out.
> 
> Cordially,
> 
> Josh
> 
> On Sun, Mar 6, 2016 at 1:52 PM Alex Hill <ashill at haverford.edu <mailto:ashill at haverford.edu>> wrote:
> Hi Josh and Nathan,
> 
> I’ve found a straightforward way to associate a particle file with each element in a DatasetSeries after creating the dataset: <http://paste.yt-project.org/show/6297/ <http://paste.yt-project.org/show/6297/>>
> 
> This is based on Yi-Hao Chen’s suggestion here: <https://bitbucket.org/yt_analysis/yt/issues/1179/loading-particle-files-in-flash-with <https://bitbucket.org/yt_analysis/yt/issues/1179/loading-particle-files-in-flash-with>>
> 
> This approach requires a bit of care before implementing as an option in the yt package since it assumes that plt_cnt_1234 and part_1234 are the same timestep, which isn’t necessarily true. (That is, checking for the existence of part_1234 isn’t good enough.) But it’s obviously fine when I’m in control of the data and know that the plot files and particle files were output simultaneously.
> 
> Cheers,
>  Alex
> 
> > On Mar 3, 2016, at 13:19, Alex Hill <ashill at haverford.edu <mailto:ashill at haverford.edu>> wrote:
> >
> > Hi Josh,
> >
> > Yeah, this is the serial kludge that I’m now using, which works for my purposes: <http://paste.yt-project.org/show/6289/ <http://paste.yt-project.org/show/6289/>>. Sounds more or less like what you’re doing.
> >
> > I could help working on a proper implementation, but it’s probably not my first priority for yt-related development at the moment.
> >
> > Cheers,
> > Alex
> >
> > ---------
> > Alex Hill
> > Senior Postdoctoral Research Associate
> > Office: KINSC Link L106
> > Department of Astronomy, Haverford College
> > 370 Lancaster Ave, Haverford, PA 19041 USA
> > phone: +1 484 297 2136
> > email/iMessage: ashill at haverford.edu <mailto:ashill at haverford.edu>
> >
> >> On Mar 3, 2016, at 12:51, Joshua Wall <joshua.e.wall at gmail.com <mailto:joshua.e.wall at gmail.com>> wrote:
> >>
> >> Dear Alex and Nathan,
> >>
> >>     I am also interested in this functionality. Initially I did some poking around trying to see how hard it might be, and decided the same as Nathan (not simple). I ended up using glob on the plot and particle file sets and then zipping them and using python's pool to analyze my data in parallel. If that might be of some use to you Alex, I'd be happy to share the code. Also I'm happy to try to take another look at this (or even just provide some test data if needed) as far as trying to implement the functionality, even though my understanding of yt (and matplotlib) under the hood is rather shallow and it therefore might take me some time to sort it out.
> >>
> >> Cordially,
> >>
> >> Josh
> >>
> >>
> >> On Thu, Mar 3, 2016 at 11:21 AM Nathan Goldbaum <nathan12343 at gmail.com <mailto:nathan12343 at gmail.com>> wrote:
> >> Hi Alex,
> >>
> >> Looking at the DatasetSeries code, I don't think there's an easy way to do this right now.
> >>
> >> It would work if you could use the setup_function argument when you create the DatasetSeries, but I don't think there's a way to associate a particle filename with a dataset *after* it's been loaded.
> >>
> >> I see two ways forward here:
> >>
> >> 1. Make keyword arguments passed to the DatasetSeries initializer get passed to `load` when DatasetSeries invokes loads each individual dataset internally.
> >>
> >> 2. Make it possible to associate a particle file with a FLASH dataset after the dataset has been loaded.
> >>
> >> I think the first option would be more generally useful, but option 2 might be useful for some FLASH users as well.
> >>
> >> Would you be interested in trying to get this to work? If not I can file an issue and try to get to it sometime soon...
> >>
> >> -Nathan
> >>
> >> On Thu, Mar 3, 2016 at 10:13 AM, Alex Hill <ashill at haverford.edu <mailto:ashill at haverford.edu>> wrote:
> >> Hi,
> >>
> >> I’d like to load particle files with a series of Flash files.
> >>
> >> If loading a single file, I can do
> >>
> >> d = yt.load(‘file_hdf5_plt_cnt_0100’, particle_filename=‘file_hdf5_part_0100’)
> >>
> >> I’d like to do something like
> >>
> >> ts = yt.DatasetSeries(‘file_hdf5_plt_cnt_0*0’, particle_filename=‘file_hdf5_part_0*0’)
> >>
> >> or load the particle files after the fact while iterating through the time series. Is this possible?
> >>
> >> Cheers,
> >> Alex
> >>
> >> ---------
> >> Alex Hill
> >> Senior Postdoctoral Research Associate
> >> Office: KINSC Link L106
> >> Department of Astronomy, Haverford College
> >> 370 Lancaster Ave, Haverford, PA 19041 USA
> >> phone: +1 484 297 2136
> >> email/iMessage: ashill at haverford.edu <mailto:ashill at haverford.edu>
> >>
> >>
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
> >>
> >>
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
> >> --
> >> Joshua Wall
> >> Doctoral Candidate
> >> Department of Physics
> >> Drexel University
> >> 3141 Chestnut Street
> >> Philadelphia, PA 19104
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
> >
> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
> -- 
> Joshua Wall
> Doctoral Candidate
> Department of Physics
> Drexel University
> 3141 Chestnut Street
> Philadelphia, PA 19104
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160307/ab237014/attachment.html>