[yt-users] loading eagle data using yt

[Britton Smith]

Hi Charlotte,

It looks like the support for Eagle data in yt is currently pretty thin,
but this could be improved with a little effort.  For the first part of
your question, what is missing is just a bit of code for the Eagle frontend
to look for these fields on disk and set them up in a similar fashion to
how it's done in the Owls frontend.  This is probably a day's worth of work
that I could help out with, but not for another 2-3 weeks.  However, if
you're interested in taking this on, I think you could do it and I would be
happy to provide some guidance.

I believe the second issue relating to your dataset only having one file is
actually a simple bug.  We don't have any testing data for this format with
only one file.  Would you be able to provide a smallish (<1GB) single-file
dataset for us to use for testing?  If so, I can probably fix this bug
quickly.

As to your last question, I'm not sure exactly what you're asking, but yt
currently needs to create an octree for particle datasets before it can do
anything with them.  For datasets of the size you're dealing with, this
takes a while.  However, Nathan Goldbaum and others are currently working
on a completely different way of handling particle data in yt that should
take care of this issue.  I'm not sure what the timeline on that is.

Britton

On Sat, Feb 18, 2017 at 1:00 AM, Charlotte Brand <charlottemarlijn at gmail.com
> wrote:

> Hi yt users!
>
> For my master research, I'm working on virtual observations from the EAGLE
> (gadget based) simulations. Currently I'm trying out pyXSIM for this, which
> uses yt to load the data, but I'm having some problems.
>
> First, yt only loads the non-smoothed element abundances, as fields
> "PartType0, (element)", while the eagle data has 2 fields:
> PartType0/SmoothedElementAbundance/(element) and
> PartType0/ElementAbundance/(element). Does anyone have any idea on how to
> get the smoothed abundances out?
>
> Also, I'm wondering: I have a script that selects the subhalos you want
> from the eagle data, and then saves the particle data you need
> (temperature, density, etc.) of only the particles in those subhalos to a
> hdf5 file. However, when I try to load this data with yt, it recognises it
> as gadget data, and then expects more than one file to be present, I guess
> because usually there is more than one snapshot file. E.g. if my file is
> called partialdata.0.hdf5 yt starts to look for partialdata.1.hdf5 (if i
> just call it partialdata.hdf5 it still looks for partialdata.0.hdf5).
> However, in gadget, the number of output files can be chosen freely, so
> you could also choose to save your entire snapshot in one hdf5 file. So it
> seems that in yt there should be a way to specify the number of files
> you're dealing with... Does anyone know how to solve this problem? I think
> that yt recognises the eagle files as owls data (they are very similar), it
> may have something to do with that.
> The reason I want to select my subhalos before loading the data is that
> when I load the entire dataset (a 100Mpc box, 1504^3 particles) and then
> select e.g. a slice, when creating a derived field, I think that yt
> allocates memory (or something like that) for it for the entire dataset,
> and not just for the slice I have selected. Is this true?
>
> Cheers and thanks in advance,
> Charlotte Brand
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20170220/c09f7960/attachment-0001.html>