[yt-users] Gadget blocks' configuration

Britton Smith brittonsmith at gmail.com
Wed Sep 28 06:08:59 PDT 2016


Hi Bernhard,

I'm just starting to look at this now.  I haven't looked to closely yet,
but I noticed an issue with your _metal_mass field function.  Right now,
you've got:

        def _metal_mass(field, data):
            return sum( data[(ptype, "%s_mass"%el)]
                    for el in self.nuclei_names if el not in ["H","He"] )

The sum function is going to reduce the entire field array to a single
number.  I think you need something like this:

field_data = np.zeros_like(data[ptype, "particle_mass"])
for el in self.nuclei_names if el not in ["H","He"]:
    field_data += data[ptype, "%s_mass"% el]

The metallicity field is likely not showing up because it is quietly
erroring.  This may fix that.  Let us know how it goes.

Britton

On Wed, Sep 28, 2016 at 1:44 PM, Bernhard Röttgers <
broett at mpa-garching.mpg.de> wrote:

> Hi there,
>
> I still have problems with the “gas”. I can access (“PartType0”,
> “metallicity”) but not (“gas”, “metallicity”). Why is the alias not being
> created automatically as I understood it should?
>
> Cheers,
> Bernhard
>
> On 26 Sep 2016, at 20:36, Britton Smith <brittonsmith at gmail.com> wrote:
>
> Hi Bernhard,
>
> The "gas" alias points to a version of the fields that has been deposited
> on the octtree.  For the ion-specific fields, Trident will take care of
> adding this for you as long as the element fields are added correctly.  I
> am at the end of my day here, but I can take a look at the dataset you've
> uploaded tomorrow.
>
> On Mon, Sep 26, 2016 at 6:51 PM, Bernhard Röttgers <
> broett at mpa-garching.mpg.de> wrote:
>
>> Got the new fields. I had a little typo in the names.
>> Couldn’t  solve the problems with the unit system though.
>> And I have the additional problem, that I need to define a particle group
>> “gas” for trident. How do I do this?
>>
>> Cheers,
>> Bernhard
>>
>> On 26 Sep 2016, at 18:55, Bernhard Röttgers <broett at MPA-Garching.MPG.DE
>> <broett at mpa-garching.mpg.de>> wrote:
>>
>> Okay, so the (HDF5) snapshot is now available at:
>> http://use.yt/upload/09111d2e
>>
>> I used the FIRE frontend by Britton (https://bitbucket.org/britton
>> smith/yt_fire) as a basis to create my own:
>> https://bitbucket.org/broett/my_yt/overview (it’s git, not hg!)
>>
>> I can load a snapshot with the front-end, but I still don’t have the
>> self-defined fields. Meaning that after
>>
>> ds = my_yt.MyGadgetDataset(‘path/to/snapshot/snap_M0664_4x_940.hdf5',
>> unit_base=…, bounding_box=…)
>> ds.index
>> ad = ds.all_data()
>>
>> the following fails:
>> ad[('PartType0','metallicity’)]
>>
>> I also cannot use self.ds.unit_system["code_metallicity”] in class
>> MyGadgetFieldInfo(GadgetFieldInfo) as Britton did.
>>
>>
>> Cheers,
>> Bernhard
>>
>>
>>
>> On 26 Sep 2016, at 18:10, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
>>
>>
>>
>> On Mon, Sep 26, 2016 at 11:07 AM, Bernhard Röttgers <
>> broett at mpa-garching.mpg.de> wrote:
>>
>>> Okay, then I upload HDF5 for now. Called the file
>>> “snap_M6782_4x_470.hdf5”.
>>> So what are we gonna do with it?
>>>
>>>
>> You can upload it using the yt curldrop:
>>
>> $ curl -T snap_M6782_4x_470.hdf5 http://use.yt/upload/
>>
>> That will print out a URL once the upload is finished, just share that
>> URL here.
>>
>> Alternatively generating a share link via google drive or dropbox will
>> also work.
>>
>>
>>> Oops… I mixed up the field info and the data structure classes. My bad!
>>>
>>> Cheers,
>>> Bernhard
>>>
>>>
>>> On 26 Sep 2016, at 17:55, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
>>>
>>>
>>>
>>> On Mon, Sep 26, 2016 at 10:42 AM, Bernhard Röttgers <
>>> broett at mpa-garching.mpg.de> wrote:
>>>
>>>> Hi Nathan,
>>>>
>>>> Yes, I have some smaller (~85MB) sample I could upload. I guess it
>>>> should ideally be HDF5? (Originally it is format 2, but I can easily
>>>> convert.) What naming should I use?
>>>> I would guess, my format is pretty much “the standard” except for the
>>>> metallicity block (which in fact is element masses for 12 different
>>>> elements).
>>>>
>>>
>>> The naming and format doesn't matter too much. HDF5 is probably easiest
>>> to deal with, although yt supports both binary formats as well.
>>>
>>>
>>>>
>>>> So again, how am I supposed the data set class I derive from
>>>> GadgetHDF5Dataset? Simply replacing `yt.load` yields an exception
>>>> complaining that the `bounding_box` keyword argument is unexpected.
>>>>
>>>
>>> I think all you need to do is declare the subclass and then load your
>>> dataset directly:
>>>
>>> my_ds = MyGadgetHDF5DatasetSubclass(unit_base=..., bbox=...)
>>>
>>> It should also be possible to make your subclass work with yt.load()
>>> (i'm not sure how to do that offhand as I've never tried to add a new
>>> Dataset subclass outside of yt), but instantiating your subclass directly
>>> should be fine for now.
>>>
>>>
>>>>
>>>> Cheers,
>>>> Bernhard
>>>>
>>>> On 26 Sep 2016, at 17:24, Nathan Goldbaum <nathan12343 at gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>> On Mon, Sep 26, 2016 at 10:15 AM, Bernhard Röttgers <
>>>> broett at mpa-garching.mpg.de> wrote:
>>>>
>>>>> Hi Britton,
>>>>>
>>>>> thanks for the answer and sorry for my late answer!
>>>>>
>>>>> No, I don’t have OWLS/EAGLE/Gizmo snapshots. I actually just wanted to
>>>>> compare my results with yt/trident, but your solution does not seem to be
>>>>> too complicated. How, am I supposed to use it? (Sorry for asking dump
>>>>> questions. I’m not familiar with yt.) I understood that I can load a Gadget
>>>>> snapshot with like:
>>>>>
>>>>> ds = yt.load(YT_FNAME, unit_base=UNIT_BASE, bounding_box=bbox)
>>>>> ds.index
>>>>>
>>>>> The data structure `ds` then is what is passed to trident. Where does
>>>>> the new frontend come into play?
>>>>>
>>>>
>>>> Right now yt makes certain assumptions about the structure and meaning
>>>> of data in Gadget output files. The assumptions currently used in yt are
>>>> based on the public data the yt developers have access to (e.g. the
>>>> datasets on yt-project.org/data) and are only valid in so far as those
>>>> datasets are representative of Gadget data in general. Due to the history
>>>> of the Gadget's code, there are in reality many different flavors of Gadget
>>>> that make different assumptions about what the data they write to disk
>>>> means, and it seems yt doesn't currently make incorrect assumptions for
>>>> your flavor of Gadget data.
>>>>
>>>> Do you happen to have a test dataset that you can share? In principle
>>>> the necessary modification should be simple, and having a dataset to test
>>>> with would be helpful. One way to share files with us is to use the yt
>>>> curldrop:
>>>>
>>>> https://docs.hub.yt/services.html#curldrop
>>>>
>>>>
>>>>>
>>>>> And some more questions:
>>>>> What exactly are the code metallicity units? In my case yt says its in
>>>>> units of 1. Is this interpreted as absolute units or in solar metallicity?
>>>>>
>>>>
>>>> In units of solar metallicity. There is also the metal_density, which
>>>> should have units of g/cm**3 (by default, if may be different if you
>>>> specified a value for the `unit_system` keyword when you called the load()
>>>> function.
>>>>
>>>>
>>>>>
>>>>> Cheers,
>>>>> Bernhard
>>>>>
>>>>>
>>>>> On 19 Sep 2016, at 15:04, Britton Smith <brittonsmith at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Bernhard,
>>>>>
>>>>> Does your data come from a specific flavor of Gadget that is supported
>>>>> by yt?  For example, we currently support OWLS, EAGLE, and Gizmo data that
>>>>> have fields for individual elements.  If not, then it is probably best to
>>>>> try and put together a new frontend for this data.  If the type of data
>>>>> that you're working with is not public or widely used, the best thing may
>>>>> be to create a frontend that exists as a yt extension (an external module
>>>>> not inside the main yt codebase).  Here is an example of one that I created
>>>>> a while back for the FIRE simulation data:
>>>>> https://bitbucket.org/brittonsmith/yt_fire
>>>>>
>>>>> If the only thing that is different about your data is the way that
>>>>> the metallicity fields are defined, then it shouldn't be too difficult to
>>>>> create a subclass of an existing Gadget frontend that overrides the field
>>>>> definitions.  Please, let us know if you're interested in pursuing this and
>>>>> we can work with you.
>>>>>
>>>>> Britton
>>>>>
>>>>> On Sat, Sep 17, 2016 at 11:45 AM, Bernhard Röttgers <
>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>
>>>>>> Hello!
>>>>>>
>>>>>> I am trying to use trident
>>>>>> <https://bitbucket.org/trident-project/trident> (yt-based code) to
>>>>>> generate spectra out of my Gadget simulations (stored in HDF5). I could get
>>>>>> the code running, but the spectra are scaled weirdly. I was able to track
>>>>>> the problem down to a weirdly scaled metallicity block within yt. I am
>>>>>> guessing that the issue is related to the fact, that my block “Metallicity”
>>>>>> is in fact element masses for individual elements, i.e. for each particle I
>>>>>> have an array of masses for the 12 elements He, C, Mg, O, Fe, Si, H, N, Ne,
>>>>>> S, Ca, and the rest (I know, a crappy naming for such a block, but that’s
>>>>>> the way it is).
>>>>>>
>>>>>> Is there a way to force yt to create the metallicity block correctly?.
>>>>>>
>>>>>> Cheers,
>>>>>> Bernhard
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160928/92245c5b/attachment.html>


More information about the yt-users mailing list