[yt-users] Gadget blocks' configuration

Nathan Goldbaum nathan12343 at gmail.com
Wed Sep 28 08:28:48 PDT 2016


On Wed, Sep 28, 2016 at 10:22 AM, Britton Smith <brittonsmith at gmail.com>
wrote:

> Hi Bernhard,
>
> Yes, I think I agree with you, but I'm not 100% sure.  If someone knows of
> an instance where that might happen, please speak up.  You should be able
> to pass units of "" (i.e., no units).  It looks like you're using
> "code_metallicity", which looks to be the same thing.  "Zsun" should be
> defined, so you can do:
> ds.r["gas", "metallicity"].to("Zsun")
> to see things in solar.  The metallicities will be in linear units.
>
> Does anyone else have any ideas?
>
> Britton
>
> On Wed, Sep 28, 2016 at 3:56 PM, Bernhard Röttgers <
> broett at mpa-garching.mpg.de> wrote:
>
>> Hi Britton,
>>
>> thanks a lot! I finally can access the metallicity block :)
>> However, I am stilled puzzled by the result: The maximum value I have for
>> the particles is ~0.08, but the maximum value I see in yt is ~0.10. That
>> seems be to wrong. Regardless of the details of the mapping onto a grid, I
>> should never get larger metallicities than for the individual particles,
>> right?
>>
>
Hi Bernhard,

Can you share the code snippet that generates that result?

-Nathan


>
>> And what units do I have to pass? Did I correctly understand that yt
>> always assumes the metallicities to be in solar metallicities? And is it in
>> linear units or logarithmic units?
>>
>> Cheers,
>> Bernhard
>>
>> On 28 Sep 2016, at 16:03, Britton Smith <brittonsmith at gmail.com> wrote:
>>
>> Hi Bernhard,
>>
>> Sorry, the previous suggestion I gave you was completely wrong.
>>
>> The issue is that the step where the ("gas", "metallicity") field is
>> being created is happening before your setup_gas_particle_fields function
>> is getting called.  The field never gets created because the ("PartType0",
>> "metallicity") field does not yet exist.  I think yt is expecting it to be
>> there because "metallicity" is typically aliased to a specific field on
>> disk and that step happens earlier on.  Some rearranging of the order or
>> operations might make this possible, but this simplest solution is to
>> directly add the ("gas", "metallicity") field in your
>> setup_gas_particle_fields.  If did it in the following way and it worked
>> for me:
>>
>> from yt.fields.particle_fields import \
>>     add_volume_weighted_smoothed_field
>>
>> # end of setup_gas_particle_fields
>>         num_neighbors = 64
>>         fn = add_volume_weighted_smoothed_field(
>>             ptype, "particle_position", "particle_mass",
>>             "smoothing_length", "density", "metallicity",
>>             self, num_neighbors)
>>         self.alias(("gas", "metallicity"), fn[0])
>>
>> Britton
>>
>> On Wed, Sep 28, 2016 at 2:44 PM, Bernhard Röttgers <
>> broett at mpa-garching.mpg.de> wrote:
>>
>>> Never mind! `sum` also accepts generator expressions
>>> <https://www.python.org/dev/peps/pep-0289/>. So in my little example
>>> the last statement is equivalent with a+b+c. Since the addition of
>>> numpy arrays
>>> <http://docs.scipy.org/doc/numpy/reference/generated/numpy.add.html> of
>>> the same shape is defined as an element-wise operation, we end up with the
>>> intended result.
>>>
>>> The thing that does not work is the for loop with the if-statement. This
>>> combination is only allowed in list comprehension and generator expressions.
>>>
>>> Cheers,
>>> Bernhard
>>>
>>>
>>> On 28 Sep 2016, at 15:34, Britton Smith <brittonsmith at gmail.com> wrote:
>>>
>>> Hi Bernhard,
>>>
>>> My apologies, I was not familiar with that syntax.  yt arrays will
>>> behave like NumPy arrays, so if that works as in your example, it should
>>> work in that function.  Can you elaborate at why my suggestion doesn't work
>>> in Python 2.7?  I will try to debug this here.
>>>
>>> Britton
>>>
>>> On Wed, Sep 28, 2016 at 2:29 PM, Bernhard Röttgers <
>>> broett at mpa-garching.mpg.de> wrote:
>>>
>>>> PS:
>>>> I also cannot access (‘PartType0’,’metallicity’) including you attempt
>>>> to fix.
>>>>
>>>> On 28 Sep 2016, at 15:27, Bernhard Röttgers <broett at MPA-Garching.MPG.DE
>>>> <broett at mpa-garching.mpg.de>> wrote:
>>>>
>>>> Hi Britton,
>>>>
>>>> thanks for spotting this error! So that means yt arrays do not behave
>>>> like numpy arrays, since:
>>>> In [2]: a = np.arange(10)
>>>>
>>>> In [3]: b = np.arange(10)
>>>>
>>>> In [4]: c = np.arange(10)
>>>>
>>>> In [5]: sum( x for x in [a,b,c] )
>>>> Out[5]: array([ 0,  3,  6,  9, 12, 15, 18, 21, 24, 27])
>>>> Which is what I had in mind, but apparently does not happen with yt
>>>> arrays.
>>>>
>>>> However, you proposed code seem to have problems, too. First of all the
>>>> loop you propose is not Python 2.7, which I use. And second, even after
>>>> fixing that, the problem is not solved. Actually even the particle_mass
>>>> block seems to be broken somehow. I cannot access it by (“gas”,
>>>> “particle_mass”) but only via (“partType0”, “particle_mass”).
>>>>
>>>> Cheers,
>>>> Bernhard
>>>>
>>>>
>>>> On 28 Sep 2016, at 15:08, Britton Smith <brittonsmith at gmail.com> wrote:
>>>>
>>>> Hi Bernhard,
>>>>
>>>> I'm just starting to look at this now.  I haven't looked to closely
>>>> yet, but I noticed an issue with your _metal_mass field function.  Right
>>>> now, you've got:
>>>>
>>>>         def _metal_mass(field, data):
>>>>             return sum( data[(ptype, "%s_mass"%el)]
>>>>                     for el in self.nuclei_names if el not in ["H","He"]
>>>> )
>>>>
>>>> The sum function is going to reduce the entire field array to a single
>>>> number.  I think you need something like this:
>>>>
>>>> field_data = np.zeros_like(data[ptype, "particle_mass"])
>>>> for el in self.nuclei_names if el not in ["H","He"]:
>>>>     field_data += data[ptype, "%s_mass"% el]
>>>>
>>>> The metallicity field is likely not showing up because it is quietly
>>>> erroring.  This may fix that.  Let us know how it goes.
>>>>
>>>> Britton
>>>>
>>>> On Wed, Sep 28, 2016 at 1:44 PM, Bernhard Röttgers <
>>>> broett at mpa-garching.mpg.de> wrote:
>>>>
>>>>> Hi there,
>>>>>
>>>>> I still have problems with the “gas”. I can access (“PartType0”,
>>>>> “metallicity”) but not (“gas”, “metallicity”). Why is the alias not being
>>>>> created automatically as I understood it should?
>>>>>
>>>>> Cheers,
>>>>> Bernhard
>>>>>
>>>>> On 26 Sep 2016, at 20:36, Britton Smith <brittonsmith at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Bernhard,
>>>>>
>>>>> The "gas" alias points to a version of the fields that has been
>>>>> deposited on the octtree.  For the ion-specific fields, Trident will take
>>>>> care of adding this for you as long as the element fields are added
>>>>> correctly.  I am at the end of my day here, but I can take a look at the
>>>>> dataset you've uploaded tomorrow.
>>>>>
>>>>> On Mon, Sep 26, 2016 at 6:51 PM, Bernhard Röttgers <
>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>
>>>>>> Got the new fields. I had a little typo in the names.
>>>>>> Couldn’t  solve the problems with the unit system though.
>>>>>> And I have the additional problem, that I need to define a particle
>>>>>> group “gas” for trident. How do I do this?
>>>>>>
>>>>>> Cheers,
>>>>>> Bernhard
>>>>>>
>>>>>> On 26 Sep 2016, at 18:55, Bernhard Röttgers <
>>>>>> broett at MPA-Garching.MPG.DE <broett at mpa-garching.mpg.de>> wrote:
>>>>>>
>>>>>> Okay, so the (HDF5) snapshot is now available at:
>>>>>> http://use.yt/upload/09111d2e
>>>>>>
>>>>>> I used the FIRE frontend by Britton (https://bitbucket.org/britton
>>>>>> smith/yt_fire) as a basis to create my own:
>>>>>> https://bitbucket.org/broett/my_yt/overview (it’s git, not hg!)
>>>>>>
>>>>>> I can load a snapshot with the front-end, but I still don’t have the
>>>>>> self-defined fields. Meaning that after
>>>>>>
>>>>>> ds = my_yt.MyGadgetDataset(‘path/to/snapshot/snap_M0664_4x_940.hdf5',
>>>>>> unit_base=…, bounding_box=…)
>>>>>> ds.index
>>>>>> ad = ds.all_data()
>>>>>>
>>>>>> the following fails:
>>>>>> ad[('PartType0','metallicity’)]
>>>>>>
>>>>>> I also cannot use self.ds.unit_system["code_metallicity”] in class
>>>>>> MyGadgetFieldInfo(GadgetFieldInfo) as Britton did.
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Bernhard
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 26 Sep 2016, at 18:10, Nathan Goldbaum <nathan12343 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Sep 26, 2016 at 11:07 AM, Bernhard Röttgers <
>>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>>
>>>>>>> Okay, then I upload HDF5 for now. Called the file
>>>>>>> “snap_M6782_4x_470.hdf5”.
>>>>>>> So what are we gonna do with it?
>>>>>>>
>>>>>>>
>>>>>> You can upload it using the yt curldrop:
>>>>>>
>>>>>> $ curl -T snap_M6782_4x_470.hdf5 http://use.yt/upload/
>>>>>>
>>>>>> That will print out a URL once the upload is finished, just share
>>>>>> that URL here.
>>>>>>
>>>>>> Alternatively generating a share link via google drive or dropbox
>>>>>> will also work.
>>>>>>
>>>>>>
>>>>>>> Oops… I mixed up the field info and the data structure classes. My
>>>>>>> bad!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Bernhard
>>>>>>>
>>>>>>>
>>>>>>> On 26 Sep 2016, at 17:55, Nathan Goldbaum <nathan12343 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Sep 26, 2016 at 10:42 AM, Bernhard Röttgers <
>>>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>>>
>>>>>>>> Hi Nathan,
>>>>>>>>
>>>>>>>> Yes, I have some smaller (~85MB) sample I could upload. I guess it
>>>>>>>> should ideally be HDF5? (Originally it is format 2, but I can easily
>>>>>>>> convert.) What naming should I use?
>>>>>>>> I would guess, my format is pretty much “the standard” except for
>>>>>>>> the metallicity block (which in fact is element masses for 12 different
>>>>>>>> elements).
>>>>>>>>
>>>>>>>
>>>>>>> The naming and format doesn't matter too much. HDF5 is probably
>>>>>>> easiest to deal with, although yt supports both binary formats as well.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> So again, how am I supposed the data set class I derive from
>>>>>>>> GadgetHDF5Dataset? Simply replacing `yt.load` yields an exception
>>>>>>>> complaining that the `bounding_box` keyword argument is unexpected.
>>>>>>>>
>>>>>>>
>>>>>>> I think all you need to do is declare the subclass and then load
>>>>>>> your dataset directly:
>>>>>>>
>>>>>>> my_ds = MyGadgetHDF5DatasetSubclass(unit_base=..., bbox=...)
>>>>>>>
>>>>>>> It should also be possible to make your subclass work with yt.load()
>>>>>>> (i'm not sure how to do that offhand as I've never tried to add a new
>>>>>>> Dataset subclass outside of yt), but instantiating your subclass directly
>>>>>>> should be fine for now.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Bernhard
>>>>>>>>
>>>>>>>> On 26 Sep 2016, at 17:24, Nathan Goldbaum <nathan12343 at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Sep 26, 2016 at 10:15 AM, Bernhard Röttgers <
>>>>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>>>>
>>>>>>>>> Hi Britton,
>>>>>>>>>
>>>>>>>>> thanks for the answer and sorry for my late answer!
>>>>>>>>>
>>>>>>>>> No, I don’t have OWLS/EAGLE/Gizmo snapshots. I actually just
>>>>>>>>> wanted to compare my results with yt/trident, but your solution does not
>>>>>>>>> seem to be too complicated. How, am I supposed to use it? (Sorry for asking
>>>>>>>>> dump questions. I’m not familiar with yt.) I understood that I can load a
>>>>>>>>> Gadget snapshot with like:
>>>>>>>>>
>>>>>>>>> ds = yt.load(YT_FNAME, unit_base=UNIT_BASE, bounding_box=bbox)
>>>>>>>>> ds.index
>>>>>>>>>
>>>>>>>>> The data structure `ds` then is what is passed to trident. Where
>>>>>>>>> does the new frontend come into play?
>>>>>>>>>
>>>>>>>>
>>>>>>>> Right now yt makes certain assumptions about the structure and
>>>>>>>> meaning of data in Gadget output files. The assumptions currently used in
>>>>>>>> yt are based on the public data the yt developers have access to (e.g. the
>>>>>>>> datasets on yt-project.org/data) and are only valid in so far as
>>>>>>>> those datasets are representative of Gadget data in general. Due to the
>>>>>>>> history of the Gadget's code, there are in reality many different flavors
>>>>>>>> of Gadget that make different assumptions about what the data they write to
>>>>>>>> disk means, and it seems yt doesn't currently make incorrect assumptions
>>>>>>>> for your flavor of Gadget data.
>>>>>>>>
>>>>>>>> Do you happen to have a test dataset that you can share? In
>>>>>>>> principle the necessary modification should be simple, and having a dataset
>>>>>>>> to test with would be helpful. One way to share files with us is to use the
>>>>>>>> yt curldrop:
>>>>>>>>
>>>>>>>> https://docs.hub.yt/services.html#curldrop
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> And some more questions:
>>>>>>>>> What exactly are the code metallicity units? In my case yt says
>>>>>>>>> its in units of 1. Is this interpreted as absolute units or in solar
>>>>>>>>> metallicity?
>>>>>>>>>
>>>>>>>>
>>>>>>>> In units of solar metallicity. There is also the metal_density,
>>>>>>>> which should have units of g/cm**3 (by default, if may be different if you
>>>>>>>> specified a value for the `unit_system` keyword when you called the load()
>>>>>>>> function.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Bernhard
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 19 Sep 2016, at 15:04, Britton Smith <brittonsmith at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Bernhard,
>>>>>>>>>
>>>>>>>>> Does your data come from a specific flavor of Gadget that is
>>>>>>>>> supported by yt?  For example, we currently support OWLS, EAGLE, and Gizmo
>>>>>>>>> data that have fields for individual elements.  If not, then it is probably
>>>>>>>>> best to try and put together a new frontend for this data.  If the type of
>>>>>>>>> data that you're working with is not public or widely used, the best thing
>>>>>>>>> may be to create a frontend that exists as a yt extension (an external
>>>>>>>>> module not inside the main yt codebase).  Here is an example of one that I
>>>>>>>>> created a while back for the FIRE simulation data:
>>>>>>>>> https://bitbucket.org/brittonsmith/yt_fire
>>>>>>>>>
>>>>>>>>> If the only thing that is different about your data is the way
>>>>>>>>> that the metallicity fields are defined, then it shouldn't be too difficult
>>>>>>>>> to create a subclass of an existing Gadget frontend that overrides the
>>>>>>>>> field definitions.  Please, let us know if you're interested in pursuing
>>>>>>>>> this and we can work with you.
>>>>>>>>>
>>>>>>>>> Britton
>>>>>>>>>
>>>>>>>>> On Sat, Sep 17, 2016 at 11:45 AM, Bernhard Röttgers <
>>>>>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>>>>>
>>>>>>>>>> Hello!
>>>>>>>>>>
>>>>>>>>>> I am trying to use trident
>>>>>>>>>> <https://bitbucket.org/trident-project/trident> (yt-based code)
>>>>>>>>>> to generate spectra out of my Gadget simulations (stored in HDF5). I could
>>>>>>>>>> get the code running, but the spectra are scaled weirdly. I was able to
>>>>>>>>>> track the problem down to a weirdly scaled metallicity block within yt. I
>>>>>>>>>> am guessing that the issue is related to the fact, that my block
>>>>>>>>>> “Metallicity” is in fact element masses for individual elements, i.e. for
>>>>>>>>>> each particle I have an array of masses for the 12 elements He, C, Mg, O,
>>>>>>>>>> Fe, Si, H, N, Ne, S, Ca, and the rest (I know, a crappy naming for such a
>>>>>>>>>> block, but that’s the way it is).
>>>>>>>>>>
>>>>>>>>>> Is there a way to force yt to create the metallicity block
>>>>>>>>>> correctly?.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Bernhard
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160928/56f1ab57/attachment-0001.htm>


More information about the yt-users mailing list