[yt-users] Gadget blocks' configuration

Britton Smith brittonsmith at gmail.com
Wed Sep 28 08:22:56 PDT 2016 

Hi Bernhard,

Yes, I think I agree with you, but I'm not 100% sure.  If someone knows of
an instance where that might happen, please speak up.  You should be able
to pass units of "" (i.e., no units).  It looks like you're using
"code_metallicity", which looks to be the same thing.  "Zsun" should be
defined, so you can do:
ds.r["gas", "metallicity"].to("Zsun")
to see things in solar.  The metallicities will be in linear units.

Does anyone else have any ideas?

Britton

On Wed, Sep 28, 2016 at 3:56 PM, Bernhard Röttgers <
broett at mpa-garching.mpg.de> wrote:

> Hi Britton,
>
> thanks a lot! I finally can access the metallicity block :)
> However, I am stilled puzzled by the result: The maximum value I have for
> the particles is ~0.08, but the maximum value I see in yt is ~0.10. That
> seems be to wrong. Regardless of the details of the mapping onto a grid, I
> should never get larger metallicities than for the individual particles,
> right?
>
> And what units do I have to pass? Did I correctly understand that yt
> always assumes the metallicities to be in solar metallicities? And is it in
> linear units or logarithmic units?
>
> Cheers,
> Bernhard
>
> On 28 Sep 2016, at 16:03, Britton Smith <brittonsmith at gmail.com> wrote:
>
> Hi Bernhard,
>
> Sorry, the previous suggestion I gave you was completely wrong.
>
> The issue is that the step where the ("gas", "metallicity") field is being
> created is happening before your setup_gas_particle_fields function is
> getting called.  The field never gets created because the ("PartType0",
> "metallicity") field does not yet exist.  I think yt is expecting it to be
> there because "metallicity" is typically aliased to a specific field on
> disk and that step happens earlier on.  Some rearranging of the order or
> operations might make this possible, but this simplest solution is to
> directly add the ("gas", "metallicity") field in your
> setup_gas_particle_fields.  If did it in the following way and it worked
> for me:
>
> from yt.fields.particle_fields import \
>     add_volume_weighted_smoothed_field
>
> # end of setup_gas_particle_fields
>         num_neighbors = 64
>         fn = add_volume_weighted_smoothed_field(
>             ptype, "particle_position", "particle_mass",
>             "smoothing_length", "density", "metallicity",
>             self, num_neighbors)
>         self.alias(("gas", "metallicity"), fn[0])
>
> Britton
>
> On Wed, Sep 28, 2016 at 2:44 PM, Bernhard Röttgers <
> broett at mpa-garching.mpg.de> wrote:
>
>> Never mind! `sum` also accepts generator expressions
>> <https://www.python.org/dev/peps/pep-0289/>. So in my little example the
>> last statement is equivalent with a+b+c. Since the addition of numpy
>> arrays
>> <http://docs.scipy.org/doc/numpy/reference/generated/numpy.add.html> of
>> the same shape is defined as an element-wise operation, we end up with the
>> intended result.
>>
>> The thing that does not work is the for loop with the if-statement. This
>> combination is only allowed in list comprehension and generator expressions.
>>
>> Cheers,
>> Bernhard
>>
>>
>> On 28 Sep 2016, at 15:34, Britton Smith <brittonsmith at gmail.com> wrote:
>>
>> Hi Bernhard,
>>
>> My apologies, I was not familiar with that syntax.  yt arrays will behave
>> like NumPy arrays, so if that works as in your example, it should work in
>> that function.  Can you elaborate at why my suggestion doesn't work in
>> Python 2.7?  I will try to debug this here.
>>
>> Britton
>>
>> On Wed, Sep 28, 2016 at 2:29 PM, Bernhard Röttgers <
>> broett at mpa-garching.mpg.de> wrote:
>>
>>> PS:
>>> I also cannot access (‘PartType0’,’metallicity’) including you attempt
>>> to fix.
>>>
>>> On 28 Sep 2016, at 15:27, Bernhard Röttgers <broett at MPA-Garching.MPG.DE
>>> <broett at mpa-garching.mpg.de>> wrote:
>>>
>>> Hi Britton,
>>>
>>> thanks for spotting this error! So that means yt arrays do not behave
>>> like numpy arrays, since:
>>> In [2]: a = np.arange(10)
>>>
>>> In [3]: b = np.arange(10)
>>>
>>> In [4]: c = np.arange(10)
>>>
>>> In [5]: sum( x for x in [a,b,c] )
>>> Out[5]: array([ 0,  3,  6,  9, 12, 15, 18, 21, 24, 27])
>>> Which is what I had in mind, but apparently does not happen with yt
>>> arrays.
>>>
>>> However, you proposed code seem to have problems, too. First of all the
>>> loop you propose is not Python 2.7, which I use. And second, even after
>>> fixing that, the problem is not solved. Actually even the particle_mass
>>> block seems to be broken somehow. I cannot access it by (“gas”,
>>> “particle_mass”) but only via (“partType0”, “particle_mass”).
>>>
>>> Cheers,
>>> Bernhard
>>>
>>>
>>> On 28 Sep 2016, at 15:08, Britton Smith <brittonsmith at gmail.com> wrote:
>>>
>>> Hi Bernhard,
>>>
>>> I'm just starting to look at this now.  I haven't looked to closely yet,
>>> but I noticed an issue with your _metal_mass field function.  Right now,
>>> you've got:
>>>
>>>         def _metal_mass(field, data):
>>>             return sum( data[(ptype, "%s_mass"%el)]
>>>                     for el in self.nuclei_names if el not in ["H","He"] )
>>>
>>> The sum function is going to reduce the entire field array to a single
>>> number.  I think you need something like this:
>>>
>>> field_data = np.zeros_like(data[ptype, "particle_mass"])
>>> for el in self.nuclei_names if el not in ["H","He"]:
>>>     field_data += data[ptype, "%s_mass"% el]
>>>
>>> The metallicity field is likely not showing up because it is quietly
>>> erroring.  This may fix that.  Let us know how it goes.
>>>
>>> Britton
>>>
>>> On Wed, Sep 28, 2016 at 1:44 PM, Bernhard Röttgers <
>>> broett at mpa-garching.mpg.de> wrote:
>>>
>>>> Hi there,
>>>>
>>>> I still have problems with the “gas”. I can access (“PartType0”,
>>>> “metallicity”) but not (“gas”, “metallicity”). Why is the alias not being
>>>> created automatically as I understood it should?
>>>>
>>>> Cheers,
>>>> Bernhard
>>>>
>>>> On 26 Sep 2016, at 20:36, Britton Smith <brittonsmith at gmail.com> wrote:
>>>>
>>>> Hi Bernhard,
>>>>
>>>> The "gas" alias points to a version of the fields that has been
>>>> deposited on the octtree.  For the ion-specific fields, Trident will take
>>>> care of adding this for you as long as the element fields are added
>>>> correctly.  I am at the end of my day here, but I can take a look at the
>>>> dataset you've uploaded tomorrow.
>>>>
>>>> On Mon, Sep 26, 2016 at 6:51 PM, Bernhard Röttgers <
>>>> broett at mpa-garching.mpg.de> wrote:
>>>>
>>>>> Got the new fields. I had a little typo in the names.
>>>>> Couldn’t  solve the problems with the unit system though.
>>>>> And I have the additional problem, that I need to define a particle
>>>>> group “gas” for trident. How do I do this?
>>>>>
>>>>> Cheers,
>>>>> Bernhard
>>>>>
>>>>> On 26 Sep 2016, at 18:55, Bernhard Röttgers <
>>>>> broett at MPA-Garching.MPG.DE <broett at mpa-garching.mpg.de>> wrote:
>>>>>
>>>>> Okay, so the (HDF5) snapshot is now available at:
>>>>> http://use.yt/upload/09111d2e
>>>>>
>>>>> I used the FIRE frontend by Britton (https://bitbucket.org/britton
>>>>> smith/yt_fire) as a basis to create my own:
>>>>> https://bitbucket.org/broett/my_yt/overview (it’s git, not hg!)
>>>>>
>>>>> I can load a snapshot with the front-end, but I still don’t have the
>>>>> self-defined fields. Meaning that after
>>>>>
>>>>> ds = my_yt.MyGadgetDataset(‘path/to/snapshot/snap_M0664_4x_940.hdf5',
>>>>> unit_base=…, bounding_box=…)
>>>>> ds.index
>>>>> ad = ds.all_data()
>>>>>
>>>>> the following fails:
>>>>> ad[('PartType0','metallicity’)]
>>>>>
>>>>> I also cannot use self.ds.unit_system["code_metallicity”] in class
>>>>> MyGadgetFieldInfo(GadgetFieldInfo) as Britton did.
>>>>>
>>>>>
>>>>> Cheers,
>>>>> Bernhard
>>>>>
>>>>>
>>>>>
>>>>> On 26 Sep 2016, at 18:10, Nathan Goldbaum <nathan12343 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Sep 26, 2016 at 11:07 AM, Bernhard Röttgers <
>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>
>>>>>> Okay, then I upload HDF5 for now. Called the file
>>>>>> “snap_M6782_4x_470.hdf5”.
>>>>>> So what are we gonna do with it?
>>>>>>
>>>>>>
>>>>> You can upload it using the yt curldrop:
>>>>>
>>>>> $ curl -T snap_M6782_4x_470.hdf5 http://use.yt/upload/
>>>>>
>>>>> That will print out a URL once the upload is finished, just share that
>>>>> URL here.
>>>>>
>>>>> Alternatively generating a share link via google drive or dropbox will
>>>>> also work.
>>>>>
>>>>>
>>>>>> Oops… I mixed up the field info and the data structure classes. My
>>>>>> bad!
>>>>>>
>>>>>> Cheers,
>>>>>> Bernhard
>>>>>>
>>>>>>
>>>>>> On 26 Sep 2016, at 17:55, Nathan Goldbaum <nathan12343 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Sep 26, 2016 at 10:42 AM, Bernhard Röttgers <
>>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>>
>>>>>>> Hi Nathan,
>>>>>>>
>>>>>>> Yes, I have some smaller (~85MB) sample I could upload. I guess it
>>>>>>> should ideally be HDF5? (Originally it is format 2, but I can easily
>>>>>>> convert.) What naming should I use?
>>>>>>> I would guess, my format is pretty much “the standard” except for
>>>>>>> the metallicity block (which in fact is element masses for 12 different
>>>>>>> elements).
>>>>>>>
>>>>>>
>>>>>> The naming and format doesn't matter too much. HDF5 is probably
>>>>>> easiest to deal with, although yt supports both binary formats as well.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> So again, how am I supposed the data set class I derive from
>>>>>>> GadgetHDF5Dataset? Simply replacing `yt.load` yields an exception
>>>>>>> complaining that the `bounding_box` keyword argument is unexpected.
>>>>>>>
>>>>>>
>>>>>> I think all you need to do is declare the subclass and then load your
>>>>>> dataset directly:
>>>>>>
>>>>>> my_ds = MyGadgetHDF5DatasetSubclass(unit_base=..., bbox=...)
>>>>>>
>>>>>> It should also be possible to make your subclass work with yt.load()
>>>>>> (i'm not sure how to do that offhand as I've never tried to add a new
>>>>>> Dataset subclass outside of yt), but instantiating your subclass directly
>>>>>> should be fine for now.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Bernhard
>>>>>>>
>>>>>>> On 26 Sep 2016, at 17:24, Nathan Goldbaum <nathan12343 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Sep 26, 2016 at 10:15 AM, Bernhard Röttgers <
>>>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>>>
>>>>>>>> Hi Britton,
>>>>>>>>
>>>>>>>> thanks for the answer and sorry for my late answer!
>>>>>>>>
>>>>>>>> No, I don’t have OWLS/EAGLE/Gizmo snapshots. I actually just wanted
>>>>>>>> to compare my results with yt/trident, but your solution does not seem to
>>>>>>>> be too complicated. How, am I supposed to use it? (Sorry for asking dump
>>>>>>>> questions. I’m not familiar with yt.) I understood that I can load a Gadget
>>>>>>>> snapshot with like:
>>>>>>>>
>>>>>>>> ds = yt.load(YT_FNAME, unit_base=UNIT_BASE, bounding_box=bbox)
>>>>>>>> ds.index
>>>>>>>>
>>>>>>>> The data structure `ds` then is what is passed to trident. Where
>>>>>>>> does the new frontend come into play?
>>>>>>>>
>>>>>>>
>>>>>>> Right now yt makes certain assumptions about the structure and
>>>>>>> meaning of data in Gadget output files. The assumptions currently used in
>>>>>>> yt are based on the public data the yt developers have access to (e.g. the
>>>>>>> datasets on yt-project.org/data) and are only valid in so far as
>>>>>>> those datasets are representative of Gadget data in general. Due to the
>>>>>>> history of the Gadget's code, there are in reality many different flavors
>>>>>>> of Gadget that make different assumptions about what the data they write to
>>>>>>> disk means, and it seems yt doesn't currently make incorrect assumptions
>>>>>>> for your flavor of Gadget data.
>>>>>>>
>>>>>>> Do you happen to have a test dataset that you can share? In
>>>>>>> principle the necessary modification should be simple, and having a dataset
>>>>>>> to test with would be helpful. One way to share files with us is to use the
>>>>>>> yt curldrop:
>>>>>>>
>>>>>>> https://docs.hub.yt/services.html#curldrop
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> And some more questions:
>>>>>>>> What exactly are the code metallicity units? In my case yt says its
>>>>>>>> in units of 1. Is this interpreted as absolute units or in solar
>>>>>>>> metallicity?
>>>>>>>>
>>>>>>>
>>>>>>> In units of solar metallicity. There is also the metal_density,
>>>>>>> which should have units of g/cm**3 (by default, if may be different if you
>>>>>>> specified a value for the `unit_system` keyword when you called the load()
>>>>>>> function.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Bernhard
>>>>>>>>
>>>>>>>>
>>>>>>>> On 19 Sep 2016, at 15:04, Britton Smith <brittonsmith at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Bernhard,
>>>>>>>>
>>>>>>>> Does your data come from a specific flavor of Gadget that is
>>>>>>>> supported by yt?  For example, we currently support OWLS, EAGLE, and Gizmo
>>>>>>>> data that have fields for individual elements.  If not, then it is probably
>>>>>>>> best to try and put together a new frontend for this data.  If the type of
>>>>>>>> data that you're working with is not public or widely used, the best thing
>>>>>>>> may be to create a frontend that exists as a yt extension (an external
>>>>>>>> module not inside the main yt codebase).  Here is an example of one that I
>>>>>>>> created a while back for the FIRE simulation data:
>>>>>>>> https://bitbucket.org/brittonsmith/yt_fire
>>>>>>>>
>>>>>>>> If the only thing that is different about your data is the way that
>>>>>>>> the metallicity fields are defined, then it shouldn't be too difficult to
>>>>>>>> create a subclass of an existing Gadget frontend that overrides the field
>>>>>>>> definitions.  Please, let us know if you're interested in pursuing this and
>>>>>>>> we can work with you.
>>>>>>>>
>>>>>>>> Britton
>>>>>>>>
>>>>>>>> On Sat, Sep 17, 2016 at 11:45 AM, Bernhard Röttgers <
>>>>>>>> broett at mpa-garching.mpg.de> wrote:
>>>>>>>>
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> I am trying to use trident
>>>>>>>>> <https://bitbucket.org/trident-project/trident> (yt-based code)
>>>>>>>>> to generate spectra out of my Gadget simulations (stored in HDF5). I could
>>>>>>>>> get the code running, but the spectra are scaled weirdly. I was able to
>>>>>>>>> track the problem down to a weirdly scaled metallicity block within yt. I
>>>>>>>>> am guessing that the issue is related to the fact, that my block
>>>>>>>>> “Metallicity” is in fact element masses for individual elements, i.e. for
>>>>>>>>> each particle I have an array of masses for the 12 elements He, C, Mg, O,
>>>>>>>>> Fe, Si, H, N, Ne, S, Ca, and the rest (I know, a crappy naming for such a
>>>>>>>>> block, but that’s the way it is).
>>>>>>>>>
>>>>>>>>> Is there a way to force yt to create the metallicity block
>>>>>>>>> correctly?.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Bernhard
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160928/fe75f059/attachment-0001.htm>

More information about the yt-users mailing list