[yt-users] Gadget blocks' configuration

Bernhard Röttgers broett at MPA-Garching.MPG.DE
Wed Sep 28 06:27:52 PDT 2016


Hi Britton,

thanks for spotting this error! So that means yt arrays do not behave like numpy arrays, since:
In [2]: a = np.arange(10)

In [3]: b = np.arange(10)

In [4]: c = np.arange(10)

In [5]: sum( x for x in [a,b,c] )
Out[5]: array([ 0,  3,  6,  9, 12, 15, 18, 21, 24, 27])
Which is what I had in mind, but apparently does not happen with yt arrays.

However, you proposed code seem to have problems, too. First of all the loop you propose is not Python 2.7, which I use. And second, even after fixing that, the problem is not solved. Actually even the particle_mass block seems to be broken somehow. I cannot access it by (“gas”, “particle_mass”) but only via (“partType0”, “particle_mass”).

Cheers,
Bernhard


> On 28 Sep 2016, at 15:08, Britton Smith <brittonsmith at gmail.com> wrote:
> 
> Hi Bernhard,
> 
> I'm just starting to look at this now.  I haven't looked to closely yet, but I noticed an issue with your _metal_mass field function.  Right now, you've got:
> 
>         def _metal_mass(field, data):
>             return sum( data[(ptype, "%s_mass"%el)]
>                     for el in self.nuclei_names if el not in ["H","He"] )
> 
> The sum function is going to reduce the entire field array to a single number.  I think you need something like this:
> 
> field_data = np.zeros_like(data[ptype, "particle_mass"])
> for el in self.nuclei_names if el not in ["H","He"]:
>     field_data += data[ptype, "%s_mass"% el]
> 
> The metallicity field is likely not showing up because it is quietly erroring.  This may fix that.  Let us know how it goes.
> 
> Britton
> 
> On Wed, Sep 28, 2016 at 1:44 PM, Bernhard Röttgers <broett at mpa-garching.mpg.de <mailto:broett at mpa-garching.mpg.de>> wrote:
> Hi there,
> 
> I still have problems with the “gas”. I can access (“PartType0”, “metallicity”) but not (“gas”, “metallicity”). Why is the alias not being created automatically as I understood it should?
> 
> Cheers,
> Bernhard
> 
>> On 26 Sep 2016, at 20:36, Britton Smith <brittonsmith at gmail.com <mailto:brittonsmith at gmail.com>> wrote:
>> 
>> Hi Bernhard,
>> 
>> The "gas" alias points to a version of the fields that has been deposited on the octtree.  For the ion-specific fields, Trident will take care of adding this for you as long as the element fields are added correctly.  I am at the end of my day here, but I can take a look at the dataset you've uploaded tomorrow.
>> 
>> On Mon, Sep 26, 2016 at 6:51 PM, Bernhard Röttgers <broett at mpa-garching.mpg.de <mailto:broett at mpa-garching.mpg.de>> wrote:
>> Got the new fields. I had a little typo in the names.
>> Couldn’t  solve the problems with the unit system though.
>> And I have the additional problem, that I need to define a particle group “gas” for trident. How do I do this?
>> 
>> Cheers,
>> Bernhard
>> 
>>> On 26 Sep 2016, at 18:55, Bernhard Röttgers <broett at MPA-Garching.MPG.DE <mailto:broett at mpa-garching.mpg.de>> wrote:
>>> 
>>> Okay, so the (HDF5) snapshot is now available at:
>>> http://use.yt/upload/09111d2e <http://use.yt/upload/09111d2e>
>>> 
>>> I used the FIRE frontend by Britton (https://bitbucket.org/brittonsmith/yt_fire <https://bitbucket.org/brittonsmith/yt_fire>) as a basis to create my own:
>>> https://bitbucket.org/broett/my_yt/overview <https://bitbucket.org/broett/my_yt/overview> (it’s git, not hg!)
>>> 
>>> I can load a snapshot with the front-end, but I still don’t have the self-defined fields. Meaning that after
>>> 
>>> ds = my_yt.MyGadgetDataset(‘path/to/snapshot/snap_M0664_4x_940.hdf5', unit_base=…, bounding_box=…)
>>> ds.index
>>> ad = ds.all_data()
>>> 
>>> the following fails:
>>> ad[('PartType0','metallicity’)]
>>> 
>>> I also cannot use self.ds.unit_system["code_metallicity”] in class MyGadgetFieldInfo(GadgetFieldInfo) as Britton did.
>>> 
>>> 
>>> Cheers,
>>> Bernhard
>>> 
>>> 
>>> 
>>>> On 26 Sep 2016, at 18:10, Nathan Goldbaum <nathan12343 at gmail.com <mailto:nathan12343 at gmail.com>> wrote:
>>>> 
>>>> 
>>>> 
>>>> On Mon, Sep 26, 2016 at 11:07 AM, Bernhard Röttgers <broett at mpa-garching.mpg.de <mailto:broett at mpa-garching.mpg.de>> wrote:
>>>> Okay, then I upload HDF5 for now. Called the file “snap_M6782_4x_470.hdf5”.
>>>> So what are we gonna do with it?
>>>> 
>>>> 
>>>> You can upload it using the yt curldrop:
>>>> 
>>>> $ curl -T snap_M6782_4x_470.hdf5 http://use.yt/upload/ <http://use.yt/upload/>
>>>> 
>>>> That will print out a URL once the upload is finished, just share that URL here.
>>>> 
>>>> Alternatively generating a share link via google drive or dropbox will also work.
>>>>  
>>>> Oops… I mixed up the field info and the data structure classes. My bad!
>>>> 
>>>> Cheers,
>>>> Bernhard
>>>> 
>>>> 
>>>>> On 26 Sep 2016, at 17:55, Nathan Goldbaum <nathan12343 at gmail.com <mailto:nathan12343 at gmail.com>> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> On Mon, Sep 26, 2016 at 10:42 AM, Bernhard Röttgers <broett at mpa-garching.mpg.de <mailto:broett at mpa-garching.mpg.de>> wrote:
>>>>> Hi Nathan,
>>>>> 
>>>>> Yes, I have some smaller (~85MB) sample I could upload. I guess it should ideally be HDF5? (Originally it is format 2, but I can easily convert.) What naming should I use?
>>>>> I would guess, my format is pretty much “the standard” except for the metallicity block (which in fact is element masses for 12 different elements).
>>>>> 
>>>>> The naming and format doesn't matter too much. HDF5 is probably easiest to deal with, although yt supports both binary formats as well.
>>>>>  
>>>>> 
>>>>> So again, how am I supposed the data set class I derive from GadgetHDF5Dataset? Simply replacing `yt.load` yields an exception complaining that the `bounding_box` keyword argument is unexpected.
>>>>> 
>>>>> I think all you need to do is declare the subclass and then load your dataset directly:
>>>>> 
>>>>> my_ds = MyGadgetHDF5DatasetSubclass(unit_base=..., bbox=...)
>>>>> 
>>>>> It should also be possible to make your subclass work with yt.load() (i'm not sure how to do that offhand as I've never tried to add a new Dataset subclass outside of yt), but instantiating your subclass directly should be fine for now.
>>>>>  
>>>>> 
>>>>> Cheers,
>>>>> Bernhard
>>>>> 
>>>>>> On 26 Sep 2016, at 17:24, Nathan Goldbaum <nathan12343 at gmail.com <mailto:nathan12343 at gmail.com>> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Mon, Sep 26, 2016 at 10:15 AM, Bernhard Röttgers <broett at mpa-garching.mpg.de <mailto:broett at mpa-garching.mpg.de>> wrote:
>>>>>> Hi Britton,
>>>>>> 
>>>>>> thanks for the answer and sorry for my late answer!
>>>>>> 
>>>>>> No, I don’t have OWLS/EAGLE/Gizmo snapshots. I actually just wanted to compare my results with yt/trident, but your solution does not seem to be too complicated. How, am I supposed to use it? (Sorry for asking dump questions. I’m not familiar with yt.) I understood that I can load a Gadget snapshot with like:
>>>>>> 
>>>>>> ds = yt.load(YT_FNAME, unit_base=UNIT_BASE, bounding_box=bbox)
>>>>>> ds.index
>>>>>> 
>>>>>> The data structure `ds` then is what is passed to trident. Where does the new frontend come into play?
>>>>>> 
>>>>>> Right now yt makes certain assumptions about the structure and meaning of data in Gadget output files. The assumptions currently used in yt are based on the public data the yt developers have access to (e.g. the datasets on yt-project.org/data <http://yt-project.org/data>) and are only valid in so far as those datasets are representative of Gadget data in general. Due to the history of the Gadget's code, there are in reality many different flavors of Gadget that make different assumptions about what the data they write to disk means, and it seems yt doesn't currently make incorrect assumptions for your flavor of Gadget data.
>>>>>> 
>>>>>> Do you happen to have a test dataset that you can share? In principle the necessary modification should be simple, and having a dataset to test with would be helpful. One way to share files with us is to use the yt curldrop:
>>>>>> 
>>>>>> https://docs.hub.yt/services.html#curldrop <https://docs.hub.yt/services.html#curldrop>
>>>>>>  
>>>>>> 
>>>>>> And some more questions:
>>>>>> What exactly are the code metallicity units? In my case yt says its in units of 1. Is this interpreted as absolute units or in solar metallicity?
>>>>>> 
>>>>>> In units of solar metallicity. There is also the metal_density, which should have units of g/cm**3 (by default, if may be different if you specified a value for the `unit_system` keyword when you called the load() function.
>>>>>>  
>>>>>> 
>>>>>> Cheers,
>>>>>> Bernhard
>>>>>> 
>>>>>> 
>>>>>>> On 19 Sep 2016, at 15:04, Britton Smith <brittonsmith at gmail.com <mailto:brittonsmith at gmail.com>> wrote:
>>>>>>> 
>>>>>>> Hi Bernhard,
>>>>>>> 
>>>>>>> Does your data come from a specific flavor of Gadget that is supported by yt?  For example, we currently support OWLS, EAGLE, and Gizmo data that have fields for individual elements.  If not, then it is probably best to try and put together a new frontend for this data.  If the type of data that you're working with is not public or widely used, the best thing may be to create a frontend that exists as a yt extension (an external module not inside the main yt codebase).  Here is an example of one that I created a while back for the FIRE simulation data:
>>>>>>> https://bitbucket.org/brittonsmith/yt_fire <https://bitbucket.org/brittonsmith/yt_fire>
>>>>>>> 
>>>>>>> If the only thing that is different about your data is the way that the metallicity fields are defined, then it shouldn't be too difficult to create a subclass of an existing Gadget frontend that overrides the field definitions.  Please, let us know if you're interested in pursuing this and we can work with you.
>>>>>>> 
>>>>>>> Britton
>>>>>>> 
>>>>>>> On Sat, Sep 17, 2016 at 11:45 AM, Bernhard Röttgers <broett at mpa-garching.mpg.de <mailto:broett at mpa-garching.mpg.de>> wrote:
>>>>>>> Hello!
>>>>>>> 
>>>>>>> I am trying to use trident <https://bitbucket.org/trident-project/trident> (yt-based code) to generate spectra out of my Gadget simulations (stored in HDF5). I could get the code running, but the spectra are scaled weirdly. I was able to track the problem down to a weirdly scaled metallicity block within yt. I am guessing that the issue is related to the fact, that my block “Metallicity” is in fact element masses for individual elements, i.e. for each particle I have an array of masses for the 12 elements He, C, Mg, O, Fe, Si, H, N, Ne, S, Ca, and the rest (I know, a crappy naming for such a block, but that’s the way it is).
>>>>>>> 
>>>>>>> Is there a way to force yt to create the metallicity block correctly?.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Bernhard
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>>>> 
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>>> 
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>>> 
>>>> 
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>>> 
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>> 
>> 
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
>> 
>> 
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
> 
> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
> 
> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160928/562f1f4e/attachment.html>


More information about the yt-users mailing list