[yt-users] profile plots combining different datasets

Britton Smith brittonsmith at gmail.com
Fri Nov 3 16:38:02 PDT 2017


Hi YT,

Nathan and I have fleshed this out a bit in the discussion on the issue he
created.

The main issue is that phase plots need to have all fields to be the same
shape.  So the x and y fields also have to be 128x128.  You can make this
happen in the following way:
Given a profile object called "profile":
x = yt.YTArray([profile.x]*128)
y = yt.YTArray([profile.y]*128).transpose()

If you then use those x and y in your save_as_dataset call, you should be
able to use the loaded data with phaseplot.

Britton

On Fri, Nov 3, 2017 at 4:34 PM, tyuta <y0u1t1a5.t at gmail.com> wrote:

> I was saving data just because I don't know how to sum up field values (in
> this case, "cell_mass") in other way. What I ultimately want to do is to
> make one phase plot which has "density", "temperature" bin fields, and the
> "cell_mass" is summed up through all different datasets (like,
> IsolatedGalaxy/galaxy0030/galaxy0030, IsolatedGalaxy/galaxy0029/galaxy0029,
> IsolatedGalaxy/galaxy0028/galaxy0028, ...).
> Is it possible to sum up "prof_2d_ds.data ["cell_mass"]" in the example
> from different datasets?
>
> 2017-11-03 19:05 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>
>> So what's happening is that when you're save_as_dataset at the end of
>> your notebook, you save data with incompatible shapes. In particular,
>> 'density' and 'temperature' have shape (128) while 'cell_mass' has shape
>> (128, 128). We should probably error out in cases like this because the
>> resulting dataset is corrupt - all fields in a single dataset should have
>> the same shape. The error you're seeing happens when yt tries to create an
>> empty array to hold the cell_mass data from your saved dataset by using the
>> shape of one of the bin fields. This leads to an error because the
>> cell_mass field in your saved dataset has a different shape.
>>
>> One thing that might be immediately helpful is to save your individual
>> profile objects you want to sum, reload those, and then combine the data?
>> You can save profile objects with save_as_dataset:
>>
>> http://yt-project.org/docs/dev/analyzing/saving_data.html#profiles
>>
>> Another option would be to eschew using the ytdata frontend entirely and
>> just save the density, temperature, and cell_mass profile fields you want
>> save using e.g. h5py and then construct the plot you want to make using
>> those data.
>>
>> It's not clear to me what you ultimately want to do with your saved data
>> - maybe if you explain that in a little more detail I can suggest what you
>> might want to do to restructure your workflow.
>>
>> On Fri, Nov 3, 2017 at 5:53 PM, tyuta <y0u1t1a5.t at gmail.com> wrote:
>>
>>> I tried on "Isolatedgalaxy" data. I made all the outputs to comments.
>>>
>>> https://gist.github.com/Yuta-Tarumi/69ed0428db92553cd4f8ef93
>>> 92429036#file-phase-plots-summing-up
>>>
>>>
>>>
>>> 2017-11-03 18:29 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>
>>>> So the error is coming from inside yt. Somehow yt is creating an array
>>>> with shape (128) and trying to fill it with values from an array with shape
>>>> (128, 128), causing numpy to complain. Unfortunately I'm still not sure
>>>> what the underlying issue is just from looking at your code and the
>>>> traceback. It's possible it's a yt bug, it's also possible that it's an
>>>> issue in your notebook.
>>>>
>>>> Is there any chance you can make a script I can run that triggers the
>>>> issue? That will make it much easier to understand what's going wrong. You
>>>> could also just share your full notebook.
>>>>
>>>> If you can, please make use of a dataset from yt-project.org/data in
>>>> your script. If that's not possible, you can also upload a dataset to the
>>>> yt curldrop using the "yt upload" command line tool:
>>>>
>>>> $ yt upload /path/to/dataset.tar.gz
>>>>
>>>> and then include the link to the dataset that yt upload will print out
>>>> once it's finished uploading the data.
>>>>
>>>> -Nathan
>>>>
>>>> On Fri, Nov 3, 2017 at 5:15 PM, tyuta <y0u1t1a5.t at gmail.com> wrote:
>>>>
>>>>> Dear Nathan,
>>>>>
>>>>> Sorry for the incompleteness.
>>>>> This is the error message from jupyter notebook:
>>>>>
>>>>> ValueErrorTraceback (most recent call last)<ipython-input-63-383c90a16917> in <module>()     11                             fields=["cell_mass"],     12                             weight_field=None,---> 13                             extrema=dict(density=(1e-30, 1e-20), temperature=(1e1, 1e8))     14                            )     15
>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in create_profile(data_source, bin_fields, fields, n_bins, extrema, logs, units, weight_field, accumulation, fractional, deposition)   1093     setattr(obj, "fractional", fractional)   1094     if fields is not None:-> 1095         obj.add_fields([field for field in fields])   1096     for field in fields:   1097         if fractional:
>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in add_fields(self, fields)    125         citer = self.data_source.chunks([], "io")    126         for chunk in parallel_objects(citer):--> 127             self._bin_chunk(chunk, fields, temp_storage)    128         self._finalize_storage(fields, temp_storage)    129
>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _bin_chunk(self, chunk, fields, storage)    565     566     def _bin_chunk(self, chunk, fields, storage):--> 567         rv = self._get_data(chunk, fields)    568         if rv is None: return    569         fdata, wdata, (bf_x, bf_y) = rv
>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _get_data(self, chunk, fields)    253         for i, field in enumerate(fields):    254             units = chunk.ds.field_info[field].output_units--> 255             arr[:,i] = chunk[field][filter].in_units(units)    256         if self.weight_field is not None:    257             units = chunk.ds.field_info[self.weight_field].output_units
>>>>> ValueError: could not broadcast input array from shape (128,128) into shape (128)
>>>>>
>>>>>
>>>>> 2017-11-03 17:08 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>>>
>>>>>> Can you share the full traceback along with the error message you're
>>>>>> seeing? You can use gist.github.com to avoid cluttering your e-mail.
>>>>>>
>>>>>> On Fri, Nov 3, 2017 at 3:54 PM, tyuta <y0u1t1a5.t at gmail.com> wrote:
>>>>>>
>>>>>>> Dear yt-users,
>>>>>>>
>>>>>>> Hi, I want to ask you about phase plot. Some days ago, I asked how
>>>>>>> to sum up different datasets to make one profile plot, and it worked fine.
>>>>>>> This time, I tried to do the same thing in phase plot, but it did not work
>>>>>>> fine. Could you help me find out what the problem is?
>>>>>>>
>>>>>>> (inside for loop):
>>>>>>>      profile_medium = yt.create_profile(
>>>>>>>              data_source=medium_region,
>>>>>>>              bin_fields=["density", "temperature"],
>>>>>>>              fields=["cell_mass"],
>>>>>>>              n_bins=[128, 128],
>>>>>>>              units=dict(),
>>>>>>>              logs=dict(),
>>>>>>>              weight_field=None,
>>>>>>>              extrema=dict(density=(1e-30, 1e-20), temperature=(1e1,
>>>>>>> 1e8))
>>>>>>>           )
>>>>>>>      plot_med = yt.ProfilePlot.from_profiles(profile_medium)
>>>>>>>      profile_med = plot_med.profiles[0]
>>>>>>>      profiles_sum_med = profiles_sum_med + profile_med["cell_mass"]
>>>>>>>
>>>>>>>
>>>>>>> my_data_med = {"density": profile_med.x,
>>>>>>>            "temperature": profile_med.y,
>>>>>>>               "cell_mass": profiles_sum_med}
>>>>>>> fake_ds_med = {"current_time": yt.YTQuantity(10, "Myr")}
>>>>>>> yt.save_as_dataset(fake_ds_med, "phaseplots_med_all.h5",
>>>>>>> my_data_med)
>>>>>>>
>>>>>>> ds = yt.load("phaseplots_med_all.h5")
>>>>>>> ad = ds.data
>>>>>>>
>>>>>>> profile = yt.create_profile(ad,
>>>>>>>                             ["density", "temperature"],
>>>>>>>                             n_bins=[128, 128],
>>>>>>>                             fields=["cell_mass"],
>>>>>>>                             weight_field=None,
>>>>>>>                             extrema=dict(density=(1e-30, 1e-20),
>>>>>>> temperature=(1e1, 1e8))
>>>>>>>                            )
>>>>>>>
>>>>>>> and I got this error:
>>>>>>>
>>>>>>> could not broadcast input array from shape (128,128) into shape (128)
>>>>>>>
>>>>>>>
>>>>>>> Sincerely,
>>>>>>> Y.T.
>>>>>>>
>>>>>>> 2017-10-26 16:15 GMT-04:00 tyuta <y0u1t1a5.t at gmail.com>:
>>>>>>>
>>>>>>>> Hi Suoqing Ji,
>>>>>>>>
>>>>>>>> Thanks! I'll try that.
>>>>>>>>
>>>>>>>> 2017-10-26 15:59 GMT-04:00 Suoqing Ji <suoqing at physics.ucsb.edu>:
>>>>>>>>
>>>>>>>>> Hi t yuta,
>>>>>>>>>
>>>>>>>>> What you can do is to get the data as numpy array directly from
>>>>>>>>> the profiles you’ve generated, and sum them up by yourself. You can follow
>>>>>>>>> the last example at: http://yt-project.org/docs
>>>>>>>>> /dev/visualizing/plots.html#d-profile-plots
>>>>>>>>>
>>>>>>>>> For each dataset, you can have one profile:
>>>>>>>>> profile = plot.profiles[0]
>>>>>>>>>
>>>>>>>>> Then you can sum up profile[‘cell_mass'] over different datasets,
>>>>>>>>> and plot it against profile.x
>>>>>>>>>
>>>>>>>>> Best wishes,
>>>>>>>>>>>>>>>>>> Suoqing Ji
>>>>>>>>> Ph.D Candidate
>>>>>>>>> Department of Physics
>>>>>>>>> University of California, Santa Barbara
>>>>>>>>> http://physics.ucsb.edu/~suoqing
>>>>>>>>>
>>>>>>>>> On Oct 26, 2017, 12:51 PM -0700, tyuta <y0u1t1a5.t at gmail.com>,
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Dear yt-users,
>>>>>>>>>
>>>>>>>>> Hi, I want to ask you about profile plot. I'm analyzing Enzo data,
>>>>>>>>> and I'm trying to plot metallicity-cellmass of different 12 datasets into
>>>>>>>>> one curve. I can plot 12 curves on one image, but that's not what I want.
>>>>>>>>> Does anyone know some good way, or links to documentation?
>>>>>>>>> I've attached the 12-curve plot. I want to sum up all the curves
>>>>>>>>> to get one curve.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Y.T.[image: 埋め込み画像 2]
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/ce446f1a/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 02C67ABD76E846418AE113C0B10469D2.png
Type: image/png
Size: 88454 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/ce446f1a/attachment-0002.png>


More information about the yt-users mailing list