[yt-users] profile plots combining different datasets
Nathan Goldbaum
nathan12343 at gmail.com
Fri Nov 3 16:45:13 PDT 2017
It might also be interesting to explore whether it makes sense to define
some kind of chained data object that can combine data from different
datasets. So you could do:
dobj = yt.chained_data([ds1.all_data(), ds2.all_data(), ...])
profile = yt.create_profile(dobj, ['density', 'temperature'], 'cell_mass')
This would be set up such that
dobj['density'] == uconcatenate([ds1.all_data()['density'],
ds2.all_data()['density'], ...])
I'm not sure how generally useful this would be but then again I've never
been in a situation where I wanted to combine data from multiple datasets
like YT is trying to do right now.
On Fri, Nov 3, 2017 at 6:38 PM, Britton Smith <brittonsmith at gmail.com>
wrote:
> Hi YT,
>
> Nathan and I have fleshed this out a bit in the discussion on the issue he
> created.
>
> The main issue is that phase plots need to have all fields to be the same
> shape. So the x and y fields also have to be 128x128. You can make this
> happen in the following way:
> Given a profile object called "profile":
> x = yt.YTArray([profile.x]*128)
> y = yt.YTArray([profile.y]*128).transpose()
>
> If you then use those x and y in your save_as_dataset call, you should be
> able to use the loaded data with phaseplot.
>
> Britton
>
> On Fri, Nov 3, 2017 at 4:34 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>
>> I was saving data just because I don't know how to sum up field values
>> (in this case, "cell_mass") in other way. What I ultimately want to do is
>> to make one phase plot which has "density", "temperature" bin fields, and
>> the "cell_mass" is summed up through all different datasets (like,
>> IsolatedGalaxy/galaxy0030/galaxy0030, IsolatedGalaxy/galaxy0029/galaxy0029,
>> IsolatedGalaxy/galaxy0028/galaxy0028, ...).
>> Is it possible to sum up "prof_2d_ds.data ["cell_mass"]" in the example
>> from different datasets?
>>
>> 2017-11-03 19:05 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>
>>> So what's happening is that when you're save_as_dataset at the end of
>>> your notebook, you save data with incompatible shapes. In particular,
>>> 'density' and 'temperature' have shape (128) while 'cell_mass' has shape
>>> (128, 128). We should probably error out in cases like this because the
>>> resulting dataset is corrupt - all fields in a single dataset should have
>>> the same shape. The error you're seeing happens when yt tries to create an
>>> empty array to hold the cell_mass data from your saved dataset by using the
>>> shape of one of the bin fields. This leads to an error because the
>>> cell_mass field in your saved dataset has a different shape.
>>>
>>> One thing that might be immediately helpful is to save your individual
>>> profile objects you want to sum, reload those, and then combine the data?
>>> You can save profile objects with save_as_dataset:
>>>
>>> http://yt-project.org/docs/dev/analyzing/saving_data.html#profiles
>>>
>>> Another option would be to eschew using the ytdata frontend entirely and
>>> just save the density, temperature, and cell_mass profile fields you want
>>> save using e.g. h5py and then construct the plot you want to make using
>>> those data.
>>>
>>> It's not clear to me what you ultimately want to do with your saved data
>>> - maybe if you explain that in a little more detail I can suggest what you
>>> might want to do to restructure your workflow.
>>>
>>> On Fri, Nov 3, 2017 at 5:53 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>>>
>>>> I tried on "Isolatedgalaxy" data. I made all the outputs to comments.
>>>>
>>>> https://gist.github.com/Yuta-Tarumi/69ed0428db92553cd4f8ef93
>>>> 92429036#file-phase-plots-summing-up
>>>>
>>>>
>>>>
>>>> 2017-11-03 18:29 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>>
>>>>> So the error is coming from inside yt. Somehow yt is creating an array
>>>>> with shape (128) and trying to fill it with values from an array with shape
>>>>> (128, 128), causing numpy to complain. Unfortunately I'm still not sure
>>>>> what the underlying issue is just from looking at your code and the
>>>>> traceback. It's possible it's a yt bug, it's also possible that it's an
>>>>> issue in your notebook.
>>>>>
>>>>> Is there any chance you can make a script I can run that triggers the
>>>>> issue? That will make it much easier to understand what's going wrong. You
>>>>> could also just share your full notebook.
>>>>>
>>>>> If you can, please make use of a dataset from yt-project.org/data in
>>>>> your script. If that's not possible, you can also upload a dataset to the
>>>>> yt curldrop using the "yt upload" command line tool:
>>>>>
>>>>> $ yt upload /path/to/dataset.tar.gz
>>>>>
>>>>> and then include the link to the dataset that yt upload will print out
>>>>> once it's finished uploading the data.
>>>>>
>>>>> -Nathan
>>>>>
>>>>> On Fri, Nov 3, 2017 at 5:15 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>>>>>
>>>>>> Dear Nathan,
>>>>>>
>>>>>> Sorry for the incompleteness.
>>>>>> This is the error message from jupyter notebook:
>>>>>>
>>>>>> ValueErrorTraceback (most recent call last)<ipython-input-63-383c90a16917> in <module>() 11 fields=["cell_mass"], 12 weight_field=None,---> 13 extrema=dict(density=(1e-30, 1e-20), temperature=(1e1, 1e8)) 14 ) 15
>>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in create_profile(data_source, bin_fields, fields, n_bins, extrema, logs, units, weight_field, accumulation, fractional, deposition) 1093 setattr(obj, "fractional", fractional) 1094 if fields is not None:-> 1095 obj.add_fields([field for field in fields]) 1096 for field in fields: 1097 if fractional:
>>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in add_fields(self, fields) 125 citer = self.data_source.chunks([], "io") 126 for chunk in parallel_objects(citer):--> 127 self._bin_chunk(chunk, fields, temp_storage) 128 self._finalize_storage(fields, temp_storage) 129
>>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _bin_chunk(self, chunk, fields, storage) 565 566 def _bin_chunk(self, chunk, fields, storage):--> 567 rv = self._get_data(chunk, fields) 568 if rv is None: return 569 fdata, wdata, (bf_x, bf_y) = rv
>>>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _get_data(self, chunk, fields) 253 for i, field in enumerate(fields): 254 units = chunk.ds.field_info[field].output_units--> 255 arr[:,i] = chunk[field][filter].in_units(units) 256 if self.weight_field is not None: 257 units = chunk.ds.field_info[self.weight_field].output_units
>>>>>> ValueError: could not broadcast input array from shape (128,128) into shape (128)
>>>>>>
>>>>>>
>>>>>> 2017-11-03 17:08 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>>>>
>>>>>>> Can you share the full traceback along with the error message you're
>>>>>>> seeing? You can use gist.github.com to avoid cluttering your e-mail.
>>>>>>>
>>>>>>> On Fri, Nov 3, 2017 at 3:54 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>>>>>>>
>>>>>>>> Dear yt-users,
>>>>>>>>
>>>>>>>> Hi, I want to ask you about phase plot. Some days ago, I asked how
>>>>>>>> to sum up different datasets to make one profile plot, and it worked fine.
>>>>>>>> This time, I tried to do the same thing in phase plot, but it did not work
>>>>>>>> fine. Could you help me find out what the problem is?
>>>>>>>>
>>>>>>>> (inside for loop):
>>>>>>>> profile_medium = yt.create_profile(
>>>>>>>> data_source=medium_region,
>>>>>>>> bin_fields=["density", "temperature"],
>>>>>>>> fields=["cell_mass"],
>>>>>>>> n_bins=[128, 128],
>>>>>>>> units=dict(),
>>>>>>>> logs=dict(),
>>>>>>>> weight_field=None,
>>>>>>>> extrema=dict(density=(1e-30, 1e-20), temperature=(1e1,
>>>>>>>> 1e8))
>>>>>>>> )
>>>>>>>> plot_med = yt.ProfilePlot.from_profiles(profile_medium)
>>>>>>>> profile_med = plot_med.profiles[0]
>>>>>>>> profiles_sum_med = profiles_sum_med + profile_med["cell_mass"]
>>>>>>>>
>>>>>>>>
>>>>>>>> my_data_med = {"density": profile_med.x,
>>>>>>>> "temperature": profile_med.y,
>>>>>>>> "cell_mass": profiles_sum_med}
>>>>>>>> fake_ds_med = {"current_time": yt.YTQuantity(10, "Myr")}
>>>>>>>> yt.save_as_dataset(fake_ds_med, "phaseplots_med_all.h5",
>>>>>>>> my_data_med)
>>>>>>>>
>>>>>>>> ds = yt.load("phaseplots_med_all.h5")
>>>>>>>> ad = ds.data
>>>>>>>>
>>>>>>>> profile = yt.create_profile(ad,
>>>>>>>> ["density", "temperature"],
>>>>>>>> n_bins=[128, 128],
>>>>>>>> fields=["cell_mass"],
>>>>>>>> weight_field=None,
>>>>>>>> extrema=dict(density=(1e-30, 1e-20),
>>>>>>>> temperature=(1e1, 1e8))
>>>>>>>> )
>>>>>>>>
>>>>>>>> and I got this error:
>>>>>>>>
>>>>>>>> could not broadcast input array from shape (128,128) into shape (128)
>>>>>>>>
>>>>>>>>
>>>>>>>> Sincerely,
>>>>>>>> Y.T.
>>>>>>>>
>>>>>>>> 2017-10-26 16:15 GMT-04:00 ｔyuta <y0u1t1a5.t at gmail.com>:
>>>>>>>>
>>>>>>>>> Hi Suoqing Ji,
>>>>>>>>>
>>>>>>>>> Thanks! I'll try that.
>>>>>>>>>
>>>>>>>>> 2017-10-26 15:59 GMT-04:00 Suoqing Ji <suoqing at physics.ucsb.edu>:
>>>>>>>>>
>>>>>>>>>> Hi t yuta,
>>>>>>>>>>
>>>>>>>>>> What you can do is to get the data as numpy array directly from
>>>>>>>>>> the profiles you’ve generated, and sum them up by yourself. You can follow
>>>>>>>>>> the last example at: http://yt-project.org/docs
>>>>>>>>>> /dev/visualizing/plots.html#d-profile-plots
>>>>>>>>>>
>>>>>>>>>> For each dataset, you can have one profile:
>>>>>>>>>> profile = plot.profiles[0]
>>>>>>>>>>
>>>>>>>>>> Then you can sum up profile[‘cell_mass'] over different datasets,
>>>>>>>>>> and plot it against profile.x
>>>>>>>>>>
>>>>>>>>>> Best wishes,
>>>>>>>>>> —
>>>>>>>>>> Suoqing Ji
>>>>>>>>>> Ph.D Candidate
>>>>>>>>>> Department of Physics
>>>>>>>>>> University of California, Santa Barbara
>>>>>>>>>> http://physics.ucsb.edu/~suoqing
>>>>>>>>>>
>>>>>>>>>> On Oct 26, 2017, 12:51 PM -0700, ｔyuta <y0u1t1a5.t at gmail.com>,
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Dear yt-users,
>>>>>>>>>>
>>>>>>>>>> Hi, I want to ask you about profile plot. I'm analyzing Enzo
>>>>>>>>>> data, and I'm trying to plot metallicity-cellmass of different 12 datasets
>>>>>>>>>> into one curve. I can plot 12 curves on one image, but that's not what I
>>>>>>>>>> want. Does anyone know some good way, or links to documentation?
>>>>>>>>>> I've attached the 12-curve plot. I want to sum up all the curves
>>>>>>>>>> to get one curve.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Y.T.[image: 埋め込み画像 2]
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/65134205/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 02C67ABD76E846418AE113C0B10469D2.png
Type: image/png
Size: 88454 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/65134205/attachment-0001.png>
More information about the yt-users
mailing list