[yt-users] profile plots combining different datasets
Nathan Goldbaum
nathan12343 at gmail.com
Fri Nov 3 16:13:10 PDT 2017
I've gone ahead and filed an issue:
https://github.com/yt-project/yt/issues/1609
On Fri, Nov 3, 2017 at 6:05 PM, Nathan Goldbaum <nathan12343 at gmail.com>
wrote:
> So what's happening is that when you're save_as_dataset at the end of your
> notebook, you save data with incompatible shapes. In particular, 'density'
> and 'temperature' have shape (128) while 'cell_mass' has shape (128, 128).
> We should probably error out in cases like this because the resulting
> dataset is corrupt - all fields in a single dataset should have the same
> shape. The error you're seeing happens when yt tries to create an empty
> array to hold the cell_mass data from your saved dataset by using the shape
> of one of the bin fields. This leads to an error because the cell_mass
> field in your saved dataset has a different shape.
>
> One thing that might be immediately helpful is to save your individual
> profile objects you want to sum, reload those, and then combine the data?
> You can save profile objects with save_as_dataset:
>
> http://yt-project.org/docs/dev/analyzing/saving_data.html#profiles
>
> Another option would be to eschew using the ytdata frontend entirely and
> just save the density, temperature, and cell_mass profile fields you want
> save using e.g. h5py and then construct the plot you want to make using
> those data.
>
> It's not clear to me what you ultimately want to do with your saved data -
> maybe if you explain that in a little more detail I can suggest what you
> might want to do to restructure your workflow.
>
> On Fri, Nov 3, 2017 at 5:53 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>
>> I tried on "Isolatedgalaxy" data. I made all the outputs to comments.
>>
>> https://gist.github.com/Yuta-Tarumi/69ed0428db92553cd4f8ef93
>> 92429036#file-phase-plots-summing-up
>>
>>
>>
>> 2017-11-03 18:29 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>
>>> So the error is coming from inside yt. Somehow yt is creating an array
>>> with shape (128) and trying to fill it with values from an array with shape
>>> (128, 128), causing numpy to complain. Unfortunately I'm still not sure
>>> what the underlying issue is just from looking at your code and the
>>> traceback. It's possible it's a yt bug, it's also possible that it's an
>>> issue in your notebook.
>>>
>>> Is there any chance you can make a script I can run that triggers the
>>> issue? That will make it much easier to understand what's going wrong. You
>>> could also just share your full notebook.
>>>
>>> If you can, please make use of a dataset from yt-project.org/data in
>>> your script. If that's not possible, you can also upload a dataset to the
>>> yt curldrop using the "yt upload" command line tool:
>>>
>>> $ yt upload /path/to/dataset.tar.gz
>>>
>>> and then include the link to the dataset that yt upload will print out
>>> once it's finished uploading the data.
>>>
>>> -Nathan
>>>
>>> On Fri, Nov 3, 2017 at 5:15 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>>>
>>>> Dear Nathan,
>>>>
>>>> Sorry for the incompleteness.
>>>> This is the error message from jupyter notebook:
>>>>
>>>> ValueErrorTraceback (most recent call last)<ipython-input-63-383c90a16917> in <module>() 11 fields=["cell_mass"], 12 weight_field=None,---> 13 extrema=dict(density=(1e-30, 1e-20), temperature=(1e1, 1e8)) 14 ) 15
>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in create_profile(data_source, bin_fields, fields, n_bins, extrema, logs, units, weight_field, accumulation, fractional, deposition) 1093 setattr(obj, "fractional", fractional) 1094 if fields is not None:-> 1095 obj.add_fields([field for field in fields]) 1096 for field in fields: 1097 if fractional:
>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in add_fields(self, fields) 125 citer = self.data_source.chunks([], "io") 126 for chunk in parallel_objects(citer):--> 127 self._bin_chunk(chunk, fields, temp_storage) 128 self._finalize_storage(fields, temp_storage) 129
>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _bin_chunk(self, chunk, fields, storage) 565 566 def _bin_chunk(self, chunk, fields, storage):--> 567 rv = self._get_data(chunk, fields) 568 if rv is None: return 569 fdata, wdata, (bf_x, bf_y) = rv
>>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _get_data(self, chunk, fields) 253 for i, field in enumerate(fields): 254 units = chunk.ds.field_info[field].output_units--> 255 arr[:,i] = chunk[field][filter].in_units(units) 256 if self.weight_field is not None: 257 units = chunk.ds.field_info[self.weight_field].output_units
>>>> ValueError: could not broadcast input array from shape (128,128) into shape (128)
>>>>
>>>>
>>>> 2017-11-03 17:08 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>>
>>>>> Can you share the full traceback along with the error message you're
>>>>> seeing? You can use gist.github.com to avoid cluttering your e-mail.
>>>>>
>>>>> On Fri, Nov 3, 2017 at 3:54 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>>>>>
>>>>>> Dear yt-users,
>>>>>>
>>>>>> Hi, I want to ask you about phase plot. Some days ago, I asked how
>>>>>> to sum up different datasets to make one profile plot, and it worked fine.
>>>>>> This time, I tried to do the same thing in phase plot, but it did not work
>>>>>> fine. Could you help me find out what the problem is?
>>>>>>
>>>>>> (inside for loop):
>>>>>> profile_medium = yt.create_profile(
>>>>>> data_source=medium_region,
>>>>>> bin_fields=["density", "temperature"],
>>>>>> fields=["cell_mass"],
>>>>>> n_bins=[128, 128],
>>>>>> units=dict(),
>>>>>> logs=dict(),
>>>>>> weight_field=None,
>>>>>> extrema=dict(density=(1e-30, 1e-20), temperature=(1e1,
>>>>>> 1e8))
>>>>>> )
>>>>>> plot_med = yt.ProfilePlot.from_profiles(profile_medium)
>>>>>> profile_med = plot_med.profiles[0]
>>>>>> profiles_sum_med = profiles_sum_med + profile_med["cell_mass"]
>>>>>>
>>>>>>
>>>>>> my_data_med = {"density": profile_med.x,
>>>>>> "temperature": profile_med.y,
>>>>>> "cell_mass": profiles_sum_med}
>>>>>> fake_ds_med = {"current_time": yt.YTQuantity(10, "Myr")}
>>>>>> yt.save_as_dataset(fake_ds_med, "phaseplots_med_all.h5", my_data_med)
>>>>>>
>>>>>> ds = yt.load("phaseplots_med_all.h5")
>>>>>> ad = ds.data
>>>>>>
>>>>>> profile = yt.create_profile(ad,
>>>>>> ["density", "temperature"],
>>>>>> n_bins=[128, 128],
>>>>>> fields=["cell_mass"],
>>>>>> weight_field=None,
>>>>>> extrema=dict(density=(1e-30, 1e-20),
>>>>>> temperature=(1e1, 1e8))
>>>>>> )
>>>>>>
>>>>>> and I got this error:
>>>>>>
>>>>>> could not broadcast input array from shape (128,128) into shape (128)
>>>>>>
>>>>>>
>>>>>> Sincerely,
>>>>>> Y.T.
>>>>>>
>>>>>> 2017-10-26 16:15 GMT-04:00 ｔyuta <y0u1t1a5.t at gmail.com>:
>>>>>>
>>>>>>> Hi Suoqing Ji,
>>>>>>>
>>>>>>> Thanks! I'll try that.
>>>>>>>
>>>>>>> 2017-10-26 15:59 GMT-04:00 Suoqing Ji <suoqing at physics.ucsb.edu>:
>>>>>>>
>>>>>>>> Hi t yuta,
>>>>>>>>
>>>>>>>> What you can do is to get the data as numpy array directly from the
>>>>>>>> profiles you’ve generated, and sum them up by yourself. You can follow the
>>>>>>>> last example at: http://yt-project.org/docs
>>>>>>>> /dev/visualizing/plots.html#d-profile-plots
>>>>>>>>
>>>>>>>> For each dataset, you can have one profile:
>>>>>>>> profile = plot.profiles[0]
>>>>>>>>
>>>>>>>> Then you can sum up profile[‘cell_mass'] over different datasets,
>>>>>>>> and plot it against profile.x
>>>>>>>>
>>>>>>>> Best wishes,
>>>>>>>> —
>>>>>>>> Suoqing Ji
>>>>>>>> Ph.D Candidate
>>>>>>>> Department of Physics
>>>>>>>> University of California, Santa Barbara
>>>>>>>> http://physics.ucsb.edu/~suoqing
>>>>>>>>
>>>>>>>> On Oct 26, 2017, 12:51 PM -0700, ｔyuta <y0u1t1a5.t at gmail.com>,
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Dear yt-users,
>>>>>>>>
>>>>>>>> Hi, I want to ask you about profile plot. I'm analyzing Enzo data,
>>>>>>>> and I'm trying to plot metallicity-cellmass of different 12 datasets into
>>>>>>>> one curve. I can plot 12 curves on one image, but that's not what I want.
>>>>>>>> Does anyone know some good way, or links to documentation?
>>>>>>>> I've attached the 12-curve plot. I want to sum up all the curves to
>>>>>>>> get one curve.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Y.T.[image: 埋め込み画像 2]
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/d88ef408/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 02C67ABD76E846418AE113C0B10469D2.png
Type: image/png
Size: 88454 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/d88ef408/attachment-0002.png>
More information about the yt-users
mailing list