[yt-users] profile plots combining different datasets
Nathan Goldbaum
nathan12343 at gmail.com
Fri Nov 3 16:05:42 PDT 2017
So what's happening is that when you're save_as_dataset at the end of your
notebook, you save data with incompatible shapes. In particular, 'density'
and 'temperature' have shape (128) while 'cell_mass' has shape (128, 128).
We should probably error out in cases like this because the resulting
dataset is corrupt - all fields in a single dataset should have the same
shape. The error you're seeing happens when yt tries to create an empty
array to hold the cell_mass data from your saved dataset by using the shape
of one of the bin fields. This leads to an error because the cell_mass
field in your saved dataset has a different shape.
One thing that might be immediately helpful is to save your individual
profile objects you want to sum, reload those, and then combine the data?
You can save profile objects with save_as_dataset:
http://yt-project.org/docs/dev/analyzing/saving_data.html#profiles
Another option would be to eschew using the ytdata frontend entirely and
just save the density, temperature, and cell_mass profile fields you want
save using e.g. h5py and then construct the plot you want to make using
those data.
It's not clear to me what you ultimately want to do with your saved data -
maybe if you explain that in a little more detail I can suggest what you
might want to do to restructure your workflow.
On Fri, Nov 3, 2017 at 5:53 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
> I tried on "Isolatedgalaxy" data. I made all the outputs to comments.
>
> https://gist.github.com/Yuta-Tarumi/69ed0428db92553cd4f8ef93924290
> 36#file-phase-plots-summing-up
>
>
>
> 2017-11-03 18:29 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>
>> So the error is coming from inside yt. Somehow yt is creating an array
>> with shape (128) and trying to fill it with values from an array with shape
>> (128, 128), causing numpy to complain. Unfortunately I'm still not sure
>> what the underlying issue is just from looking at your code and the
>> traceback. It's possible it's a yt bug, it's also possible that it's an
>> issue in your notebook.
>>
>> Is there any chance you can make a script I can run that triggers the
>> issue? That will make it much easier to understand what's going wrong. You
>> could also just share your full notebook.
>>
>> If you can, please make use of a dataset from yt-project.org/data in
>> your script. If that's not possible, you can also upload a dataset to the
>> yt curldrop using the "yt upload" command line tool:
>>
>> $ yt upload /path/to/dataset.tar.gz
>>
>> and then include the link to the dataset that yt upload will print out
>> once it's finished uploading the data.
>>
>> -Nathan
>>
>> On Fri, Nov 3, 2017 at 5:15 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>>
>>> Dear Nathan,
>>>
>>> Sorry for the incompleteness.
>>> This is the error message from jupyter notebook:
>>>
>>> ValueErrorTraceback (most recent call last)<ipython-input-63-383c90a16917> in <module>() 11 fields=["cell_mass"], 12 weight_field=None,---> 13 extrema=dict(density=(1e-30, 1e-20), temperature=(1e1, 1e8)) 14 ) 15
>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in create_profile(data_source, bin_fields, fields, n_bins, extrema, logs, units, weight_field, accumulation, fractional, deposition) 1093 setattr(obj, "fractional", fractional) 1094 if fields is not None:-> 1095 obj.add_fields([field for field in fields]) 1096 for field in fields: 1097 if fractional:
>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in add_fields(self, fields) 125 citer = self.data_source.chunks([], "io") 126 for chunk in parallel_objects(citer):--> 127 self._bin_chunk(chunk, fields, temp_storage) 128 self._finalize_storage(fields, temp_storage) 129
>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _bin_chunk(self, chunk, fields, storage) 565 566 def _bin_chunk(self, chunk, fields, storage):--> 567 rv = self._get_data(chunk, fields) 568 if rv is None: return 569 fdata, wdata, (bf_x, bf_y) = rv
>>> /home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _get_data(self, chunk, fields) 253 for i, field in enumerate(fields): 254 units = chunk.ds.field_info[field].output_units--> 255 arr[:,i] = chunk[field][filter].in_units(units) 256 if self.weight_field is not None: 257 units = chunk.ds.field_info[self.weight_field].output_units
>>> ValueError: could not broadcast input array from shape (128,128) into shape (128)
>>>
>>>
>>> 2017-11-03 17:08 GMT-04:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>
>>>> Can you share the full traceback along with the error message you're
>>>> seeing? You can use gist.github.com to avoid cluttering your e-mail.
>>>>
>>>> On Fri, Nov 3, 2017 at 3:54 PM, ｔyuta <y0u1t1a5.t at gmail.com> wrote:
>>>>
>>>>> Dear yt-users,
>>>>>
>>>>> Hi, I want to ask you about phase plot. Some days ago, I asked how to
>>>>> sum up different datasets to make one profile plot, and it worked fine.
>>>>> This time, I tried to do the same thing in phase plot, but it did not work
>>>>> fine. Could you help me find out what the problem is?
>>>>>
>>>>> (inside for loop):
>>>>> profile_medium = yt.create_profile(
>>>>> data_source=medium_region,
>>>>> bin_fields=["density", "temperature"],
>>>>> fields=["cell_mass"],
>>>>> n_bins=[128, 128],
>>>>> units=dict(),
>>>>> logs=dict(),
>>>>> weight_field=None,
>>>>> extrema=dict(density=(1e-30, 1e-20), temperature=(1e1,
>>>>> 1e8))
>>>>> )
>>>>> plot_med = yt.ProfilePlot.from_profiles(profile_medium)
>>>>> profile_med = plot_med.profiles[0]
>>>>> profiles_sum_med = profiles_sum_med + profile_med["cell_mass"]
>>>>>
>>>>>
>>>>> my_data_med = {"density": profile_med.x,
>>>>> "temperature": profile_med.y,
>>>>> "cell_mass": profiles_sum_med}
>>>>> fake_ds_med = {"current_time": yt.YTQuantity(10, "Myr")}
>>>>> yt.save_as_dataset(fake_ds_med, "phaseplots_med_all.h5", my_data_med)
>>>>>
>>>>> ds = yt.load("phaseplots_med_all.h5")
>>>>> ad = ds.data
>>>>>
>>>>> profile = yt.create_profile(ad,
>>>>> ["density", "temperature"],
>>>>> n_bins=[128, 128],
>>>>> fields=["cell_mass"],
>>>>> weight_field=None,
>>>>> extrema=dict(density=(1e-30, 1e-20),
>>>>> temperature=(1e1, 1e8))
>>>>> )
>>>>>
>>>>> and I got this error:
>>>>>
>>>>> could not broadcast input array from shape (128,128) into shape (128)
>>>>>
>>>>>
>>>>> Sincerely,
>>>>> Y.T.
>>>>>
>>>>> 2017-10-26 16:15 GMT-04:00 ｔyuta <y0u1t1a5.t at gmail.com>:
>>>>>
>>>>>> Hi Suoqing Ji,
>>>>>>
>>>>>> Thanks! I'll try that.
>>>>>>
>>>>>> 2017-10-26 15:59 GMT-04:00 Suoqing Ji <suoqing at physics.ucsb.edu>:
>>>>>>
>>>>>>> Hi t yuta,
>>>>>>>
>>>>>>> What you can do is to get the data as numpy array directly from the
>>>>>>> profiles you’ve generated, and sum them up by yourself. You can follow the
>>>>>>> last example at: http://yt-project.org/docs
>>>>>>> /dev/visualizing/plots.html#d-profile-plots
>>>>>>>
>>>>>>> For each dataset, you can have one profile:
>>>>>>> profile = plot.profiles[0]
>>>>>>>
>>>>>>> Then you can sum up profile[‘cell_mass'] over different datasets,
>>>>>>> and plot it against profile.x
>>>>>>>
>>>>>>> Best wishes,
>>>>>>> —
>>>>>>> Suoqing Ji
>>>>>>> Ph.D Candidate
>>>>>>> Department of Physics
>>>>>>> University of California, Santa Barbara
>>>>>>> http://physics.ucsb.edu/~suoqing
>>>>>>>
>>>>>>> On Oct 26, 2017, 12:51 PM -0700, ｔyuta <y0u1t1a5.t at gmail.com>,
>>>>>>> wrote:
>>>>>>>
>>>>>>> Dear yt-users,
>>>>>>>
>>>>>>> Hi, I want to ask you about profile plot. I'm analyzing Enzo data,
>>>>>>> and I'm trying to plot metallicity-cellmass of different 12 datasets into
>>>>>>> one curve. I can plot 12 curves on one image, but that's not what I want.
>>>>>>> Does anyone know some good way, or links to documentation?
>>>>>>> I've attached the 12-curve plot. I want to sum up all the curves to
>>>>>>> get one curve.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Y.T.[image: 埋め込み画像 2]
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/f8b05af6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 02C67ABD76E846418AE113C0B10469D2.png
Type: image/png
Size: 88454 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20171103/f8b05af6/attachment-0001.png>
More information about the yt-users
mailing list