[yt-dev] Profile Plotting

Tue Dec 11 14:46:07 PST 2012

Hi all,

As a first step to this, I've submitted a PR (which needs iteration
before acceptance) that closes a long-standing fork of mine, where I
rewrote the actual underlying profiles to be much simpler.  The next
step in this process will be rewriting the profile_plotter.py code,
and then updating the YTEP.

As noted in the PR for the profiles, because this has an API change,
the (up for discussion) plan will be:

1) Introduce new, backwards incompat objects in 2.5, and swap out all
calls to the old profile objects internal to yt.  Raise deprecation
warnings for BinnedProfile* instantiation.
2) In 3.0, remove old BinnedProfile* objects.

The YTEP will expand to include this.  Does anyone object to this?

-Matt

On Wed, Dec 5, 2012 at 9:13 AM, Matthew Turk <matthewturk at gmail.com> wrote:
> Hi Nathan, John and Britton,
>
> Your points make sense.  I'll reduce the complexity, try to move it
> out from being quite as backend-neutral as it is, and then update with
> a new PR for the YTEP and some new code.
>
> Britton, I think you're right -- the success of the plot window was
> that it "just works."  And it's not worth the relatively small use
> case of pickling plots to add these additional levels of indirection.
>
> Nathan, your sketch of the API (inasmuch as it does look quite a lot
> like the plot window ;-) looks pretty good to me.  I think I can
> consolidate classes without much trouble, and use the
> validate/invalidate plot setup from the plot window.  In fact I may
> split those into a generic plot control area, to reuse them.
>
> -Matt
>
> On Tue, Dec 4, 2012 at 10:45 PM, Britton Smith <brittonsmith at gmail.com> wrote:
>> Following the question from the YTEP, should the new plotter objects
>> explicit depend on matplotlib, I think yes.  I think this is what has made
>> the PlotWindow objects so successful.  This is about convenience and I think
>> that's what matplotlib offers.  As it mentions in the alternative section,
>> people can still use the BinnedProfile objects if they want to, so we might
>> as well not try to make this too general and just go the most popular route.
>> The same thing goes for the third question in the YTEP about pickling.
>> There are already pretty easy ways of generating the profile data, so I
>> think it's better to just focus on the plot generation itself.
>>
>> Britton
>>
>>
>>
>> On Tue, Dec 4, 2012 at 8:08 PM, John ZuHone <jzuhone at gmail.com> wrote:
>>>
>>> I'm very much with Nathan on this one.
>>>
>>> John
>>>
>>> On Dec 4, 2012, at 7:47 PM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
>>>
>>> Hi Matt,
>>>
>>> I'm personally not a fan of the level of abstraction necessary to produce
>>> an MPL object and save it to disk (i.e. in the notebook linked in the YTEP:
>>> http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/8df3719ddfd248ce9a3f10ca38a4ad57.ipynb)
>>>
>>> One of the things I like about the plot window is that it's completely
>>> separate from the generation of the actual image buffers and depends on the
>>> FixedResolutionBuffer to do all the nasty work of touching the grids,
>>> pixelizing, and dealing with the real-world nature of the actual data.
>>>
>>> It would be nice if we had a similar situation on the profile/phase plot
>>> side of things.  We could use ProfilePlotter and PhasePlotter to generate
>>> the raw histogram data and then use something similar to the PlotWindow to
>>> generate the plots, save them to disk or display them in iPython notebooks.
>>>
>>> I guess I'm proposing another level of abstraction:
>>>
>>> pf = load(filename)
>>> dd = pf.h.all_data()
>>> pha = PhasePlot(dd, 'Density', 'Temperature')
>>>
>>> One could then access the MPL Figure object via something similar to
>>> PlotWindow:
>>>
>>> fig = pha.Figure
>>>
>>> and it would be just as easy to save the figure to disk:
>>>
>>> pha.save()
>>>
>>> I'll confess that I haven't looked at PhasePlotter or ProfilePlotter too
>>> carefully so it might be easier to just change them to match this API.  I
>>> also don't make profile plots very often so someone who uses them all the
>>> time should probably have more weight in this discussion.
>>>
>>> Cheers,
>>>
>>> Nathan
>>>
>>> On Dec 4, 2012, at 1:21 PM, Matthew Turk wrote:
>>>
>>> Hi everyone,
>>>
>>> I've put in a YTEP, which has been built here:
>>>
>>> https://yt.readthedocs.org/projects/ytep/en/latest/YTEPs/YTEP-0002.html
>>>
>>> Before it goes forward I'd like to hear feedback on what it describes
>>> and proposes, and then I will continue with implementing and shoring
>>> things up.  In particular, hearing how it could be made to provide as
>>> seamless an experience as the PlotWindow would be nice.
>>>
>>> -Matt
>>>
>>> On Mon, Dec 3, 2012 at 2:03 PM, Matthew Turk <matthewturk at gmail.com>
>>> wrote:
>>>
>>> On Mon, Dec 3, 2012 at 2:02 PM, Matthew Turk <matthewturk at gmail.com>
>>> wrote:
>>>
>>> Hi all,
>>>
>>>
>>> As I noted in my other email, there was one major feature we'd talked
>>>
>>> about for yt 2.5 that we never even really worked on.  Nathan and
>>>
>>> others pushed really hard on rethinking and redesigning the way image
>>>
>>> plots were made and handled back in the 2.4 series, which eliminated
>>>
>>> the main use case for the PlotCollection.  However, the remaining
>>>
>>> firewalls on PlotCollection are the 1-D and 2-D phase plots.
>>>
>>>
>>> There's been broad consensus that we need a new method for doing this,
>>>
>>> analogous to how PlotWindow serves to replace "add_slice" and
>>>
>>> "add_projection" from the PlotCollection.  Something that gets out of
>>>
>>> the way and lets people modify their plots as they see fit.
>>>
>>>
>>> A while back I implemented a first pass at this, which you can see in
>>>
>>> yt/visualization/profile_plotter.py:
>>>
>>>
>>>
>>> https://bitbucket.org/yt_analysis/yt/src/711e95aff04b54e6ccda7dfefb0b0ae17c3830a7/yt/visualization/profile_plotter.py?at=yt
>>>
>>>
>>> The idea here is that you create an instance of ProfilePlotter or
>>>
>>> PhasePlotter, which then "Does the right thing" and creates the
>>>
>>> necessary data objects and the like.  This object can then either
>>>
>>> create its own axes+figure in matplotlib, or it can deposit itself
>>>
>>> into an existing axes object.  It includes axis objects and plot
>>>
>>> containers that know how to plot themselves.
>>>
>>>
>>> I don't know that I necessarily like how this is done.  It's very
>>>
>>> declarative, and step by step, but I think it could be easier.  Here's
>>>
>>> an example:
>>>
>>>
>>> https://hub.yt-project.org/go/vlilvw
>>>
>>>
>>> That should read:
>>>
>>>
>>> https://hub.yt-project.org/nb/vlilvw
>>>
>>>
>>>
>>> One thing I think *is* quite nice is that the *plot* object is
>>>
>>> independent of the profile itself.  This makes it easier to pickle and
>>>
>>> unpickle things, and is the reason for the existence of the various
>>>
>>> sub-objects off of ProfilePlotter.  This is useful for the use case of
>>>
>>> making very large datasets into profiles, pickling the resulting
>>>
>>> plots, and modifying them later.
>>>
>>>
>>> So here are my specific questions:
>>>
>>>
>>> * Does this rough outline of how the object works seem good?  As in,
>>>
>>> making a plotter object, making it have a sub .plot object, and then
>>>
>>> dispatching that plot object to various backends?  (Probably nearly
>>>
>>> always matplotlib.)
>>>
>>> * Should we add on additional convenience operations?
>>>
>>> * How should it interface with existing profiles?
>>>
>>> * Should implementing this and finalizing the design be a blocker for 2.5?
>>>
>>>
>>> I will create and update a YTEP with the resulting discussion from
>>>
>>> this.  Nathan, Jeff and Britton -- I would very much appreciate your
>>>
>>> feedback, as I know you have experience with (respectively) plot
>>>
>>> windows, matplotlib, and large datasets being profiled.
>>>
>>>
>>> -Matt
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>>
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>