[yt-dev] 2012 Community Survey Summary

Casey W. Stark caseywstark at gmail.com
Tue May 29 10:19:19 PDT 2012

Hi all. Just a few quick comments.

> yt-isms, which I'd like to see disappear --
> making a "collection" of a single image, "c" implicitly indicating
> domain center, how do you know which fields are available, blah blah.

This is the most important thing to me. Some broad refactor for this would
really help the code quality and maintainability. I think it would really
clean up my yt scripts as well.

I'd like it if I could make a plot with nice-looking tick marks, labels,
> and colorbar by manipulating a plotting window and without writing a line
> of matplotlib code or python code.

While I don't exactly like the matplotlib API, I think it could get worse
if we try to wrap it all up. I have found that I would rather have full
control, even if it gets messy. If I really want to tinker with a plot, I
would rather use matplotlib directly.

Right now, random API references tend to appear at the top of the search
> results while the narrative docs appear at the bottom which is very
> confusing.

Completely agree. This is one of the main reasons I stopped using sphinx.
Even for the python docs, I have found it easier to google the site than to
use the sphinx search.


On Tue, May 29, 2012 at 7:59 AM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Nathan and Elizabeth (and everyone else),
> Thanks very much for your comments.  My delay in replying was to allow
> ample time for anyone else interested to chime in.  (And anyone still
> thinking of doing so, please do!)
> On Wed, May 23, 2012 at 3:00 AM, Nathan Goldbaum <goldbaum at ucolick.org>
> wrote:
> > Hi all, just a few comments:
> >
> > I think the main issue is that yt is at its heart an extension of the
> python language.  If a new user is not familiar with python initially then
> they will have a much more difficult time following the docs and learning
> how to control yt.  At the same time, the fact that yt is embedded in a
> high level programming language means that it is very powerful and easily
> extensible to someone familiar with python and willing to wade through the
> decently-sized codebase.
> I think that is definitely true.  And in the past, we've tried to ease
> the process of getting people on board with Python -- through the
> short orientation document, links to other places, and so on, but I
> think trying to address this on our own may be a losing battle.
> Python is rapidly gaining mindshare in astronomy and so perhaps this
> will be less and less of a problem as time goes on.
> This brings up Elizabeth's point, though, that there *are* easier
> introductions to yt than strictly through the Python interface.  A
> week and a half ago I gave a morning lecture in Sapporo, with Britton,
> John and Elizabeth, on yt to a room full of people who hadn't done
> much or any Python programming.  (There were a handful of Ruby
> programmers, though, which was interesting.)  During that lecture, I
> remembered just how weird some things are.  If you're not used to
> Python, or even if you are but aren't used to yt, the following couple
> of lines can be really daunting:
> from yt.mods import *
> pf = load("DD0076/DD0076")
> pc = PlotCollection(pf, "c")
> pc.add_slice("Density", "x")
> pc.save()
> Yes, each line makes sense by itself, but taken as a whole to swallow
> with no exposure to the idioms or syntax of Python this can be
> frustrating.  (And, let's not forget that this includes a lot of
> organically developed yt-isms, which I'd like to see disappear --
> making a "collection" of a single image, "c" implicitly indicating
> domain center, how do you know which fields are available, blah blah.)
> >
> > Elizabeth's point about Reason in the docs is worth thinking about.  I
> don't think Reason is quite there yet, but if we can point to the gui in
> the docs before we even mention a python tutorial, I think far fewer people
> will be scared off in the same way as the negative commenter.
> I agree.  I spent some time last week in a jetlagged stupor evaluating
> where I thought Reason was weak, inspecting the code, and playing with
> it a little bit.  I've come up with a plan, but I don't see it being
> implementable in the next couple weeks, but rather sometime over the
> next couple months.
>  * Add a file open dialog.  I did this last week, and I'm actually
> quite pleased with how it turned out.  Now you can specify files on
> the command line, open from a window, or use -f to load parameter
> files in Reason.
>  * Refactor the javascript code.  This has no direct implications for
> the user, but it will provide much more maintainable software.  This
> includes:
>  * Migrate to ExtJS4
>  * Move to using classes provided by ExtJS for widgets, etc etc
> (i.e., make creating a new widget *way* easier)
>  * Reorganize .js files
>  * Make ExtJS get loaded from a single .zip file rather than an
> unzipped directory in $YT_DEST/src/
>  * Add a few more widgets for 3D viz and improve the existing ones
>  * Change the view to be parameter-file centric, rather than notebook
> centric.  This should make choosing fields, making new widgets, etc
> etc much easier.  The notebook will still be available, but rather
> than having the user right-click on the pf, present a pf widget that
> has all the available widgets as well as stats and whatnot about the
> parameter file.
>  * Test out linked view widgets (we used to have this in Reason v1).
> See http://arxiv.org/abs/1205.4747 for more details.
> Right now it's all a bit widget-centric, but I think moving to more
> parameter file -centric would improve things.  This would then nicely
> migrate into time series-centric analysis...  Way down the line we can
> consider using the IPython infrastructure.  I thought we could try to
> do that on a much shorter term, but I was wrong.
> Jeff has a set of things he'd like to polish about reason, but I'm not
> going to try to summarize or list them here.  I think it just needs
> lots of polish.
> >
> > For that to happen It should be much easier to write new analysis
> modules for reason.  If there were an API to add new functionality in the
> same way as we can add new derived quantities or time series analysis tasks
> in pure python, I think reason would get much more attention.
> I like this idea.  It would be great if we could have the main widgets
> and then a set of contributed analysis modules.  Like, imagine we had
> the usual slices and plots and stuff available, and then made it easy
> to add on a new "canned analysis" that prompted the user for some
> inputs, applied itself, and then returned either a display or a data
> object back.  This could probably be done with introspection and
> annotations.  I took a stab at something like this a while back:
> http://bitbucket.org/MatthewTurk/yt.web_interactors
> but that was in a standanlone environment.  You decorated your
> function, and then yt figured out how to create widgets it showed to
> the user in a web browser.  You set values, clicked "Go" and it
> returned a new value back to you.
> >
> > One option that may be worth thinking about is a gui plotting interface
> separate from reason.   I know we want the new plotting interface to expose
> the matplotlib backend more readily - something I am completely for - but I
> think matplotlib can be quite confusing for newcomers.
> Sigh ... yeah, I see where you're coming from with wanting a native
> interface.  We have a simple interface to matplotlib that can do the
> zooms and whatnot.  But I'm not sure what to do here, as even thinking
> about dealing with GUI toolkits gives me headaches.  Can you elaborate
> a bit on what you'd want to see?
> >
> > I'd like it if I could make a plot with nice-looking tick marks, labels,
> and colorbar by manipulating a plotting window and without writing a line
> of matplotlib code or python code.  It would also be cool to set up a plot
> interactively and save a script that could reproduce that plot or press a
> button to interactively make the exact same plot with a different dataset.
>  This is all stuff that visit does and I think there are some lessons to be
> learned with how visit handles data, even if the plotting and scripting is
> sometimes pretty awful (sorry visit developers…).
> There's definitely a long list of things we could learn from visit.  I
> see what you mean about making simple plots.
> Perhaps we should envision an interface where we expose the matplotlib
> interface readily, but we also provide a number of commonly done
> idioms, named with plaintext names.  For instance, it's pretty common
> to want to stagger the ticks every other.  So, make that a recipe that
> can be applied to the matplotlib interface.
> >
> > One other thing that I've mentioned before that would make the docs much
> easier to use for a newcomer would be an improved search.  Right now,
> random API references tend to appear at the top of the search results while
> the narrative docs appear at the bottom which is very confusing.
> I'm going to try to improve this.  Britton and I went over some of the
> docs, and I think we're going to try to improve the examples/cookbook,
> and also remove the API docs from the search function.  This is a
> common complaint.
> >
> > Thanks for posting the results of the survey Matt.  I'm looking forward
> to working on this stuff this summer :)
> >
> > Cheers,
> >
> > Nathan
> >
> > On May 22, 2012, at 9:20 PM, Elizabeth Tasker wrote:
> >
> >> Hi everyone,
> >>
> >> I have a couple of comments related to the negative comment on the yt
> survey. In particular, there is one part I agree with and one I do not:
> >>
> >> The comment about yt being heavily cosmology based I do not agree with.
> As a non-cosmo simulator --one who still intends to torch a fellow
> developer's house for declaring all my research 'test problems' at a user
> meeting-- I use yt for all my simulation analysis and use a wide range of
> its capabilities. In fact, the only part I can think that I do not use is
> the halo finder, but I do use connected sets which is a similar idea for
> gas.
> >>
> >> I therefore feel this comment is not part of the general consensus and
> can be ignored.
> :-)
> >>
> >> However, the comment about the documentation being from the user's
> rather than the programmer's, point of view is interesting and think worth
> some thought. For a while, I skirted around yt, aware that it was powerful
> but baulking at the effort needed to use it comfortably. Last summer, I had
> to write an entirely new analysis program for my simulations and I bit the
> bullet and starting working extensively with yt and python. I've never
> picked up an IDL script since, but there was definitely an energy bump. I
> also think that I do find the documentation **significantly** clearer now I
> approach it from a programmer's perspective.
> >>
> >> I have a couple of suggestions of what I think might help with this:
> >>
> >> (1) The quickest option is to keep everything the same but add more
> actual examples. For instance, the routine call, e.g.
> >>
> >> contour(self, field, ncont=5, factor=4, take_log=False, clim=None,
> plot_args=None):
> >>
> >> is confusion for a non-programmer. When I first came to yt, I spent a
> while with thoughts like:
> >>
> >> What is 'self'? Do I need to pass it like in C++ with Grid-> ?
> >>
> >> Although it's slightly painful, I think it would be much improved if
> there was an actual call example for each one of the callbacks and other
> tools. So we'd have:
> >>
> >> contour(self, field, ncont=5, factor=4, take_log=False, clim=None,
> plot_args=None):
> >> (This is a proxy for ContourCallback.)
> >> pc.modify["contours"]("Density", ncont=1, take_log=False)
> >> Add contours in field to the plot. ncont governs the number of contours
> generated, factor governs the number of points used in the interpolation,
> take_log governs how it is contoured and clim gives the (upper, lower)
> limits for contouring.
> This is a great suggestion, and I will work to implement this.  The
> organization also needs some work, but I'm the wrong person for that
> job.
> I'll also see if we can have example images below each of these.
> >>
> >>
> >> (2) My second suggestion involves the gui and concerns the way yt is
> presented. Currently, we talk about scripts and then mention there is a
> gui. How about we do it the other way around? So a new user's first port of
> call should be the gui. Then, we get to the section "beyond the gui" and
> talk about scripting and cookbook-ing and then finally onto full simulation
> analysis, i.e. writing whatever analysis package you like in python, but
> making use of the routines yt has.
> >>
> >> For instance, in the workshop we just had at Hokudai, we could have
> stood up and brought up the yt gui before anything else. *Then* gone onto
> scripts.
> >>
> >> The docs could be rearranged in a similar fashion.
> This is an interesting idea that had not occurred to me.  While Nathan
> brings up the notion that Reason is not ready, I think it is perhaps
> worthwhile to figure out in what way it isn't, address those, and then
> move forward with making this the standard method of interfacing with
> yt for newcomers.  I'm going to be preparing a screencast of "yt for
> beginners", just a short little demo, and I'll do it using Reason
> first.  I think it would ease the transition, and also help those
> people new to Python to avoid having the big daunting task of their
> first experience being to write a script.  While of course that's
> where (in an ideal world) they'd end up, maybe it shouldn't be where
> they start.
> >>
> >> I think this might provide a much faster way for new users to start
> getting into yt and make the gui of much more interest. Currently, people
> come to it after they learn to script at which point, its value is much
> less because it's inevitably more limited.
> Good point.
> Again, thanks to everybody for their feedback, and any other comments
> would be greatly appreciated.
> Best,
> Matt
> >>
> >>
> >> Elizabeth
> >>
> >> PS today's google-doodle (or possible tomorrow's for the USA) is
> awesome.
> >>
> >>
> >>
> >>
> >> !DSPAM:10175,4fbc659f1748213191834!
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> >>
> >> !DSPAM:10175,4fbc659f1748213191834!
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20120529/640cc2b2/attachment.html>

More information about the yt-dev mailing list