[yt-dev] Request comments/vote on use of notebooks in yt docs

Sat Oct 26 13:16:02 PDT 2013

  

I'm personally +1 on Brian's email on this subject 

On Sat, 26 Oct
2013 16:08:45 -0400, Brian O'Shea wrote: 

> Hi folks,
> 
> I'm sorry
for not responding to Nathan and Cameron's emails from a couple of days
ago in a timely fashion.
> 
> I believe that people who use yt to do
data analysis in a production setting are primarily using scripts, as
Britton said. This is certainly true in my own research group, and I
think generally true when people are doing batch analysis of large
datasets (and/or lots of datasets) on supercomputers. If the cookbook is
truly meant to help users that are new to yt, it makes sense to present
the cookbook code in the manner that most closely approximates how a new
user will actually use it, so presenting the code as scripts in the
cookbook makes the most sense to me. Also, the current implementation of
the cookbook is relatively compact, so it's easy to scroll through and
extract what one needs, which is useful for the OTHER use of the
cookbook, which is for more advanced users to quickly zip through and
remind themselves of how the heck a piece of functionality works. Using
iPython notebooks would make the cookbook much longer, or force us to
break it up into a bunch of segments. Either way, it would also be
harder to quickly parse through because the actual commands the user
cares about are embedded inside quite a bit of extraneous (for this
purpose) text. I'm concerned that this mailing list's enthusiasm for the
new, shiny thing, while commendable in many ways, is driving us away
from the original purposes of the cookbook, at least as I understand
them.
> 
> That said, I do think that iPython notebooks are very cool,
and I intend to use them as a teaching tool in a class I'm teaching next
semester. It's also pretty clear which way the wind is blowing on this
mailing list, so putting iPython notebooks into the cookbook in some way
appears to be inevitable. So, why not flip it around and have the script
be the standard, compact way of viewing the data, but one can click on a
link to take the user to a notebook that they can look at, and which can
be downloaded? That allows the cookbook to be relatively easy to parse,
and also lets people who want to see the notebooks do so.
> 
> --Brian
(though possibly "Grampa Brian" is a more appropriate signoff, given the
tone of my email)
> 
> On Sat, Oct 26, 2013 at 12:09 PM, Britton Smith
wrote:
> 
>> There seems to be enough utility in having both the
notebooks and at least some way of downloading pure python scripts. On
the one hand, the notebooks are illustrative for both yt usage and for
learning ipython notebooks themselves. As a teaching tool, they seem
great. On the other, I think the primary usage of yt for analysis is
through scripting and we should try to preserve that notion and make
these available as seeds for new users looking to build their own
scripts. I think that presenting the recipes with the notebook, but with
a button to download as a script nearby, would be an option that
satisfies everyone. 
>> Britton 
>> 
>> On Thu, Oct 24, 2013 at 9:54 PM,
Chris Malone wrote:
>> 
>>> I'm +1 on this, with the possibility of
providing also just the .py files from nbconvert for those that
won't/can't use notebooks. 
>>> Chris 
>>> 
>>> On Thu, Oct 24, 2013 at
12:28 PM, Sam Skillman wrote:
>>> 
>>>> Hi all, 
>>>> I think if we
don't take this opportunity to use notebooks during this major update to
the documentation, we'll be kicking ourselves in about a year for
missing it. I'd like to voice my support for the following layout: 
>>>>
1) Write the cookbook examples in a notebook, annotating it with
comments and reasoning in markdown cells. Use some amount of conventions
for data loading so that with minimal work users could change the path
to the data and run themselves. 
>>>> 2) Display the notebooks in the
docs 
>>>> 3) Allow for download of both a stripped down (no images
included) .ipynb and nbconverted script. 
>>>> The neat thing is that
now you have all these .ipynb files in the doc repo. It would be stupid
simple to then show people how to go to that folder, launch yt notebook,
you can then interactively execute the examples after pointing to the
data locations. This would be really really nice in my opinion. 
>>>>
Cheers, 
>>>> Sam 
>>>> 
>>>> On Wed, Oct 23, 2013 at 10:02 PM,
Elizabeth Tasker wrote:
>>>> 
>>>>> Hi everyone, 
>>>>> I agree with
Brian. I've only recently discovered python notebooks and I love them
for my own analysis and when I'm looking at data together with my
students and we're exploring data sets, but I feel they are unnecessary
cumbersome for code examples. 
>>>>> Most of the cookbook snippets have
only one simple product (image, plot etc), so you don't gain a great
deal by showing the results from each line in the notebook. 
>>>>>
Additionally, however great notebooks are, they're not as handy for
tasks that you need to perform repeatedly or for writing full length
analysis scripts. In my opinion, using yt as a front end to more
detailed analysis is one of its major strengths. If you take away the
cookbook scripts, we lack examples of yt in python programs. 
>>>>> The
notebooks are also not quite as easy to use as a downloaded code
snippet. 
>>>>> Elizabeth 
>>>>> 
>>>>> On Oct 23, 2013, at 7:23 PM,
Brian O'Shea wrote: 
>>>>> 
>>>>>> Hi Cameron,
>>>>>> 
>>>>>> For what
it's worth, as a user of yt I find the current cookbook format to be
incredibly useful. I don't think notebooks would add to the utility -
it's easy enough for me to download the script and load it into a
notebook on my own machine if that's what I want to do. It definitely
seems that the challenges (and possible downsides) substantially
outweigh the benefits, at least for me and my usage patterns.
>>>>>>

>>>>>> --Brian
>>>>>> 
>>>>>> On Tue, Oct 22, 2013 at 4:13 PM, Cameron
Hummels wrote:
>>>>>> 
>>>>>>> Hey everyone, 
>>>>>>> The documentation
sprint is next Monday and Tuesday for those of you who want to
participate. I'll send out another email regarding that in the next day
or so. 
>>>>>>> In preparation for that, though, I wanted to request
input from the developer community on something related to the docs.

>>>>>>> Right now, the cookbook page contains a lot of recipes for
doing various things, and I think it is hugely beneficial to the
community to maintain this (I personally use this page a lot too!).
However, with the advent of ipython notebooks over the last year, we are
faced with a question: should we move toward incorporating more
notebooks into our documentation, and specifically, do you we want to
transfer the existing cookbook to a series of notebooks for each task?

>>>>>>> Benefits: 
>>>>>>> --Portability: users can download an entire
notebook for both viewing how it should work as well as being able to
execute it locally on their own datasets 
>>>>>>> --Illustrative:
Interim steps in a cookbook can produce output that can show up inside
the notebook, instead of being a single script which generates an
image/output at the end (as is the case in the current paradigm)

>>>>>>> --Narrative: notebooks provide more space for narrating each
step, instead of confining any narrative to comments in the recipe
itself 
>>>>>>> Disadvantages: 
>>>>>>> --Work: it is going to take a
decent amount of work to move all of the recipes over from the existing
cookbook to individual notebooks 
>>>>>>> --Bulking of repo: In the
current paradigm, images associated with each recipe are generated
dynamically on the server by executing each script, thereby minimizing
the number of files that need to be tracked by mercurial. By moving to a
notebook with images that are embedded in each notebook, we'd
potentially increase the footprint of the repository substantially,
especially if there were frequent updates of individual recipes.

>>>>>>> I also like the yt bootcamp notebooks that Matt put together a
year ago. I think they are great for getting new users up to speed on
how to use various aspects of the code. Perhaps this notebook could make
its way into the beginning of the cookbook for a more streamlined
approach to the documentation? 
>>>>>>> So now is your chance to vote:

>>>>>>> Move cookbook to ipython notebooks? +/- 0-1? 
>>>>>>> Move yt
bootcamp to cookbook? +/- 0-1? 
>>>>>>> Comments? Suggestions? 
>>>>>>>
Cameron
>>>>>>> -- 
>>>>>>> Cameron Hummels 
>>>>>>> Postdoctoral
Researcher 
>>>>>>> Steward Observatory 
>>>>>>> University of Arizona

>>>>>>> http://chummels.org [1] 
>>>>>>>
_______________________________________________
>>>>>>> yt-dev mailing
list
>>>>>>> yt-dev at lists.spacepope.org [2]
>>>>>>>
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org [3]
>>>>>>
_______________________________________________
>>>>>> yt-dev mailing
list
>>>>>> yt-dev at lists.spacepope.org [5]
>>>>>>
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org [6]
>>>>>

>>>>> _______________________________________________
>>>>> yt-dev
mailing list
>>>>> yt-dev at lists.spacepope.org [8]
>>>>>
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org [9]
>>>>

>>>> _______________________________________________
>>>> yt-dev
mailing list
>>>> yt-dev at lists.spacepope.org [11]
>>>>
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org [12]
>>>

>>> _______________________________________________
>>> yt-dev mailing
list
>>> yt-dev at lists.spacepope.org [14]
>>>
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org [15]
>> 
>>
_______________________________________________
>> yt-dev mailing
list
>> yt-dev at lists.spacepope.org [17]
>>
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org [18]

 
Links:
------
[1] http://chummels.org/
[2]
mailto:yt-dev at lists.spacepope.org
[3]
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
[4]
mailto:chummels at gmail.com
[5] mailto:yt-dev at lists.spacepope.org
[6]
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
[7]
mailto:bwoshea at gmail.com
[8] mailto:yt-dev at lists.spacepope.org
[9]
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
[10]
mailto:tasker at astro1.sci.hokudai.ac.jp
[11]
mailto:yt-dev at lists.spacepope.org
[12]
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
[13]
mailto:samskillman at gmail.com
[14] mailto:yt-dev at lists.spacepope.org
[15]
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
[16]
mailto:chris.m.malone at gmail.com
[17]
mailto:yt-dev at lists.spacepope.org
[18]
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
[19]
mailto:brittonsmith at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20131026/1c63c60e/attachment.html>