[yt-dev] Fwd: [IPython-dev] Data viz in the notebook with JS

[Matthew Turk]

Hi Fernando,

On Mon, Jun 4, 2012 at 12:34 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Howdy,
>
> On Sat, Jun 2, 2012 at 2:47 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>> Disadvantages:
>>
>>  * Unclear amount of our REPL/Notebook code will have to be rewritten
>>  * The IPython notebook's storage format (SQLite) may not play nicely
>> with all Lustre systems
>
> You mean JSON?  IPython uses sqlite only for the user history, and
> that happens even in the terminal.  The notebook makes no additional
> use of SQLite, the notebook files themselves are JSON files stored
> directly on disk via regular filesystem mechanisms.

Ah, I guess I was referring to Doug's issue here:

http://mail.scipy.org/pipermail/ipython-user/2012-May/010107.html

While Doug doesn't use yt, his issue was on a system we deploy on at
SLAC.  We've also had trouble in the past with SQLite on Lustre
machines because of file locking, and so I guess what I was getting at
was that we'll need to be aware of this possibly stumbling block if
IPython becomes part of a piece of core functionality.

>
>>  * Possible impedance mismatch between JS libraries used in Notebook and ExtJS4
>
> Very possible.
>
>>  * My understanding of how WebSockets work doesn't cover how they work
>> when SSH tunneling to bind to a socket, so I am putting potential
>> glitches from this approach under "disadvantage"
>
> That should be fine.  We've run qtcosoles and entire clusters with
> zeromq communications forwarded over SSH.  That's not to say there
> couldn't be some issues with peculiar network/firewall setups (we've
> recently seen a hard to diagnose such case at a supercomputing
> facility), but by and large things have been OK on that front.

Okay, that's very good to hear!  Thanks very much for clearing these things up.

I've now had time to study the way the widget code and the new
refactored JS in the IPython html notebook are set up, and I think
they're both pretty awesome and would be relatively easy to integrate
our stuff into, with probably quite large payoffs in terms of
maintainability and functionality.

If anybody wants to work on this, I think it would be straightforward enough:

 * Subclass or reimplement the notebook launching in IPython to launch
slightly different HTML templates for the notebook itself
 * Replace our notebook/cell handling code with the CodeMirror-based
JS+HTML from IPython (which would be a big step up in functionality by
itself)
 * Make the notebook menus and display play nice, display-wise
 * Swap out RPC calls to the yt engine with kernel evaluation calls

-Matt

>
> Cheers,
>
> f
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org