[Yt-dev] Embedded yt and IPython

Matthew Turk matthewturk at gmail.com
Sat Jun 6 13:01:50 PDT 2009


Hi Stephen,

Thank you very much for the kind words!  I hope we can do some fun
stuff with this.  I've committed my first pass at the magic functions,
and I've sent out some inquiries about proxy objects for matplotlib
figures and axes, which I think will be necessary for local plotting
of remote data.

> You say you don't want any, but have you put any thought on how you might do it if you had to? An inline halo finder is something that many people want, me in particular. That would require both spatial decomposition and some associated communication.

I have thought about it.  I think we could do this in a couple
different ways -- one of which would be a multithreaded approach where
DataQueues listen for requests and supply them as necessary.  This
shouldn't bottleneck things too much, but I'm not sure it will work.

My thought process was that any domain decomp would serve to probably
double the memory usage of any individual process -- and I want to try
to avoid that.  But perhaps it's unavoidable.  I'll see if I can get a
working implementation of this going sometime over the next little
while.

> Alternatively, would it be possible to call yt to run on data on disk in this model? One would just have enzo write out to disk for every time step they wanted to find haloes, run parallel HOP off the data on disk? Of course, this could just be accomplished by running enzo/yt/enzo/yt using a shell script as the batch job.

One thing we could do is to write out the fields necessary for each
timestep that we want to examine.  We have access to essentially
everything necessary to output the data to disk -- in fact, YT could
serve as a selective-output function.  You could envision outputting
just the particles on each processor, for instance, and avoiding
anything else.  This could be accomplished (of course) in C as well,
but I'm just suggesting it as a point of reference.

-Matt



More information about the yt-dev mailing list