[Yt-dev] [yt-users] parallel data retrieval

[Matthew Turk]

This was not meant to be relevant to setting _processing.  _processing
exists so that sub-calls withing a given calculation of halo
properties don't block.  The original intent of the proxy class was to
avoid any per-processor changes in operation; that's why the
_processing flag still has to be set manually.  I mean, if you put the
particle dumping routines *as methods of the class itself* they should
work perfectly fine, because they'll be inside the proxy handling.
This is the path I would recommend.  If PyTables had parallel HDF5 IO,
we could look into how to write multiple at a time.  It doesn't, so we
can simply write a routine that accepts a filename, opens it for
append, writes, returns.

The proxy class was not designed to handled differently based on which
processor each group is on.  The idea here is to avoid that, and to
make the API independent of processor location; this amounts to a
contract between the user and the code: "If you promise to do your
best to make it so I don't have to mess with the internals, I promise
not to touch them."  If you start ignoring the tacit agreement that in
the process of executing a script, you're going to have to recognize
that the system might break down, and you might have to mess with
internals.  Things aren't perfect, but they do sort of work.

-Matt

On Fri, Feb 13, 2009 at 2:54 PM, Stephen Skory <stephenskory at yahoo.com> wrote:
> Matt,
>
>> I've done it in r1168.
>
>
> It works now without giving an error, but I still need to set _processing = True for it to work. Was that your intention? Without fippping _processing, it just hangs, as it did at the beginning of this thread.
>
> Thanks!
>
> _______________________________________________________
> sskory at physics.ucsd.edu           o__  Stephen Skory
> http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
> ________________________________(_)_\(_)_______________
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>