[Yt-dev] [yt-users] parallel data retrieval

[Stephen Skory]

Matt,

I think I don't quite understand something fundamental about how parallel python works. In particular, how do I make the loop at line 29 run on all processors at the same time? As an experiment I'm trying to write separate HDF5 files per processor to speed things up, but from the timestamp (line 73) I can see that it's going serially in order of halo.id. For example, the first halo on proc 0 is the sixth, but it isn't written until the first five are written by proc 1, so it's going in strict order.

http://paste.enzotools.org/show/49/

wrote halo 0 from proc 1 at 2009-02-14 10:53:05.206974
wrote halo 1 from proc 1 at 2009-02-14 10:53:05.296788
wrote halo 2 from proc 1 at 2009-02-14 10:53:05.406035
wrote halo 3 from proc 1 at 2009-02-14 10:53:05.506907
wrote halo 4 from proc 1 at 2009-02-14 10:53:05.626588
wrote halo 5 from proc 0 at 2009-02-14 10:53:06.474605
wrote halo 6 from proc 0 at 2009-02-14 10:53:06.553487
wrote halo 7 from proc 1 at 2009-02-14 10:53:06.688611
wrote halo 8 from proc 1 at 2009-02-14 10:53:06.790755
wrote halo 9 from proc 0 at 2009-02-14 10:53:06.879232
wrote halo 10 from proc 1 at 2009-02-14 10:53:06.978932


Thanks for answering my naive questions... 

 _______________________________________________________
sskory at physics.ucsd.edu           o__  Stephen Skory
http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
________________________________(_)_\(_)_______________