[yt-dev] missing objects using parallel_objects()

[Geoffrey So]

I've just thought of something, I did a pull from dev-yt to my fork of YT
sometime last week (to get the functionality of parallel_objects and my
ellipsoid stuff), so maybe I broke something, I'll re-run the thing with
dev-yt and see if the error is still there.  I recall I had to merge
halo_objects.py manually and so maybe I screwed up something there.  I
wasn't suspecting the merge to be problematic because parallel HOP ran just
fine.

From
G.S.

On Fri, Mar 23, 2012 at 10:31 AM, Geoffrey So <gsiisg at gmail.com> wrote:

> Hi,
>
> Originally I was outputting something like 17 or 18 columns of attributes
> of my ellipsoids associated with the haloes, but I've ripped them out to
> the bare essential of just out putting the mass of halo and a "0" for
> attribute to narrow down the problem.  So in this script I did nothing with
> the ellipsoids.
>
> http://paste.yt-project.org/show/2250/
>
> For each halo, the output should have for the first column the halo DM
> particle mass, the second column a zero.
>
> The output when using
> for halo in haloes:
> http://paste.yt-project.org/show/2251/
>
> The output when using
> for sto, halo in parallel_objects(haloes, num_procs, storage = my_storage):
> http://paste.yt-project.org/show/2252/
>
> The original halo list from parallelHOP:
> http://paste.yt-project.org/show/2253/
>
> So there's agreement of the number of haloes with the "for halo in haloes"
> method and disagreement with the parallel_objects() method.
>
> From
> G.S.
>
>
> in the DD0273_z5.00_halo_list.out file I have 24 lines, first two
>
>
> On Fri, Mar 23, 2012 at 8:24 AM, Stephen Skory <s at skory.us> wrote:
>
>> Hi Geoffrey,
>>
>>
>> >> haloes = LoadHaloes(pf, HaloListname)
>> >>
>> >> for sto, halo in parallel_objects(haloes, num_procs, storage =
>> my_storage):
>>
>> Can you paste the whole script? Thanks.
>>
>> > Stephen might be able to shed some light on this, but I think
>> > LoadHalos will pre-assign processors to the halo objects, whereas
>> > parallel_objects will operate independently of that, distributing
>> > halos first come, first server.
>>
>> In fact, LoadHaloes should not work that way. Each task should have a
>> full copy of the halos data, but initially only the data in the
>> HopAnalysis.out file, of course, the particles are loaded on demand.
>>
>> I've been using something like what Geoffrey's trying to do for a
>> while with no issue. I'm hoping maybe there's something in Geoffrey's
>> script... but I've been wrong before.
>>
>> --
>> Stephen Skory
>> s at skory.us
>> http://stephenskory.com/
>> 510.621.3687 (google voice)
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20120323/deb4c166/attachment.html>