[yt-users] parallel_objects with projection hanging

[Semyeong Oh]

Hi yt,

I have two questions on using parallel_objects. I am using yt 2.6.

1. I have a problem of parallel_objects hanging at the end.

def do(i, pf):
   cube = pf.h.region(..)
   proj = pf.h.proj(…., source=cube)
   frb = proj.to_frb(..)
   ….

objects = [list of indices]
pf = load(..)
for i in parallel_objects(objects):
    do(i, pf)

and I run the script as
mpirun -np Nprocs python myscripy.py —parallel

When I tested with a simple print operation in do instead of proj, the parallel_objects seem to handle
cases when Nobjects is not divisible by Nprocs just fine. But with my real script that has proj in do, it seems to hang at the end. For example, if Nobjects is 3 and Nprocs is 2, the first two objects goes without problem, but the projection of the third completes, but the process sort of hangs there. Why so?

2. Is it possible to use a portion of Nprocs assigned? Also playing around with simple print operation, it seems that because of the way parallel_objects divide work, the work is duplicated. e..g, when I do mpirun -np 5 but have parallal_objects(objects, njobs=3)
rank i_object
0 1
1 1
2 2
3 2
4 3
…
so object 1 would still run simultaneously on rank 0 and 1.

To prevent this, would something like below work?

size = MPI.COMM_WORLD.Get_size()
rank = MPI.COMM_WORLD.Get_rank()
njobs = 3
for ind in parallel_objects(objects, njobs):
    if rank % int(size/njobs) != 0:
        continue
    else:
        do(ind)

Thanks,
Semyeong