[yt-users] Parallel data reading and covering grid

[Yingchao Lu]

Dear yt users,

I am trying to read some AMR data and convert them into 3D array. The test
code:
######################### BEGIN #########################
import yt
from time import time
yt.enable_parallelism()

ds = yt.load("flash_hdf5_plt_cnt_0000")
tstart = time()
cg = ds.covering_grid(level=ds.max_level, left_edge=ds.domain_left_edge,
dims=ds.domain_dimensions*2**ds.max_level)
cg['dens']

if yt_isroot(): print "It takes {0}s".format(time()-tstart)
######################### END #########################

I tried to run to serial or parallel on an interactive session on stampede:
 ######################### BEGIN #########################
[yclu@ test]$ ls
flash_hdf5_plt_cnt_0000  test.py
[yclu@ test]$ python test.py
It takes 34.0571820736s
[yclu@ test]$ export OMP_NUM_THREADS=68
[yclu@ test]$ python test.py
It takes 33.1969199181s
[yclu@ test]$ export OMP_NUM_THREADS=1
[yclu@ test]$ mpirun -np 68 python test.py
It takes 58.0391800404s
######################### END #########################

The time does not seem to be reduced by parallelism. And multi-process seem
to have huge communication overhead. Is there a way to increase the speed
by parallelism?

Thanks,
Yingchao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20170919/4c73d288/attachment.htm>