[yt-users] Parallelizing a bunch of projections
John ZuHone
jzuhone at gmail.com
Thu Dec 4 14:54:25 PST 2014
Hi all,
I know that parallelizing a single projection doesn’t really give you that much speed benefit.
But what would happen if I tried to do something like this (please excuse the Python pseudocode):
fields = [big list of fields I want to project]
prj = ds.proj(“ones”, 2)
frb = prj.to_frb(…)
storage = {}
for sto, fd in parallel_objects(fields, storage=storage):
buf = frb[fd]
sto.result_id = fd
sto.result = buf
I did something like this but it hung at the end, which makes me think that I’ve got some kind of blocking operation here that I’m not aware of.
Best,
John
_______________________________________________
yt-users mailing list
yt-users at lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
More information about the yt-users
mailing list