[yt-users] Storing multiple calculations using yt.parallel_objects

[Joshua Wall]

Dear Britton,


I tried something like that, but clearly I'm not doing it correctly. I'm
noticing two errors, 1) everything is reporting running on the root proc
(with a "Yo I'm root!") and 2) I'm getting four copies of my_storage back.
Here's the code:

#########################################################################

from matplotlib import rc
import matplotlib.pyplot as plt
import yt
from yt import derived_field
import numpy as np
import yt.units as u
import glob

# Use YT in parallel.
yt.enable_parallelism()

# Number of procs on the machine
num_procs=4


@derived_field(name = "total_KE", units = "erg")
def _KE(field, data):
    return
np.sum(data["kinetic_energy"].in_mks()*data["cell_volume"].in_cgs())

@derived_field(name = "total_PE", units = "erg")
def _PE(field, data):
    return 0.5*np.sum(data["gpot"].in_mks()*data["cell_mass"].in_cgs())

@derived_field(name = "total_Eint", units = "erg")
def _Eint(field, data):
    return np.sum(data["eint"]*data["cell_mass"]).in_cgs()

@derived_field(name = "total_energy", units = "erg")
def _Etotal(field, data):
    return data["total_Eint"] + data["total_PE"] + data["total_KE"]

@derived_field(name = "total_px", units = "g*cm/s")
def _px(field, data):
    return np.sum(data["cell_mass"]*data["velx"]).in_cgs()

# Use glob to gather up the data files. Note here we use plt
# and particle files because they are much smaller than
# checkpoint files.

plot = glob.glob("./SinkMomTest/*plt_cnt_000?")
plot.sort()
part = glob.glob("./SinkMomTest/*part_000?")
part.sort()

# Make a list of variables you are collecting to plot/store as
# dictionaries.
my_storage = {}

# Now the parallel loop.

i = 0

for sto,p in yt.parallel_objects(plot, num_procs, storage = my_storage):

    ds = yt.load(plot[i], particle_filename=part[i])
    dd = ds.all_data()

    if (yt.is_root()):
        print "Yo I'm root!"
    else:
        print "Not root!"


    energy = dd["total_energy"]
    mom    = dd["total_px"]

    sto.result_id = p[-4:]
    sto.result = [energy, mom]

    i=i+1


print "Done!"

print my_storage

#########################################################################

Is this what you mean?

Kindly,

Josh



On Sun, May 24, 2015 at 12:19 PM Britton Smith <brittonsmith at gmail.com>
wrote:

> Hi Josh,
>
> The reason that's not working is that there is still only one storage item
> (the sto variable) per iteration of the loop.  You could keep both things
> if you make sto.result a list object and just append each of those things
> to that list.
>
> Britton
>
> On Sun, May 24, 2015 at 11:12 AM, Joshua Wall <joshua.e.wall at gmail.com>
> wrote:
>
>> Dear users,
>>
>>
>>      This there a simple way to store multiple values at each step using
>> parallel_objects? I'm doing:
>>
>> ########################################################
>> yt.enable_parallelism()
>> num_procs = 4
>> plot = glob.glob(./plot*)
>> my_storage = {}
>>
>> for sto, p in yt.parallel_objects(plots, num_procs, my_storage):
>>
>>   sto.result_id = 'energy' + p
>>   sto.resutl     = dd["total_energy"]
>>
>>   sto.result_id = 'Px' + p
>>   sto.result     = dd["total_mom_x"]
>>
>> ######################################################
>>
>> which is just overwriting my energy with the momentum. I also tried to
>> just set to different dictionaries without passing it to
>> yt.parallel_objects as storage, but of course each proc as only a local
>> copy then.
>>
>> Cordially,
>>
>> Josh
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20150524/8aa075ec/attachment.htm>
-------------- next part --------------
_______________________________________________
yt-users mailing list
yt-users at lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org