[yt-users] parallel quantities

Tue Jul 23 11:40:28 PDT 2013

On Tue, Jul 23, 2013 at 11:36 AM, David Collins <dcollins4096 at gmail.com> wrote:
> I've simplified my test a bit, and now I'm only examining MagneticField_C_1,
> which is the name of the primitive field that lives in the file (This is an
> old dataset, before the mhdct names were unified with the rest of Enzo)
>
> I can query MagneticField_C_1 directly in parallel, but trying to do the
> weighted average as above fails.  Adding preload=False does work--what does
> that do and why does it work?
>
> I'm adding the field with the following in the plugin file-- this is for
> historical reasons, I guess, I haven't touched this in several years:
>
> for fff in mag_fields:
>     exec('maglam_%s = lambda a: a.convert("%s")'%(fff,fff))
>     exec('add_field("%s",lambda a, b: None, convert_function = maglam_%s,
> take_log = False)'%(fff,fff))
>
> Is that a possible source of problem?

Yeah, I think so.  I would encourage you not to use execs but to use
closures for this, but at the bare minimum changing from "lambda a, b:
None" to NullFunc (defined in field_info_container.py) may fix the
problem.

-Matt

>
> Thanks a ton,
>
> d.
>
>
>
>
> On Tue, Jul 23, 2013 at 12:08 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
>>
>> Hi Dave,
>>
>> Usually this means that it's not getting a field -- sorry for the
>> obtuse error message.  What happens if you try specifying
>> preload=False, or if you query MagneticEnergy directly?
>>
>> -Matt
>>
>> On Tue, Jul 23, 2013 at 1:58 PM, David Collins <dcollins4096 at gmail.com>
>> wrote:
>> > Hi, all--
>> >
>> > I have an odd error trying to get some averaged quantities in parallel.
>> > I'm
>> > trying to do
>> >
>> >
>> > pf.h.all_data().quantities['WeightedAverageQuantity']('MagneticEnergy','CellVolume')
>> >
>> > on two processors, and I get the following error:
>> >
>> > File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 396, in __call__
>> > dd *= self._convert_function(data)
>> > TypeError: unsupported operand type(s) for *=: 'NoneType' and 'int'
>> >
>> > It works fine in serial, and it works fine if I use "VelocityMagnitude"
>> > instead of "MagneticEnergy"
>> >
>> > Does anyone have an idea about what I'm doing wrong?
>> >
>> > I run it as
>> >> mpirun -np 2 python p23_go.py --parallel
>> >
>> > and the full trace back is:
>> >
>> > Traceback (most recent call last):
>> >   File "ptmp.py", line 5, in <module>
>> >     bavg =
>> > ad.quantities['WeightedAverageQuantity']('MagneticEnergy','CellVolume')
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/derived_quantities.py",
>> > line 87, in __call__
>> > Traceback (most recent call last):
>> >   File "ptmp.py", line 5, in <module>
>> >     bavg =
>> > ad.quantities['WeightedAverageQuantity']('MagneticEnergy','CellVolume')
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/derived_quantities.py",
>> > line 87, in __call__
>> >     self.func(e, *args, **kwargs)
>> >     self.func(e, *args, **kwargs)
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/derived_quantities.py",
>> > line 203, in _WeightedAverageQuantity
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/derived_quantities.py",
>> > line 203, in _WeightedAverageQuantity
>> >     num = (data[field] * data[weight]).sum()
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 257, in __missing__
>> >     num = (data[field] * data[weight]).sum()
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 257, in __missing__
>> >     vv = FI[item](self)
>> >     vv = FI[item](self)
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 395, in __call__
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 395, in __call__
>> >     dd = self._function(self, data)
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/universal_fields.py",
>> > line 1016, in _MagneticEnergy
>> >     dd = self._function(self, data)
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/universal_fields.py",
>> > line 1016, in _MagneticEnergy
>> >     return (data["Bx"]**2 + data["By"]**2 + data["Bz"]**2)/(8*np.pi)
>> >     return (data["Bx"]**2 + data["By"]**2 + data["Bz"]**2)/(8*np.pi)
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 257, in __missing__
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 257, in __missing__
>> >     vv = FI[item](self)
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 396, in __call__
>> >     vv = FI[item](self)
>> >   File
>> >
>> > "/nics/b/home/collins/local_2013_03_12/src/yt-hg/yt/data_objects/field_info_container.py",
>> > line 396, in __call__
>> >     dd *= self._convert_function(data)
>> > TypeError: unsupported operand type(s) for *=: 'NoneType' and 'int'
>> >     dd *= self._convert_function(data)
>> > TypeError: unsupported operand type(s) for *=: 'NoneType' and 'int'
>> >
>> > Thanks a ton!
>> >
>> >
>> > --
>> > -- Sent from a computer.
>> >
>> > _______________________________________________
>> > yt-users mailing list
>> > yt-users at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
>
>
> --
> -- Sent from a computer.
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>