[yt-users] Problem with Derived Quantities/Fields while running in parallel

Matthew Turk matthewturk at gmail.com
Mon May 23 20:29:16 PDT 2011


Hi Anthony,

I can replicate this problem, as can Sam.

Here's what I think is going on.  When you run in parallel and access
a derived quantity, it really wants to try to batch the IO as best it
can.  To do this, it uses a FieldDector.  This is a fake object that
accepts requests for fields, data, parameters, etc.  It does this by
tracking accesses to fields and parameters, but it doesn't mock up a
complete parameter file.

The baryon overdensity field uses a non-standard method of accessing
data from the parameter file, which the fake parameter file didn't
know.  Rather than access the dictionary directly
(pf['omega_baryon_now']) it accessed it by peeking into the underlying
data structure 'parameters.'  I have changed this and pushed a fix to
the main repository, in changeset 7270e9e83be7.

If you're running from an installation you maintain, you should be
able to do "yt instinfo -u" to get the latest version.  Let me know if
that doesn't fix it for you -- it does for me, but there might be an
additional complicating factor I'm missing.

-Matt

On Mon, May 23, 2011 at 5:49 PM, Anthony Harness
<anthony.harness at colorado.edu> wrote:
> I am running yt-2.1stable with python2.7 on Kraken and run into this
> problem:
>
> When I call a derived quantity that calls a derived field that calls a
> universal field, I get an error: "AttributeError: 'fake_parameter_file'
> object has no attribute 'parameters'" which arises in the universal field's
> convert function in 'data_objects/universal_fields.py'
>
>
> With the following code:
> #derived Field
> def _test2(field,data):
>         return data['Baryon_Overdensity']
>
> add_field('test2',function=_test2,units=r"")
>
> #derived quantity
> def _test(data):
>         return data['test2']
>
> def _combTest(data,arr):
>         return arr.sum()
>
> add_quantity('test',function=_test,combine_function=_combTest,n_ret=1)
>
> dd.quantities['test']()
>
>
> I get the error:
>
> AttributeError: 'fake_parameter_file' object has no attribute 'parameters'
>
> from line 418, in _Convert_Baryon_Overdensity in universal_fields.py:
> "data.pf.parameters.has_key('omega_baryon_now'):"
>
>
> So it seems as if the parameter file is not being passed from the derived
> quantitty all the way to the universal field. This works fine when I run it
> in serial, but fails when it is ran in parallel.
> Any suggestions?
>
> Thanks
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>



More information about the yt-users mailing list