[yt-dev] yt-3.0 parallel_capable

Matthew Turk matthewturk at gmail.com
Wed Apr 23 05:18:38 PDT 2014 

Hi all,

Sorry for the quick followup, but it actually looks like the situation
is more complex as we have a few decorators, not just
@parallel_root_only.  These decorators will also be affected:

parallel_simple_proxy => only called by parameter_file_storage.py
parallel_blocking_call => called almost exclusively in the halo and
two point functions, but not in time-sensitive loops I think, so I
will add runtime checking.
parallel_splitter => not used, and I will remove this

parallel_passthrough already has a runtime check.

-Matt

On Wed, Apr 23, 2014 at 7:13 AM, Matthew Turk <matthewturk at gmail.com> wrote:
> Hi all,
>
> Sorry, I was away nearly all day Monday and Tuesday, and I'm just now
> getting back online.  I think Josh is right; the decorators are
> designed to swap out functions at import time, not at execution time.
> This worked just fine when we always made sure parallelism was set up
> at import time and never after.  But now that things work slightly
> differently, so does the decorator stuff.  We went with import-time
> changes because it can reduce branching logic calls, and function
> overhead, which can be important in tight loops.  These are the
> functions that are affected:
>
> __check_directory
> _export_obj
> _export_ply
> parallel_root_only
> print_key_parameters
> print_stats
> _save_light_cone_solution
> _save_light_cone_stack
> save_profiles
> _upload_to_sketchfab
> _write_filtered_halo_list
> write_fits_file
> write_fits_image
> write_fits
> write_h5_file
> _write_halo_map
> _write_halo_mask
> write_hdf5
> _write_light_ray
> _write_light_ray_solution
> write_out_arrays
> write_out_correlation
> write_png
> _write_seed_file
> write_simput_file
> write_spectrum
>
> I do not believe any of these should be significantly adversely
> affected by the inclusion of a conditional that is evaluated at
> execution time.  I'll issue a PR that does this.
>
> -Matt
>
> On Tue, Apr 22, 2014 at 5:06 PM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
>> Yup, I came to a similar conclusion:
>> https://bitbucket.org/yt_analysis/yt/issue/833/parallel_capable-getting-reset-to-false#comment-9785096
>>
>> I agree that being careful with imports is probably the best solution.
>>
>>
>> On Tue, Apr 22, 2014 at 3:04 PM, Josh Moloney <Joshua.Moloney at colorado.edu>
>> wrote:
>>>
>>> I think I've figured out what's going on here, but I don't have much
>>> experience with decorators in python so I'm not 100% sure. My guess is that
>>> parallel_root_only(func) is evaluated as soon as the @parallel_root_only
>>> decorator is encountered in each module as it's imported. At this point
>>> enable_parallelism hasn't been called yet, so parallel_capable is still
>>> False (it's initial assigned value) and the original function gets returned
>>> to all the processors (instead of None to everything except root). This
>>> would explain why Britton got the expected parallel behavior when he
>>> switched it so parallel_capable was initially assigned True instead
>>> (although this might create problems if it isn't actually being run in
>>> parallel).
>>> I tried enclosing a call to enable_parallelism inside parallel_root_only
>>> right before the "if parallel_capable" and this makes it behave as expected
>>> on both mpi and single processor runs. Unfortunately it means
>>> enable_parallelism gets called multiple times on each processor, which again
>>> probably isn't ideal behavior.
>>> I think the best way to solve this is to make sure enable_parallelism gets
>>> called before other imports are done, but I'm not sure how feasible this is.
>>> Someone who knows more about how decorators work can probably make a better
>>> suggestion.
>>>        - Josh
>>>
>>>
>>> On Tue, Apr 22, 2014 at 2:14 PM, Britton Smith <brittonsmith at gmail.com>
>>> wrote:
>>>>
>>>> Hi Nathan,
>>>>
>>>> I tried your suggestion, but it did not change anything.  I'll keep
>>>> looking around, but will file an issue for now.
>>>>
>>>> Britton
>>>>
>>>>
>>>> On Tue, Apr 22, 2014 at 8:57 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi Britton,
>>>>>
>>>>> It looks like parallel_capable on line 43 of
>>>>> parallel_analysis_interface.py *wants* to be a global mutable variable that
>>>>> is used elsewhere in yt, but it is not. Perhaps at one point it was imported
>>>>> into yt.funcs?
>>>>>
>>>>> I think you could fix this by explicitly importing parallel_cabable from
>>>>> parallel_analysis_interface.py into startup_tasks.py.
>>>>>
>>>>> Hope that helps,
>>>>>
>>>>> Nathan
>>>>>
>>>>>
>>>>> On Tue, Apr 22, 2014 at 12:42 PM, Britton Smith <brittonsmith at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi again,
>>>>>>
>>>>>> I don't know how much this helps, but if I change line 43 of
>>>>>> parallel_analysis_interface.py from parallel_capable = False to
>>>>>> parallel_capable = True, things work as they are supposed to in parallel.
>>>>>> Could there be an import somewhere that is resetting the value of
>>>>>> parallel_capable?
>>>>>>
>>>>>> Britton
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 22, 2014 at 8:28 PM, Britton Smith <brittonsmith at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> HI Nathan,
>>>>>>>
>>>>>>> Adding that call did not change the result.  To be clear, this is what
>>>>>>> I ran:
>>>>>>> http://paste.yt-project.org/show/4548/
>>>>>>>
>>>>>>> What is strange is that the logger was clearly set up for parallel as
>>>>>>> it correctly prints the different processor numbers.  Enable parallelism
>>>>>>> shows that parallel_capable is True, so somewhere along the line this is
>>>>>>> getting changed.
>>>>>>>
>>>>>>> Britton
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 22, 2014 at 8:23 PM, Nathan Goldbaum
>>>>>>> <nathan12343 at gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi Britton,
>>>>>>>>
>>>>>>>> What happens when you call yt.enable_parallelism() at the top of the
>>>>>>>> script?
>>>>>>>>
>>>>>>>> There were some adjustments to the way parallelism works related to
>>>>>>>> YTEP-0019.  See this PR:
>>>>>>>> https://bitbucket.org/yt_analysis/yt/pull-request/729/ytep-0019/diff
>>>>>>>>
>>>>>>>> Do you have a short test script I can try running over here to
>>>>>>>> confirm the issue?
>>>>>>>>
>>>>>>>> -Nathan
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 22, 2014 at 12:20 PM, Britton Smith
>>>>>>>> <brittonsmith at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I don't recall when I first noticed this, but I'm finding that the
>>>>>>>>> parallel_root_only decorator doesn't seem to be working.  A simple test is
>>>>>>>>> to run a script that only loads a dataset.  The print_key_parameters
>>>>>>>>> function is inside of a parallel_root_only decorator, so all that should
>>>>>>>>> only be printed by the root processor, but when I run that script in
>>>>>>>>> parallel it is printed by all processors.  I am still looking into this, but
>>>>>>>>> so far I have found that printing the value of parallel_capable inside this
>>>>>>>>> decorator always gives False even when running in parallel.
>>>>>>>>>
>>>>>>>>> Can someone confirm this?  If so, does anyone know what's going on?
>>>>>>>>>
>>>>>>>>> Britton
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-dev mailing list
>>>>>>>>> yt-dev at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-dev mailing list
>>>>>>>> yt-dev at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-dev mailing list
>>>>>> yt-dev at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-dev mailing list
>>>>> yt-dev at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-dev mailing list
>>>> yt-dev at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>
>>>
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>

More information about the yt-dev mailing list