[yt-users] Problem when run yt dev in parallel on Blue Waters

Pengfei Chen madcpf at gmail.com
Mon Aug 22 18:02:15 PDT 2016


Hi all,

I tried to use mpi-2.0.0 instead of mpi-1.3.1, and compiled with the same
procedure. Now yt could run in parallel.

Thanks,
Pengfei

On Sun, Aug 21, 2016 at 5:09 PM, Pengfei Chen <madcpf at gmail.com> wrote:

> Hi Brian and Nathan,
>
> I am trying to install yt-3 on Blue Waters again, since after updating yt
> to the current tip it somehow can't run in parallel. When I came to step 3)
> following Brian's instructions, I get the following error message:
>
> ~/mpi4py-1.3.1 $python setup.py build --mpi=cray
> running build
> running build_src
> running build_py
> creating build
> creating build/lib.linux-x86_64-2.7
> creating build/lib.linux-x86_64-2.7/mpi4py
> copying src/rc.py -> build/lib.linux-x86_64-2.7/mpi4py
> copying src/__init__.py -> build/lib.linux-x86_64-2.7/mpi4py
> creating build/lib.linux-x86_64-2.7/mpi4py/include
> creating build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.MPI.h -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.MPI_api.h -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.h -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi_c.pxd -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/MPI.pxd -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/__init__.pxd -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/__init__.pyx -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi.pxi -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.i -> build/lib.linux-x86_64-2.7/
> mpi4py/include/mpi4py
> copying src/MPI.pxd -> build/lib.linux-x86_64-2.7/mpi4py
> copying src/mpi_c.pxd -> build/lib.linux-x86_64-2.7/mpi4py
> running build_clib
> MPI configuration: [cray] from 'mpi.cfg'
> MPI C compiler:    /opt/cray/craype/2.5.4/bin/cc
> MPI C++ compiler:  /opt/cray/craype/2.5.4/bin/CC
> Traceback (most recent call last):
>   File "setup.py", line 588, in <module>
>     main()
>   File "setup.py", line 585, in main
>     run_setup()
>   File "setup.py", line 459, in run_setup
>     **metadata)
>   File "/mnt/a/u/sciteam/madcpf/mpi4py-1.3.1/conf/mpidistutils.py", line
> 494, in setup
>     return fcn_setup(**attrs)
>   File "/u/sciteam/madcpf/anaconda2/lib/python2.7/distutils/core.py",
> line 151, in setup
>     dist.run_commands()
>   File "/u/sciteam/madcpf/anaconda2/lib/python2.7/distutils/dist.py",
> line 953, in run_commands
>     self.run_command(cmd)
>   File "/u/sciteam/madcpf/anaconda2/lib/python2.7/distutils/dist.py",
> line 972, in run_command
>     cmd_obj.run()
>   File "/u/sciteam/madcpf/anaconda2/lib/python2.7/distutils/command/build.py",
> line 127, in run
>     self.run_command(cmd_name)
>   File "/u/sciteam/madcpf/anaconda2/lib/python2.7/distutils/cmd.py", line
> 326, in run_command
>     self.distribution.run_command(command)
>   File "/u/sciteam/madcpf/anaconda2/lib/python2.7/distutils/dist.py",
> line 972, in run_command
>     cmd_obj.run()
>   File "/mnt/a/u/sciteam/madcpf/mpi4py-1.3.1/conf/mpidistutils.py", line
> 894, in run
>     configure_compiler(self.compiler, config)
>   File "/mnt/a/u/sciteam/madcpf/mpi4py-1.3.1/conf/mpidistutils.py", line
> 249, in configure_compiler
>     compiler.compiler += cc_args
> TypeError: coercing to Unicode: need string or buffer, list found
>
> Do you have any idea what happens here?
>
> Thanks a lot!
> Pengfei
>
> On Mon, Nov 23, 2015 at 12:13 PM, Pengfei Chen <madcpf at gmail.com> wrote:
>
>> Hi Brian,
>>
>> It works perfectly! Thank you so much!
>> I've saved the instructions for later use.
>>
>> Thanks,
>> Pengfei
>>
>>
>> On Fri, Nov 20, 2015 at 3:34 AM, Brian O'Shea <bwoshea at gmail.com> wrote:
>>
>>> Hi Pengfei,
>>>
>>> Sorry for the delay in response.  Here are instructions for getting yt-3
>>> to work on Blue Waters using mpi4py.  I believe that Rob Sisneros has
>>> gotten yt-3 working as a module; however, it is likely that it will not be
>>> updated very frequently, at least compared to the pace of yt development.
>>>
>>> --Brian
>>>
>>> -----------------------------------------------------------------------
>>>
>>> Getting yt-3 to work on Blue Waters:
>>>
>>> 0) Get the GNU software environment set up:  “module swap PrgEnv-cray
>>> PrgEnv-gnu” .  This will be necessary later.
>>>
>>> 1)  Install the Anaconda python distribution (
>>> https://store.continuum.io/cshop/anaconda/) for 64-bit Linux systems.
>>> Refresh your path so the Anaconda python’s in your path.
>>>
>>> 2)  Download mpi4py from https://bitbucket.org/mpi4py/mpi4py/, or grab
>>> it from my home directory on Blue Waters (~bwoshea/mpi4py-1.3.1.tar.gz).
>>>
>>> 3)  Once you’ve unpacked it in your home directory, edit the file
>>> mpi.cfg, which is in the top-level mpi4py directory, and add the following
>>> chunk of text at the end of the file:
>>>
>>> [cray]
>>> mpicc = cc
>>> mpicxx = CC
>>> extra_link_args = -shared
>>>
>>> then, in the top-level mpi4py directory, type:
>>>
>>> python setup.py build --mpi=cray
>>> python setp.py install
>>>
>>> At this point, you should have a functional version of mpi4py that uses
>>> the cray MPI libraries.
>>>
>>> 4)  You can then install yt via source in your home directory:
>>>
>>> hg clone https://bitbucket.org/yt_analysis/yt
>>> hg update yt
>>> python setup.py develop
>>>
>>> And it should work, though you’re going to need to modify your
>>> LD_LIBRARY_PATH to point to your anaconda /lib directory.  You can verify
>>> that yt and mpi4py are playing nicely together by running the scripts
>>> hw_MPI.py and yt_parallel_test.py, both of which are sitting in my home
>>> directory.  (Note that you can do this via batch script or through the
>>> interactive queue, both with aprun - you can’t just do it from the login
>>> node.)
>>>
>>> 5) and, optionally, if you want to install Rockstar using yt from
>>> source, you do the following:
>>>
>>> hg clone http://bitbucket.org/MatthewTurk/rockstar
>>> cd rockstar
>>> hg pull
>>> hg up -C tip
>>> make lib
>>>
>>> and *then* make a file called rockstar.cfg that has a single line of
>>> text: the path of the rockstar source code directory.  At that point, you
>>> need to run “python setup.py develop” again, and yt will recognize that
>>> rockstar has been installed and play nicely with it.  You will likely also
>>> have to modify your LD_LIBRARY_PATH to point at your rockstar directory so
>>> it can find the rockstar library (librockstar.so).
>>>
>>>
>>>
>>> On Fri, Nov 20, 2015 at 3:46 AM, Pengfei Chen <madcpf at gmail.com> wrote:
>>>
>>>> Hi Brian,
>>>>
>>>> Thank you very much for your reply! No, I followed the instructions here
>>>>  http://yt-project.org/docs/dev/installing.html
>>>> <http://yt-project.org/docs/dev/installing.html>. I need the
>>>> yt-development, but failed when using All-in-One Installation Script to
>>>> install it on BW, so I used miniconda to install. Could you please send me
>>>> your installation instructions for yt on BW? I will try it out.
>>>>
>>>> Thanks,
>>>> Pengfei
>>>>
>>>> On Thu, Nov 19, 2015 at 8:45 AM, Brian O'Shea <bwoshea at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Pengfei,
>>>>>
>>>>> Did you use my installation instructions for yt on Blue Waters?  If
>>>>> not, I can send those along.
>>>>>
>>>>> B
>>>>>
>>>>>
>>>>> On Thu, Nov 19, 2015 at 10:02 AM, Nathan Goldbaum <
>>>>> nathan12343 at gmail.com> wrote:
>>>>>
>>>>>> So it looks like either mpi4py isn't able to properly link against
>>>>>> the MPI installation on Blue waters or properly initialize itself at
>>>>>> runtime.
>>>>>>
>>>>>> In either case, this points to the issue being a problem with mpi4py,
>>>>>> not yt.
>>>>>>
>>>>>> Three things to try:
>>>>>>
>>>>>> * check which version version of mpi4py you have installed in your
>>>>>> yt-2.x environment and install that version in your yt-3.x environment. You
>>>>>> may have found a bug in mpi4py 2.0, which made major changes to the mpi4py
>>>>>> library.
>>>>>>
>>>>>> * contact the Blue Waters sysadmins.
>>>>>>
>>>>>> * contact the mpi4py developers
>>>>>>
>>>>>> Hope that helps,
>>>>>>
>>>>>> Nathan
>>>>>>
>>>>>>
>>>>>> On Thursday, November 19, 2015, Pengfei Chen <madcpf at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Nathan,
>>>>>>>
>>>>>>> Thank you very much for your reply! When I run that test script in
>>>>>>> parallel with 16 cores, I get 16 0s. Any suggestions to fix this?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Pengfei
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Nov 18, 2015 at 3:40 PM, Nathan Goldbaum <
>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Pengfei,
>>>>>>>>
>>>>>>>> Can you try to run the following test script in parallel? This will
>>>>>>>> determine if the issue is on the yt side of things or the mpi4py side of
>>>>>>>> things:
>>>>>>>>
>>>>>>>>     from mpi4py import MPI
>>>>>>>>
>>>>>>>>     print(MPI.COMM_WORLD.rank)
>>>>>>>>
>>>>>>>> -Nathan
>>>>>>>>
>>>>>>>> On Wed, Nov 18, 2015 at 5:36 PM, Pengfei Chen <madcpf at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I have trouble running yt in parallel on Blue Waters. I installed
>>>>>>>>> yt using miniconda, the version of yt is
>>>>>>>>>
>>>>>>>>> ~/miniconda/lib $yt version
>>>>>>>>> yt module located at:
>>>>>>>>>     /u/sciteam/madcpf/miniconda/lib/python2.7/site-packages/yt-
>>>>>>>>> 3.3.dev0-py2.7-linux-x86_64.egg
>>>>>>>>> The current version and changeset for the code is:
>>>>>>>>> ---
>>>>>>>>> Version = 3.3-dev
>>>>>>>>> Changeset = 90f900be7a36+ yt
>>>>>>>>>
>>>>>>>>> Then with miniconda/bin in PATH I installed mpi4py-2.0.0. But when
>>>>>>>>> I tried to make the following simple output in parallel, I get:
>>>>>>>>>
>>>>>>>>> import yt
>>>>>>>>> yt.enable_parallelism()
>>>>>>>>> from yt.utilities.parallel_tools.parallel_analysis_interface
>>>>>>>>> import\
>>>>>>>>>  parallel_objects, communication_system
>>>>>>>>>
>>>>>>>>> comm = communication_system.communicators[-1]
>>>>>>>>> print comm.rank, comm.size
>>>>>>>>>
>>>>>>>>> 0 1
>>>>>>>>> 0 1
>>>>>>>>> 0 1
>>>>>>>>> 0 1
>>>>>>>>> 0 1
>>>>>>>>> 0 1
>>>>>>>>> 0 1
>>>>>>>>> 0 1
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> When I run a similar code but with yt-2.x also on Blue Waters, I
>>>>>>>>> get what I expect:
>>>>>>>>>
>>>>>>>>> 7 16
>>>>>>>>> 15 16
>>>>>>>>> 6 16
>>>>>>>>> 9 16
>>>>>>>>> 11 16
>>>>>>>>> 8 16
>>>>>>>>> 0 16
>>>>>>>>> 4 16
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> I'm confused about it. Could anyone give me some suggestions
>>>>>>>>> please?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Pengfei
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160822/3c65a065/attachment.html>


More information about the yt-users mailing list