[yt-users] yt on Blue Waters

Jonah Miller jonah.maxwell.miller at gmail.com
Wed Apr 20 14:41:40 PDT 2016


Hi Nathan, Matt, Brian,

I see. Yes, I'll want to use the parallelism. Cool! I didn't realize the 
all-in-one script could pull from different repos. Thanks for your help!

Best,
Jonah

On 16-04-20 05:39 PM, Nathan Goldbaum wrote:
> You'll probably also want to follow Brian's instructions for mpi4py if 
> you're going t be using yt's parallelism. The install script doesn't 
> set that up since we can't assume people have an MPI installation.
>
> On Wednesday, April 20, 2016, Matthew Turk <matthewturk at gmail.com 
> <mailto:matthewturk at gmail.com>> wrote:
>
>     Hi Jonah,
>
>     Just make sure you use the from-source option, and you can pull
>     whatever changesets you like into your source directory in your
>     installation.
>
>     On Wed, Apr 20, 2016 at 4:34 PM, Jonah Miller
>     <jonah.maxwell.miller at gmail.com <javascript:;>> wrote:
>     > Hi Brian, Matt,
>     >
>     > Thanks a lot! These instructions are exactly what I was hoping for!
>     > Regarding the all-in-one install script, that won't work if I
>     want to
>     > install a fork of yt though, will it? This is to test my carpet
>     frontend on
>     > some production data.
>     >
>     > Best,
>     > Jonah
>     >
>     >
>     > On 16-04-20 05:24 PM, Matthew Turk wrote:
>     >>
>     >> I think it's a lot easier now that the new install script has
>     been merged
>     >> in:
>     >>
>     >>
>     >>
>     http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2016-April/006350.html
>     >>
>     >> -Matt
>     >>
>     >> On Wed, Apr 20, 2016 at 4:22 PM, Brian O'Shea
>     <bwoshea at gmail.com <javascript:;>> wrote:
>     >>>
>     >>> Yes indeed.  It's a gigantic pain.  Here are instructions. 
>     Good luck!
>     >>>
>     >>> -----------------
>     >>>
>     >>> 0) Get the GNU software environment set up: “module swap
>     PrgEnv-cray
>     >>> PrgEnv-gnu” .  This will be necessary later.
>     >>>
>     >>> 1)  Install the Anaconda python distribution
>     >>> (https://store.continuum.io/cshop/anaconda/) for 64-bit Linux
>     systems.
>     >>> Refresh your path so the Anaconda python’s in your path.
>     >>>
>     >>> 2)  Download mpi4py from https://bitbucket.org/mpi4py/mpi4py/,
>     or grab it
>     >>> from my home directory on Blue Waters
>     (~bwoshea/mpi4py-1.3.1.tar.gz).
>     >>>
>     >>> 3)  Once you’ve unpacked it in your home directory, edit the file
>     >>> mpi.cfg,
>     >>> which is in the top-level mpi4py directory, and add the
>     following chunk
>     >>> of
>     >>> text at the end of the file:
>     >>>
>     >>> [cray]
>     >>> mpicc = cc
>     >>> mpicxx = CC
>     >>> extra_link_args = -shared
>     >>>
>     >>> then, in the top-level mpi4py directory, type:
>     >>>
>     >>> python setup.py build --mpi=cray
>     >>> python setp.py install
>     >>>
>     >>> At this point, you should have a functional version of mpi4py
>     that uses
>     >>> the
>     >>> cray MPI libraries.
>     >>>
>     >>> 4)  You can then install yt via source in your home directory:
>     >>>
>     >>> hg clone https://bitbucket.org/yt_analysis/yt
>     >>> hg update yt
>     >>> python setup.py develop
>     >>>
>     >>> And it should work, though you’re going to need to modify your
>     >>> LD_LIBRARY_PATH to point to your anaconda /lib directory.  You
>     can verify
>     >>> that yt and mpi4py are playing nicely together by running the
>     scripts
>     >>> hw_MPI.py and yt_parallel_test.py, both of which are sitting
>     in my home
>     >>> directory.  (Note that you can do this via batch script or
>     through the
>     >>> interactive queue, both with aprun - you can’t just do it from
>     the login
>     >>> node.)
>     >>>
>     >>> 5) and, optionally, if you want to install Rockstar using yt
>     from source,
>     >>> you do the following:
>     >>>
>     >>> hg clone http://bitbucket.org/MatthewTurk/rockstar
>     >>> cd rockstar
>     >>> hg pull
>     >>> hg up -C tip
>     >>> make lib
>     >>>
>     >>> and *then* make a file called rockstar.cfg that has a single
>     line of
>     >>> text:
>     >>> the path of the rockstar source code directory.  At that
>     point, you need
>     >>> to
>     >>> run “python setup.py develop” again, and yt will recognize
>     that rockstar
>     >>> has
>     >>> been installed and play nicely with it.  You will likely also
>     have to
>     >>> modify
>     >>> your LD_LIBRARY_PATH to point at your rockstar directory so it
>     can find
>     >>> the
>     >>> rockstar library (librockstar.so).
>     >>>
>     >>>
>     >>> On Wed, Apr 20, 2016 at 5:04 PM, Jonah Miller
>     >>> <jonah.maxwell.miller at gmail.com <javascript:;>> wrote:
>     >>>>
>     >>>> Hi yt users,
>     >>>>
>     >>>> Does anybody have experience installing the development
>     version of yt on
>     >>>> blue waters (as opposed to relying on the module
>     pre-provided)? If so,
>     >>>> do
>     >>>> you have any tips or gotchas? I will be attempting an
>     installation in
>     >>>> the
>     >>>> next few days and I'm sort of hoping it's a solved problem.
>     >>>>
>     >>>> Best,
>     >>>> Jonah Miller
>     >>>> _______________________________________________
>     >>>> yt-users mailing list
>     >>>> yt-users at lists.spacepope.org <javascript:;>
>     >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>     >>>
>     >>>
>     >>>
>     >>> _______________________________________________
>     >>> yt-users mailing list
>     >>> yt-users at lists.spacepope.org <javascript:;>
>     >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>     >>>
>     >> _______________________________________________
>     >> yt-users mailing list
>     >> yt-users at lists.spacepope.org <javascript:;>
>     >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>     >
>     >
>     > _______________________________________________
>     > yt-users mailing list
>     > yt-users at lists.spacepope.org <javascript:;>
>     > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>     _______________________________________________
>     yt-users mailing list
>     yt-users at lists.spacepope.org <javascript:;>
>     http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160420/df457280/attachment-0001.htm>


More information about the yt-users mailing list