[yt-users] Running YT in Parallel on Nautilus

[gso at physics.ucsd.edu]

I guess this would be a sort of followup to Stephen's email of running YT
on Kraken, here's what I figured out combined with what I got from the
system admin:

Installing mpi4py can be a little more troublesome because there's no
mpicc wrapper, you'll have to edit the mpi.cfg file and put in:

mpicc         = gcc -lmpi
mpicxx        = g++ -lmpi++ -lmpi

then do:
python setup.py build --mpi=<your mpi section name in mpi.cfg>
python setup.py install

(following http://mpi4py.scipy.org/docs/usrman/install.html)

next you'll have to set these environment variables, I have it in a
.bashrc and source it every time I log into Nautilus.

module swap mpt mpt/2.03
export MPI_SHARED_NEIGHBORHOOD=HOST

Last but not least, the system admin told me SGI identified a bug so that
mpi at least under python works for only up to 8 processors.  I've tried
16 and failed, tried 8 and ran, so I believe their story.

so under an interactive queue, you'll want to do something like:
mpiexec -np 8 python script.py --parallel
to run YT in parallel

Hope this helps

From
G.S.

> Hi All,
>
> I've just updated the common install of yt on Kraken to 2.1, and added
> an install of the unstable branch. Both are full copies of yt that is
> kept on lustre in a non-volatile place. This means that it is
> accessible from both the login nodes and the compute nodes, but unlike
> most files on Lustre, will not be deleted automatically.
>
> If you wish to use yt 2.1, which will only see changes for bug fixes,
> please set your environment using these settings:
>
> YT_DEST         => /lustre/scratch/proj/yt_common/2.1
> PATH            => /lustre/scratch/proj/yt_common/2.1/bin/
> PYTHONPATH      =>
> /lustre/scratch/proj/yt_common/2.1/lib/python2.7/site-packages/
> LD_LIBRARY_PATH => /lustre/scratch/proj/yt_common/2.1/lib/
>
> If you wish to use the unstable branch, which I'll update on Kraken
> when I remember to do it (or you can bug me if I haven't):
>
> YT_DEST         => /lustre/scratch/proj/yt_common/unstable
> PATH            => /lustre/scratch/proj/yt_common/unstable/bin/
> PYTHONPATH      =>
> /lustre/scratch/proj/yt_common/unstable/lib/python2.7/site-packages/
> LD_LIBRARY_PATH => /lustre/scratch/proj/yt_common/unstable/lib/
>
> The older versions of yt at /lustre/scratch/proj/yt_common/2.0 and
> /lustre/scratch/proj/yt_common/trunk will be deleted within a week or
> so.
>
>
> --
> Stephen Skory
> s at skory.us
> http://stephenskory.com/
> 510.621.3687 (google voice)
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>