[yt-users] inline yt in parallel
Christine Simpson
csimpson at astro.columbia.edu
Thu Apr 11 16:29:07 PDT 2013
Thanks so much for your help!
So I did some tests. I ran each test on 3 processors for 10 cycles and
set PythonTopGridSkip = 1. The hierarchy is static (so no amr).
> type 1:
>
> import numpy as np
>
> def main():
> print "Type 1"
>
This worked. Type 1 printed a total of 60 times, which is
ncycles*nproc*2 (for the 2 calls to python, one at the end of evolve
level and the one at the end of evolve hierarchy).
> type 2:
>
> from yt.mods import *
>
> def main:
> print "Type 2"
>
This failed. The error I get is
Successfully read in parameter file StarParticleTest.enzo.
INITIALIZATION TIME = 4.38757896e-01
yt : [INFO ] 2013-04-11 13:14:08,924 Global parallel computation
enabled: 1 / 3
yt : [INFO ] 2013-04-11 13:14:08,924 Global parallel computation
enabled: 2 / 3
yt : [INFO ] 2013-04-11 13:14:08,924 Global parallel computation
enabled: 0 / 3
MPI process (rank: 0) terminated unexpectedly on trestles-1-10.local
Exit code -5 signaled from trestles-1-10
> type 3:
>
> type 1:
>
> from mpi4py import MPI
>
> def main():
> print "Type 3"
>
>
This works and prints out Type 3 a total of 60 times as in case 1.
> One last thing to check is the output of ldd on the MPI.so file inside
> your yt site-packages directory, under the mpi4py subdirectory, versus
> the output of ldd for enzo.exe. These should both link to the same
> MPI library.
>
I'm not sure what the MPI library is called, but all the directory files
listed for MPI.so from the yt site-packages directory and for the enzo
executable look the same. They are pasted below.
So it looks like it is something with yt.mods/yt.pmods? (I wasn't quite
sure which one to use, by the way). So I can try putting some print
statements in that file to see how far it gets. Are there likely
problem places that I should focus on?
Thanks
Christine
For MPI.so in the yt site-packages directory
(yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd MPI.so
libpython2.7.so.1.0
=> /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
(0x00002b9930541000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b993091d000)
librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x00002b9930b39000)
libibverbs.so.1 => /usr/lib64/libibverbs.so.1
(0x00002b9930d3e000)
libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b9930f4b000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002b9931152000)
librt.so.1 => /lib64/librt.so.1 (0x00002b9931356000)
libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
(0x00002b993155f000)
libsvml.so
=> /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so
(0x00002b99318f4000)
libm.so.6 => /lib64/libm.so.6 (0x00002b9931b0a000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b9931d8d000)
libintlc.so.5
=> /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
(0x00002b9931f9c000)
libc.so.6 => /lib64/libc.so.6 (0x00002b99320da000)
libutil.so.1 => /lib64/libutil.so.1 (0x00002b9932431000)
/lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
For enzo.exe:
(yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd
~/temp_mom/enzo-dev-mom/src/enzo/enzo.exe
libhdf5.so.7 => /home/csimpson/yt-x86_64-shared/lib/libhdf5.so.7
(0x00002b9496c96000)
libifcore.so.5
=> /opt/intel/Compiler/11.1/072/lib/intel64/libifcore.so.5
(0x00002b9497139000)
libifport.so.5
=> /opt/intel/Compiler/11.1/072/lib/intel64/libifport.so.5
(0x00002b94973b1000)
libpython2.7.so.1.0
=> /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
(0x00002b94974ea000)
libreadline.so.5 => /usr/lib64/libreadline.so.5
(0x0000003d96c00000)
libtermcap.so.2 => /lib64/libtermcap.so.2 (0x000000346bc00000)
libutil.so.1 => /lib64/libutil.so.1 (0x0000003452200000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x000000344ca00000)
librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x000000344ce00000)
libibverbs.so.1 => /usr/lib64/libibverbs.so.1
(0x000000344c200000)
libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b94978c8000)
libdl.so.2 => /lib64/libdl.so.2 (0x000000344c600000)
librt.so.1 => /lib64/librt.so.1 (0x000000344da00000)
libm.so.6 => /lib64/libm.so.6 (0x00002b9497acf000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000035f8a00000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000035f8e00000)
libc.so.6 => /lib64/libc.so.6 (0x000000344be00000)
libz.so.1 => /home/csimpson/yt-x86_64-shared/lib/libz.so.1
(0x00002b9497d53000)
libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
(0x00002b9497f6a000)
libintlc.so.5
=> /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
(0x00002b94982fe000)
/lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
More information about the yt-users
mailing list