[yt-users] inline yt in parallel

Christine Simpson csimpson at astro.columbia.edu
Thu Apr 11 16:46:14 PDT 2013


Hi Nathan,

Yes, I get the same errors with 2 and 4 processors, the parity of the
processor number doesn't seem to matter.  I'm using 3 processors only
because for the problem I'm doing, I don't want enzo to draw a grid
boundary through the center of the simulation box (which it would do for
an even number of processors).

Thanks
Christine

On Thu, 2013-04-11 at 16:35 -0700, Nathan Goldbaum wrote:
> Hi Christine,
> 
> This is a total stab in the dark, but have you tried running on four processors?  I think in principle it should work on 3 cores but I bet it's not a commonly used configuration.
> 
> -Nathan
> 
> On Apr 11, 2013, at 4:29 PM, Christine Simpson <csimpson at astro.columbia.edu> wrote:
> 
> > Thanks so much for your help!
> > 
> > So I did some tests.  I ran each test on 3 processors for 10 cycles and
> > set PythonTopGridSkip = 1.  The hierarchy is static (so no amr).
> > 
> >> type 1:
> >> 
> >> import numpy as np
> >> 
> >> def main():
> >>    print "Type 1"
> >> 
> > This worked.  Type 1 printed a total of 60 times, which is
> > ncycles*nproc*2 (for the 2 calls to python, one at the end of evolve
> > level and the one at the end of evolve hierarchy).
> > 
> >> type 2:
> >> 
> >> from yt.mods import *
> >> 
> >> def main:
> >>    print "Type 2"
> >> 
> > 
> > This failed.  The error I get is 
> > 
> > Successfully read in parameter file StarParticleTest.enzo.
> > INITIALIZATION TIME =   4.38757896e-01
> > yt : [INFO     ] 2013-04-11 13:14:08,924 Global parallel computation
> > enabled: 1 / 3
> > yt : [INFO     ] 2013-04-11 13:14:08,924 Global parallel computation
> > enabled: 2 / 3
> > yt : [INFO     ] 2013-04-11 13:14:08,924 Global parallel computation
> > enabled: 0 / 3
> > MPI process (rank: 0) terminated unexpectedly on trestles-1-10.local
> > Exit code -5 signaled from trestles-1-10
> > 
> > 
> >> type 3:
> >> 
> >> type 1:
> >> 
> >> from mpi4py import MPI
> >> 
> >> def main():
> >>    print "Type 3"
> >> 
> >> 
> > 
> > This works and prints out Type 3 a total of 60 times as in case 1.
> > 
> >> One last thing to check is the output of ldd on the MPI.so file inside
> >> your yt site-packages directory, under the mpi4py subdirectory, versus
> >> the output of ldd for enzo.exe.  These should both link to the same
> >> MPI library.
> >> 
> > I'm not sure what the MPI library is called, but all the directory files
> > listed for MPI.so from the yt site-packages directory and for the enzo
> > executable look the same.  They are pasted below.
> > 
> > So it looks like it is something with yt.mods/yt.pmods?  (I wasn't quite
> > sure which one to use, by the way).  So I can try putting some print
> > statements in that file to see how far it gets.  Are there likely
> > problem places that I should focus on?
> > 
> > Thanks
> > Christine
> > 
> > For MPI.so in the yt site-packages directory
> > (yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd MPI.so
> >        libpython2.7.so.1.0
> > => /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
> > (0x00002b9930541000)
> >        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b993091d000)
> >        librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x00002b9930b39000)
> >        libibverbs.so.1 => /usr/lib64/libibverbs.so.1
> > (0x00002b9930d3e000)
> >        libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b9930f4b000)
> >        libdl.so.2 => /lib64/libdl.so.2 (0x00002b9931152000)
> >        librt.so.1 => /lib64/librt.so.1 (0x00002b9931356000)
> >        libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
> > (0x00002b993155f000)
> >        libsvml.so
> > => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so
> > (0x00002b99318f4000)
> >        libm.so.6 => /lib64/libm.so.6 (0x00002b9931b0a000)
> >        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b9931d8d000)
> >        libintlc.so.5
> > => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
> > (0x00002b9931f9c000)
> >        libc.so.6 => /lib64/libc.so.6 (0x00002b99320da000)
> >        libutil.so.1 => /lib64/libutil.so.1 (0x00002b9932431000)
> >        /lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
> > 
> > For enzo.exe:
> > (yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd
> > ~/temp_mom/enzo-dev-mom/src/enzo/enzo.exe 
> >        libhdf5.so.7 => /home/csimpson/yt-x86_64-shared/lib/libhdf5.so.7
> > (0x00002b9496c96000)
> >        libifcore.so.5
> > => /opt/intel/Compiler/11.1/072/lib/intel64/libifcore.so.5
> > (0x00002b9497139000)
> >        libifport.so.5
> > => /opt/intel/Compiler/11.1/072/lib/intel64/libifport.so.5
> > (0x00002b94973b1000)
> >        libpython2.7.so.1.0
> > => /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
> > (0x00002b94974ea000)
> >        libreadline.so.5 => /usr/lib64/libreadline.so.5
> > (0x0000003d96c00000)
> >        libtermcap.so.2 => /lib64/libtermcap.so.2 (0x000000346bc00000)
> >        libutil.so.1 => /lib64/libutil.so.1 (0x0000003452200000)
> >        libpthread.so.0 => /lib64/libpthread.so.0 (0x000000344ca00000)
> >        librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x000000344ce00000)
> >        libibverbs.so.1 => /usr/lib64/libibverbs.so.1
> > (0x000000344c200000)
> >        libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b94978c8000)
> >        libdl.so.2 => /lib64/libdl.so.2 (0x000000344c600000)
> >        librt.so.1 => /lib64/librt.so.1 (0x000000344da00000)
> >        libm.so.6 => /lib64/libm.so.6 (0x00002b9497acf000)
> >        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000035f8a00000)
> >        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000035f8e00000)
> >        libc.so.6 => /lib64/libc.so.6 (0x000000344be00000)
> >        libz.so.1 => /home/csimpson/yt-x86_64-shared/lib/libz.so.1
> > (0x00002b9497d53000)
> >        libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
> > (0x00002b9497f6a000)
> >        libintlc.so.5
> > => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
> > (0x00002b94982fe000)
> >        /lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
> > 
> > 
> > 
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> 





More information about the yt-users mailing list