[yt-users] inline yt in parallel

Nathan Goldbaum nathan12343 at gmail.com
Thu Apr 11 16:35:09 PDT 2013


Hi Christine,

This is a total stab in the dark, but have you tried running on four processors?  I think in principle it should work on 3 cores but I bet it's not a commonly used configuration.

-Nathan

On Apr 11, 2013, at 4:29 PM, Christine Simpson <csimpson at astro.columbia.edu> wrote:

> Thanks so much for your help!
> 
> So I did some tests.  I ran each test on 3 processors for 10 cycles and
> set PythonTopGridSkip = 1.  The hierarchy is static (so no amr).
> 
>> type 1:
>> 
>> import numpy as np
>> 
>> def main():
>>    print "Type 1"
>> 
> This worked.  Type 1 printed a total of 60 times, which is
> ncycles*nproc*2 (for the 2 calls to python, one at the end of evolve
> level and the one at the end of evolve hierarchy).
> 
>> type 2:
>> 
>> from yt.mods import *
>> 
>> def main:
>>    print "Type 2"
>> 
> 
> This failed.  The error I get is 
> 
> Successfully read in parameter file StarParticleTest.enzo.
> INITIALIZATION TIME =   4.38757896e-01
> yt : [INFO     ] 2013-04-11 13:14:08,924 Global parallel computation
> enabled: 1 / 3
> yt : [INFO     ] 2013-04-11 13:14:08,924 Global parallel computation
> enabled: 2 / 3
> yt : [INFO     ] 2013-04-11 13:14:08,924 Global parallel computation
> enabled: 0 / 3
> MPI process (rank: 0) terminated unexpectedly on trestles-1-10.local
> Exit code -5 signaled from trestles-1-10
> 
> 
>> type 3:
>> 
>> type 1:
>> 
>> from mpi4py import MPI
>> 
>> def main():
>>    print "Type 3"
>> 
>> 
> 
> This works and prints out Type 3 a total of 60 times as in case 1.
> 
>> One last thing to check is the output of ldd on the MPI.so file inside
>> your yt site-packages directory, under the mpi4py subdirectory, versus
>> the output of ldd for enzo.exe.  These should both link to the same
>> MPI library.
>> 
> I'm not sure what the MPI library is called, but all the directory files
> listed for MPI.so from the yt site-packages directory and for the enzo
> executable look the same.  They are pasted below.
> 
> So it looks like it is something with yt.mods/yt.pmods?  (I wasn't quite
> sure which one to use, by the way).  So I can try putting some print
> statements in that file to see how far it gets.  Are there likely
> problem places that I should focus on?
> 
> Thanks
> Christine
> 
> For MPI.so in the yt site-packages directory
> (yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd MPI.so
>        libpython2.7.so.1.0
> => /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
> (0x00002b9930541000)
>        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b993091d000)
>        librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x00002b9930b39000)
>        libibverbs.so.1 => /usr/lib64/libibverbs.so.1
> (0x00002b9930d3e000)
>        libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b9930f4b000)
>        libdl.so.2 => /lib64/libdl.so.2 (0x00002b9931152000)
>        librt.so.1 => /lib64/librt.so.1 (0x00002b9931356000)
>        libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
> (0x00002b993155f000)
>        libsvml.so
> => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so
> (0x00002b99318f4000)
>        libm.so.6 => /lib64/libm.so.6 (0x00002b9931b0a000)
>        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b9931d8d000)
>        libintlc.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
> (0x00002b9931f9c000)
>        libc.so.6 => /lib64/libc.so.6 (0x00002b99320da000)
>        libutil.so.1 => /lib64/libutil.so.1 (0x00002b9932431000)
>        /lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
> 
> For enzo.exe:
> (yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd
> ~/temp_mom/enzo-dev-mom/src/enzo/enzo.exe 
>        libhdf5.so.7 => /home/csimpson/yt-x86_64-shared/lib/libhdf5.so.7
> (0x00002b9496c96000)
>        libifcore.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libifcore.so.5
> (0x00002b9497139000)
>        libifport.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libifport.so.5
> (0x00002b94973b1000)
>        libpython2.7.so.1.0
> => /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
> (0x00002b94974ea000)
>        libreadline.so.5 => /usr/lib64/libreadline.so.5
> (0x0000003d96c00000)
>        libtermcap.so.2 => /lib64/libtermcap.so.2 (0x000000346bc00000)
>        libutil.so.1 => /lib64/libutil.so.1 (0x0000003452200000)
>        libpthread.so.0 => /lib64/libpthread.so.0 (0x000000344ca00000)
>        librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x000000344ce00000)
>        libibverbs.so.1 => /usr/lib64/libibverbs.so.1
> (0x000000344c200000)
>        libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b94978c8000)
>        libdl.so.2 => /lib64/libdl.so.2 (0x000000344c600000)
>        librt.so.1 => /lib64/librt.so.1 (0x000000344da00000)
>        libm.so.6 => /lib64/libm.so.6 (0x00002b9497acf000)
>        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000035f8a00000)
>        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000035f8e00000)
>        libc.so.6 => /lib64/libc.so.6 (0x000000344be00000)
>        libz.so.1 => /home/csimpson/yt-x86_64-shared/lib/libz.so.1
> (0x00002b9497d53000)
>        libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
> (0x00002b9497f6a000)
>        libintlc.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
> (0x00002b94982fe000)
>        /lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
> 
> 
> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org




More information about the yt-users mailing list