[yt-users] inline yt in parallel
Nathan Goldbaum
nathan12343 at gmail.com
Thu Apr 11 16:35:09 PDT 2013
Hi Christine,
This is a total stab in the dark, but have you tried running on four processors? I think in principle it should work on 3 cores but I bet it's not a commonly used configuration.
-Nathan
On Apr 11, 2013, at 4:29 PM, Christine Simpson <csimpson at astro.columbia.edu> wrote:
> Thanks so much for your help!
>
> So I did some tests. I ran each test on 3 processors for 10 cycles and
> set PythonTopGridSkip = 1. The hierarchy is static (so no amr).
>
>> type 1:
>>
>> import numpy as np
>>
>> def main():
>> print "Type 1"
>>
> This worked. Type 1 printed a total of 60 times, which is
> ncycles*nproc*2 (for the 2 calls to python, one at the end of evolve
> level and the one at the end of evolve hierarchy).
>
>> type 2:
>>
>> from yt.mods import *
>>
>> def main:
>> print "Type 2"
>>
>
> This failed. The error I get is
>
> Successfully read in parameter file StarParticleTest.enzo.
> INITIALIZATION TIME = 4.38757896e-01
> yt : [INFO ] 2013-04-11 13:14:08,924 Global parallel computation
> enabled: 1 / 3
> yt : [INFO ] 2013-04-11 13:14:08,924 Global parallel computation
> enabled: 2 / 3
> yt : [INFO ] 2013-04-11 13:14:08,924 Global parallel computation
> enabled: 0 / 3
> MPI process (rank: 0) terminated unexpectedly on trestles-1-10.local
> Exit code -5 signaled from trestles-1-10
>
>
>> type 3:
>>
>> type 1:
>>
>> from mpi4py import MPI
>>
>> def main():
>> print "Type 3"
>>
>>
>
> This works and prints out Type 3 a total of 60 times as in case 1.
>
>> One last thing to check is the output of ldd on the MPI.so file inside
>> your yt site-packages directory, under the mpi4py subdirectory, versus
>> the output of ldd for enzo.exe. These should both link to the same
>> MPI library.
>>
> I'm not sure what the MPI library is called, but all the directory files
> listed for MPI.so from the yt site-packages directory and for the enzo
> executable look the same. They are pasted below.
>
> So it looks like it is something with yt.mods/yt.pmods? (I wasn't quite
> sure which one to use, by the way). So I can try putting some print
> statements in that file to see how far it gets. Are there likely
> problem places that I should focus on?
>
> Thanks
> Christine
>
> For MPI.so in the yt site-packages directory
> (yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd MPI.so
> libpython2.7.so.1.0
> => /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
> (0x00002b9930541000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b993091d000)
> librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x00002b9930b39000)
> libibverbs.so.1 => /usr/lib64/libibverbs.so.1
> (0x00002b9930d3e000)
> libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b9930f4b000)
> libdl.so.2 => /lib64/libdl.so.2 (0x00002b9931152000)
> librt.so.1 => /lib64/librt.so.1 (0x00002b9931356000)
> libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
> (0x00002b993155f000)
> libsvml.so
> => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so
> (0x00002b99318f4000)
> libm.so.6 => /lib64/libm.so.6 (0x00002b9931b0a000)
> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b9931d8d000)
> libintlc.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
> (0x00002b9931f9c000)
> libc.so.6 => /lib64/libc.so.6 (0x00002b99320da000)
> libutil.so.1 => /lib64/libutil.so.1 (0x00002b9932431000)
> /lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
>
> For enzo.exe:
> (yt-x86_64-shared)[csimpson at trestles-login1 mpi4py]$ ldd
> ~/temp_mom/enzo-dev-mom/src/enzo/enzo.exe
> libhdf5.so.7 => /home/csimpson/yt-x86_64-shared/lib/libhdf5.so.7
> (0x00002b9496c96000)
> libifcore.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libifcore.so.5
> (0x00002b9497139000)
> libifport.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libifport.so.5
> (0x00002b94973b1000)
> libpython2.7.so.1.0
> => /home/csimpson/yt-x86_64-shared/lib/libpython2.7.so.1.0
> (0x00002b94974ea000)
> libreadline.so.5 => /usr/lib64/libreadline.so.5
> (0x0000003d96c00000)
> libtermcap.so.2 => /lib64/libtermcap.so.2 (0x000000346bc00000)
> libutil.so.1 => /lib64/libutil.so.1 (0x0000003452200000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x000000344ca00000)
> librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x000000344ce00000)
> libibverbs.so.1 => /usr/lib64/libibverbs.so.1
> (0x000000344c200000)
> libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b94978c8000)
> libdl.so.2 => /lib64/libdl.so.2 (0x000000344c600000)
> librt.so.1 => /lib64/librt.so.1 (0x000000344da00000)
> libm.so.6 => /lib64/libm.so.6 (0x00002b9497acf000)
> libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000035f8a00000)
> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000035f8e00000)
> libc.so.6 => /lib64/libc.so.6 (0x000000344be00000)
> libz.so.1 => /home/csimpson/yt-x86_64-shared/lib/libz.so.1
> (0x00002b9497d53000)
> libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
> (0x00002b9497f6a000)
> libintlc.so.5
> => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5
> (0x00002b94982fe000)
> /lib64/ld-linux-x86-64.so.2 (0x000000344ba00000)
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
More information about the yt-users
mailing list