<div>Hi Joseph,</div><div><br></div>The last line which says MemoryError makes me want to attribute to the machine running out of memory, I'm just guessing so it might not be the case at all. Can you tell us a little bit about the memory available to your machine (GB per core), and the number of particles in your simulation? <div>
<br>In my past experience with Parallel HOP I believe a safe guideline has been to have 1MB of RAM per 5000 particles. YT has since been optimized further so that number should be smaller now, but it would be a safe place to start if you're having trouble. I'm guessing if you have 1 particle per cell, then 1024**2/5000/32 ~ 6710, so you'll need about 7GB per core if using 32 cores. If your machine has 4GB per core, you might want to try 64 cores for the job.</div>
<div><div><br></div><div>Hope this helps.</div><div><br></div><div>From</div><div>G.S.<br><br><div class="gmail_quote">On Thu, Oct 4, 2012 at 10:14 PM, Joseph Smidt <span dir="ltr"><<a href="mailto:josephsmidt@gmail.com" target="_blank">josephsmidt@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">Hey everyone,<br>
<br>
I am trying to use Parallel Hop in YT to analyze enzo data. I<br>
installed mpi4py, forthon and did the whole "python setup.py install"<br>
afterwards. I next try to find halos with this code on 2 nodes with<br>
16 processors each (32 total):<br>
<br>
<br>
</div><div class="im">from yt.mods import *<br>
from yt.analysis_modules.halo_finding.api import *<br>
<br>
i = 5<br>
filename = 'RD%04d/RedshiftOutput%04d' % (i,i)<br>
pf = load(filename)<br>
halos = parallelHF(pf)<br>
<br>
dumpn = 'RD%04d/MergerHalos' %i<br>
halos.dump(dumpn)<br>
<br>
<br>
</div>The output is rather long since it has 32 processors of output. The<br>
full output is here: <a href="http://paste.yt-project.org/show/2761/" target="_blank">http://paste.yt-project.org/show/2761/</a><br>
<br>
However, here are some highlights:<br>
<br>
$ mpirun -np 32 python findhalo.py --parallel<br>
Reported: 2 (out of 2) daemons - 32 (out of 32) procs<br>
yt : [INFO ] 2012-10-04 22:54:51,855 Global parallel computation<br>
enabled: 1 / 32<br>
yt : [INFO ] 2012-10-04 22:54:51,855 Global parallel computation<br>
enabled: 21 / 32<br>
....<br>
yt : [INFO ] 2012-10-04 22:54:51,858 Global parallel computation<br>
enabled: 10 / 32<br>
<div class="im">--------------------------------------------------------------------------<br>
An MPI process has executed an operation involving a call to the<br>
"fork()" system call to create a child process. Open MPI is currently<br>
operating in a condition that could result in memory corruption or<br>
other system errors; your MPI job may hang, crash, or produce silent<br>
data corruption. The use of fork() (or system() or other calls that<br>
create child processes) is strongly discouraged.<br>
<br>
The process that invoked fork was:<br>
<br>
</div> Local host: mu0002.localdomain (PID 9624)<br>
<div class="im"> MPI_COMM_WORLD rank: 3<br>
<br>
If you are *absolutely sure* that your application will successfully<br>
and correctly survive a call to fork(), you may disable this warning<br>
by setting the mpi_warn_on_fork MCA parameter to 0.<br>
--------------------------------------------------------------------------<br>
</div>P000 yt : [INFO ] 2012-10-04 22:54:55,571 Parameters: current_time<br>
= 89.9505268216<br>
P000 yt : [INFO ] 2012-10-04 22:54:55,571 Parameters:<br>
<div class="im">domain_dimensions = [1024 1024 1024]<br>
</div>P000 yt : [INFO ] 2012-10-04 22:54:55,572 Parameters:<br>
<div class="im">domain_left_edge = [ 0. 0. 0.]<br>
</div>P000 yt : [INFO ] 2012-10-04 22:54:55,572 Parameters:<br>
<div class="im">domain_right_edge = [ 1. 1. 1.]<br>
</div>P000 yt : [INFO ] 2012-10-04 22:54:55,573 Parameters:<br>
cosmological_simulation = 1<br>
P000 yt : [INFO ] 2012-10-04 22:54:55,573 Parameters:<br>
current_redshift = 5.99999153008<br>
P000 yt : [INFO ] 2012-10-04 22:54:55,573 Parameters: omega_lambda<br>
= 0.724<br>
...<br>
P000 yt : [INFO ] 2012-10-04 23:04:33,681 Getting particle_index<br>
using ParticleIO<br>
P001 yt : [INFO ] 2012-10-04 23:05:09,222 Getting particle_index<br>
using ParticleIO<br>
<div class="im">Traceback (most recent call last):<br>
</div> File "findhalo.py", line 7, in <module><br>
<div class="im"> halos = parallelHF(pf)<br>
File "/usr/projects/magnetic/jsmidt/yt-x86_64/lib/python2.7/site-packages/yt-2.5dev-py2.7-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py",<br>
line 2268, in __init__<br>
</div> premerge=premerge, tree=self.tree)<br>
<div class="im"> File "/usr/projects/magnetic/jsmidt/yt-x86_64/lib/python2.7/site-packages/yt-2.5dev-py2.7-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py",<br>
line 1639, in __init__<br>
HaloList.__init__(self, data_source, dm_only)<br>
File "/usr/projects/magnetic/jsmidt/yt-x86_64/lib/python2.7/site-packages/yt-2.5dev-py2.7-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py",<br>
line 1067, in __init__<br>
self._run_finder()<br>
File "/usr/projects/magnetic/jsmidt/yt-x86_64/lib/python2.7/site-packages/yt-2.5dev-py2.7-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py",<br>
</div>line 1648, in _run_finder<br>
if np.unique(self.particle_fields["particle_index"]).size != \<br>
File "/usr/projects/magnetic/jsmidt/yt-x86_64/lib/python2.7/site-packages/numpy/lib/arraysetops.py",<br>
line 193, in unique<br>
return ar[flag]<br>
MemoryError<br>
mpirun: killing job...<br>
--------------------------------------------------------------------------<br>
mpirun noticed that process rank 0 with PID 6295 on node<br>
mu0001.localdomain exited on signal 0 (Unknown signal 0).<br>
<div class="im">--------------------------------------------------------------------------<br>
32 total processes killed (some possibly by mpirun during cleanup)<br>
<br>
<br>
<br>
</div> Anyways, if anyone recognizes this or has any advice it would be<br>
appreciated. Thanks.<br>
<div class="im"><br>
--<br>
------------------------------------------------------------------------<br>
Joseph Smidt <<a href="mailto:josephsmidt@gmail.com">josephsmidt@gmail.com</a>><br>
<br>
Theoretical Division<br>
P.O. Box 1663, Mail Stop B283<br>
Los Alamos, NM 87545<br>
Office: <a href="tel:505-665-9752" value="+15056659752">505-665-9752</a><br>
Fax: <a href="tel:505-667-1931" value="+15056671931">505-667-1931</a><br>
</div>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
</blockquote></div><br></div></div>