<div dir="ltr">> With the last good changeset the total memory usage never gets above ~8.5GB (estimated from top).<br><div class="gmail_extra"><br></div><div class="gmail_extra" style>Sorry, forgot to say: the last good changeset is 41358eecdad5.</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Feb 27, 2013 at 7:58 PM, Michael Kuhlen <span dir="ltr"><<a href="mailto:mqk@astro.berkeley.edu" target="_blank">mqk@astro.berkeley.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Hi all and Nathan specifically</div><div>
<br></div><div>I've found that the changes having to do with periodic fields are causing excessive memory usage for me when doing parallel halo profiling, to the point where I can no longer do analysis that used to work fine on my 24GB memory workstation.</div>
<div><br></div><div>A simple HaloProfiler script (<a href="http://paste.yt-project.org/show/3211/" target="_blank">http://paste.yt-project.org/show/3211/</a>), run in parallel on 8 processors on the halo catalog obtained from one of my cosmology simulations, currently almost immediate runs out of memory, while it used to complete without problems.</div>
<div><br></div><div>I tried to use hg bisect to find the problematic changeset but had to skip the testing of several revisions because of runtime yt errors, so in the end bisect only gave me a range of potentially bad revisions: <a href="http://paste.yt-project.org/show/3210/" target="_blank">http://paste.yt-project.org/show/3210/</a>. As you can see, they're all related to the periodic radius mods from about a month ago.</div>
<div><br></div><div><div>I tried replicating this with the standard Enzo_64 example dataset from <a href="http://yt-project.org/data/" target="_blank">http://yt-project.org/data/</a>, but I guess that one is too small to produce this problem on my machine. If you want to see the problem for yourself, then you can download this tarball (<a href="http://astro.berkeley.edu/~mqk/transfer/RD0003.tar" target="_blank">http://astro.berkeley.edu/~mqk/transfer/RD0003.tar</a> 4.5GB) and run this script (<a href="http://paste.yt-project.org/show/3211/" target="_blank">http://paste.yt-project.org/show/3211/</a>) on it, like so:</div>
<div><br></div><div>$ mpirun -np 8 python ./profile_halos.py --parallel</div><div><br></div><div>When I run this with the current tip (ccfe34e70803) the total memory usage grows to >24GB. With the last good changeset the total memory usage never gets above ~8.5GB (estimated from top).</div>
<div><br></div><div>It'd be great if we could find out what the problem is and fix it, because this is a major performance regression for me, that, as I already said, is seriously impacting my ability to do analysis with the current yt tip. Let me know if I should file a BB issue about this, and/or if there's some way I can assist with fixing this.</div>
</div><div><br></div><div>Cheers,</div><div>Mike</div><span class=""><font color="#888888"><div><br></div>-- <br>*********************************************************************<br>* *<br>
* Dr. Michael Kuhlen Theoretical Astrophysics Center *<br>* email: <a href="mailto:mqk@astro.berkeley.edu" target="_blank">mqk@astro.berkeley.edu</a> UC Berkeley *<br>* cell phone: <a href="tel:%28831%29%20588-1468" value="+18315881468" target="_blank">(831) 588-1468</a> B-116 Hearst Field Annex # 3411 *<br>
* skype username: mikekuhlen Berkeley, CA 94720 *<br>* *<br>*********************************************************************<br>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>*********************************************************************<br>* *<br>* Dr. Michael Kuhlen Theoretical Astrophysics Center *<br>
* email: <a href="mailto:mqk@astro.berkeley.edu" target="_blank">mqk@astro.berkeley.edu</a> UC Berkeley *<br>* cell phone: (831) 588-1468 B-116 Hearst Field Annex # 3411 *<br>* skype username: mikekuhlen Berkeley, CA 94720 *<br>
* *<br>*********************************************************************<br>
</div></div>