I was able to reproduce the missing halo problem for the HOP in parallel compare to serial, but I wasn't able to reproduce the parallelHF problem. I'm running this on a 64 cube test dataset, so it's not as big as the 512 cube. This is using both my old YT and the version after Matt's pull request was merged. So the HOP in parallel bug existed before the memory update.<div>
<br></div><div>From</div><div>G.S.<br><br><div class="gmail_quote">On Fri, Feb 24, 2012 at 3:36 PM, Michael Kuhlen <span dir="ltr"><<a href="mailto:mqk@astro.berkeley.edu">mqk@astro.berkeley.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Sorry to be the bearer of bad news again, but ParallelHF failed for me.<br>
<br>
The installation of Forthon, etc. went smoothly. Only took 5 minutes.<br>
Note that in addition to replacing HaloFinder with<br>
parallelHF you also have to add "from<br>
yt.analysis_modules.halo_finding.api import *" to the top of the<br>
script.<br>
<br>
The error I encounter seems to be related to the large grid count that<br>
was also giving me problems yesterday. Here's the traceback.<br>
<br>
File "test.py", line 6, in <module><br>
halo_list = parallelHF(pf)<br>
File "/home/mqk/local/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py",<br>
line 2047, in __init__<br>
total_mass =<br>
self.comm.mpi_allreduce((self._data_source["ParticleMassMsun"].astype('float64')).sum(),<br>
File "/home/mqk/local/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/data_objects/data_containers.py",<br>
line 321, in __getitem__<br>
self.get_data(key)<br>
File "/home/mqk/local/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/data_objects/data_containers.py",<br>
line 2423, in get_data<br>
self.particles.get_data(field)<br>
File "/home/mqk/local/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/data_objects/particle_io.py",<br>
line 99, in get_data<br>
count=len(grid_list), dtype='float64'))<br>
ValueError: iterator too short<br>
<br>
Again it's the total_mass calculation that is causing the problem.<br>
Note that this yt install includes the changes that Matt brought in<br>
yesterday to fix the memory consumption.<br>
<br>
When I specify total_mass manually, everything works and I get the<br>
correct number of halos in the final catalog. The number of halos and<br>
their properties are not exactly identical to serial Hop, but it's<br>
close enough. I got a speed-up of about a factor of 3, when running<br>
parallelHF on 8 cores vs. HaloFinder on 1 core.<br>
<br>
Mike<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
On Fri, Feb 24, 2012 at 2:37 PM, Michael Kuhlen <<a href="mailto:mqk@astro.berkeley.edu">mqk@astro.berkeley.edu</a>> wrote:<br>
> No, I haven't yet. From the documentation ParallelHop seems to require<br>
> installing Forthon, and I was lazy. ;) I'll give that a shot. Thanks<br>
> for the suggestion.<br>
><br>
> Mike<br>
><br>
> On Fri, Feb 24, 2012 at 2:31 PM, Britton Smith <<a href="mailto:brittonsmith@gmail.com">brittonsmith@gmail.com</a>> wrote:<br>
>> Hi Mike,<br>
>><br>
>> Perhaps I missed something from the previous discussion. I've been locked<br>
>> away working on proposals all week. Out of curiosity, have you tried using<br>
>> ParallelHop, which is Stephen's specially designed parallel Hop. It is<br>
>> entirely distinct from simply running hop in parallel. Stephen, please<br>
>> correct me if I'm wrong.<br>
>><br>
>> You should be able to use that simply by replacing HaloFinder with<br>
>> parallelHF in your script.<br>
>><br>
>> Britton<br>
>><br>
>> On Fri, Feb 24, 2012 at 5:23 PM, Stephen Skory <<a href="mailto:s@skory.us">s@skory.us</a>> wrote:<br>
>>><br>
>>> Hi Mike,<br>
>>><br>
>>> > Is running Hop in parallel currently broken? I get way fewer halos in<br>
>>> > my catalog when I run Hop with 8 processors than with 1. Am I doing<br>
>>> > something wrong?<br>
>>><br>
>>> Why do you always got to start trouble? I'm getting this too. I'll dig<br>
>>> into this.<br>
>>><br>
>>> --<br>
>>> Stephen Skory<br>
>>> <a href="mailto:s@skory.us">s@skory.us</a><br>
>>> <a href="http://stephenskory.com/" target="_blank">http://stephenskory.com/</a><br>
>>> <a href="tel:510.621.3687" value="+15106213687">510.621.3687</a> (google voice)<br>
>>> _______________________________________________<br>
>>> yt-users mailing list<br>
>>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> yt-users mailing list<br>
>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>><br>
><br>
><br>
><br>
> --<br>
> *********************************************************************<br>
> * *<br>
> * Dr. Michael Kuhlen Theoretical Astrophysics Center *<br>
> * email: <a href="mailto:mqk@astro.berkeley.edu">mqk@astro.berkeley.edu</a> UC Berkeley *<br>
> * cell phone: <a href="tel:%28831%29%20588-1468" value="+18315881468">(831) 588-1468</a> B-116 Hearst Field Annex # 3411 *<br>
> * skype username: mikekuhlen Berkeley, CA 94720 *<br>
> * *<br>
> *********************************************************************<br>
<br>
<br>
<br>
--<br>
*********************************************************************<br>
* *<br>
* Dr. Michael Kuhlen Theoretical Astrophysics Center *<br>
* email: <a href="mailto:mqk@astro.berkeley.edu">mqk@astro.berkeley.edu</a> UC Berkeley *<br>
* cell phone: <a href="tel:%28831%29%20588-1468" value="+18315881468">(831) 588-1468</a> B-116 Hearst Field Annex # 3411 *<br>
* skype username: mikekuhlen Berkeley, CA 94720 *<br>
* *<br>
*********************************************************************<br>
_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
</div></div></blockquote></div><br></div>