[yt-users] Running Rockstar Halo Finder to create a merger tree

Tue Nov 5 06:28:22 PST 2013

Hi Hilary,

I think Rockstar is your best bet.  Unfortunately there have been some
changes in the yt/Rockstar interface that may be causing issues.

We've improved these considerably in the yt-3.0 branch, and in fact
I'd recommend you give that a shot.  If you are able, try using
yt_analysis/yt-3.0 and the Rockstar branch that can be found here:
http://bitbucket.org/MatthewTurk/rockstar .  This includes a number of
forward-facing changes that we're hoping to roll out as we revamp
halos in the 3.x series.

-Matt

On Mon, Nov 4, 2013 at 8:37 PM, Hilary Egan <hilaryye at gmail.com> wrote:
> Sam: Yep, I am definitely sending the infiniband-disabling flags and running
> it as follows
>
> mpirun -np 48 --mca btl ^openib python halos.py --parallel
>
> Cameron: I had originally switched to rockstar from HOP because I couldn't
> get it to run on kraken due to (very clear) out of memory issues. I suppose
> that since I've switched supercomputers anyway, I might as well play around
> with HOP again.
>
>
> On Mon, Nov 4, 2013 at 5:57 PM, Sam Skillman <samskillman at gmail.com> wrote:
>>
>> Cameron -- My experience has been that HOP/FOF are slow on datasets this
>> large, and PHOP uses a large amount of memory.  Anyways, regardless of that
>> we should figure out what is going on with rockstar given its use in the
>> community.
>>
>> Hilary -- Just to be clear, you are sending the infiniband-disabling flags
>> to this last run, correct?
>>
>>
>>
>> On Mon, Nov 4, 2013 at 4:36 PM, Cameron Hummels <chummels at gmail.com>
>> wrote:
>>>
>>> Are you tied to using rockstar?  I know yt works well with HOP, PHOP, and
>>> FOF without many problems...
>>>
>>>
>>> On Mon, Nov 4, 2013 at 3:50 PM, Hilary Egan <hilaryye at gmail.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I'm still having trouble getting rockstar to run on my dataset. I've
>>>> moved my data to a different super computer that uses the mpirun command
>>>> rather than kraken's aprun so that I could make sure there aren't any
>>>> infiniband issues, but Im still seeing similar issues. I've also determined
>>>> I can run the halo finder just fine on a smaller test dataset, which is
>>>> leading me to believe that its some sort of memory issue, but I can't quite
>>>> figure out how I would go about fixing it. I've tried playing with the
>>>> number of readers and the number of nodes I'm running on, to no avail. For
>>>> reference, the dataset is a 1024^3 unigrid enzo run. If anyone has any
>>>> suggestions, I'd love to hear them!
>>>>
>>>> Thanks!
>>>> Hilary
>>>>
>>>> Script: http://paste.yt-project.org/show/4025/
>>>>
>>>> Error message: http://paste.yt-project.org/show/4024/
>>>>
>>>>
>>>> On Wed, Oct 30, 2013 at 8:50 AM, John Wise <jwise at physics.gatech.edu>
>>>> wrote:
>>>>>
>>>>> Hi Hilary,
>>>>>
>>>>>
>>>>> On 10/29/2013 12:30 PM, Hilary Egan wrote:
>>>>>
>>>>>> I'm quite confused on a number of points related to running the
>>>>>> rockstar
>>>>>> halo finder, so I hope its alright that I put all these questions into
>>>>>> this one email!
>>>>>
>>>>>
>>>>> It's no problem at all to include all of your questions in a single
>>>>> email.  It's probably better this way!
>>>>>
>>>>>
>>>>>> 1. I can't seem to run the rockstar halo finder at all without getting
>>>>>> this error followed by a segmentation fault and crash.
>>>>>>
>>>>>> [Warning] Network IO Failure (PID XXXXXX): Connection reset by peer
>>>>>> [Network] Packet receive retry count at: 1
>>>>>>
>>>>>> It sort of seems like this issue
>>>>>>
>>>>>> (http://lists.spacepope.org/htdig.cgi/yt-dev-spacepope.org/2012-November/002681.html)
>>>>>> but I couldn't really figure out what the resolution was from the
>>>>>> thread. Im attempting to run this on kraken and it doesn't matter if I
>>>>>> use a single compute node or multiple, I get the same error. (I hope
>>>>>> this isn't the infiniband issue the docs warned about, I couldn't
>>>>>> figure
>>>>>> out if that is how kraken is connected and I got an error that the
>>>>>> suggested flag doesn't exist so I didn't press the issue.)
>>>>>
>>>>>
>>>>> I haven't seen that error before, but I still have to specific to *not*
>>>>> run on infiniband when running Rockstar on a single node.  With OpenMPI, you
>>>>> would use "mpirun -n 32 --mca btl ^openib ...", but I haven't done this on
>>>>> kraken with their aprun but hopefully it's easily accompolished!
>>>>>
>>>>>
>>>>>> 2. Whenever I finally do get the halo finder to work, I need the
>>>>>> results
>>>>>> to be in a form that the merger tree can use. It seems as though the
>>>>>> MergerTree needs the results in the same form as the other halo
>>>>>> finders
>>>>>> give, so would getting the halo list and then dumping it as usual be
>>>>>> the
>>>>>> appropriate strategy? Ie:
>>>>>>
>>>>>>          rh.run()
>>>>>>          halo_list = rh.halo_list()
>>>>>> halo_list.dump('MergerHalos')
>>>>>>
>>>>>> 2.5. The docs sort of give mixed messages on whether or not I could
>>>>>> just
>>>>>> be calling MergerTree with the argument halo_finder_function =
>>>>>> RockstarHaloFinder. At this point I've pretty thoroughly convinced
>>>>>> myself that I can't, but it would be nice if that was clarified. (Just
>>>>>> a
>>>>>> thoroughly overwhelmed new user's perspective!)
>>>>>
>>>>>
>>>>> I'm not sure whether you can use yt's merger tree code with the
>>>>> Rockstar halos.  I haven't tried.
>>>>>
>>>>> However, I've used Consistent Trees
>>>>>
>>>>> https://code.google.com/p/consistent-trees/
>>>>>
>>>>> with Rockstar's halo lists, which is also written by Peter Behroozi.
>>>>> I've chosen this route because the algorithm seems to be more physically
>>>>> robust in constructing parent/child relationships and boundness.  All of the
>>>>> instructions are in the README of the code, and it's pretty straightforward
>>>>> and fast (probably 5-10 minutes for a 512^3 simulation with 60 outputs) to
>>>>> run.
>>>>>
>>>>> I also have a visualizing script for consistent tree's output.
>>>>>
>>>>> https://bitbucket.org/jwise77/rockstar-dot
>>>>>
>>>>> From consistent tree's output, you can use the provided script,
>>>>> halo_trees_to_catalog.pl, (instructions also in the README) to convert the
>>>>> tree output into halo lists.
>>>>>
>>>>>
>>>>>> 3. I'm a little confused as to whether or not I have to use a
>>>>>> TimeSeriesData object rather than the usual single time output when
>>>>>> instantiating the halo finder. Under "Rockstar Halo Finding" it uses
>>>>>> TimeSeriesData, unlike the rest of the examples, but under the
>>>>>> subheading "Output Analysis" it just uses pf. The "Output Analysis"
>>>>>> example also doesn't call the run() method, which leads me to believe
>>>>>> something else entirely is going on, but its not quite clear.
>>>>>
>>>>>
>>>>> This actually came up recently.  It's best to supply a TimeSeriesData
>>>>> object.  Here's the link to the email for more details.
>>>>>
>>>>>
>>>>> http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2013-August/003845.html
>>>>>
>>>>> Cheers,
>>>>> John
>>>>>
>>>>> --
>>>>> John Wise
>>>>> Assistant Professor of Physics
>>>>> Center for Relativistic Astrophysics, Georgia Tech
>>>>> http://cosmo.gatech.edu
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>
>>>
>>>
>>> --
>>> Cameron Hummels
>>> Postdoctoral Researcher
>>> Steward Observatory
>>> University of Arizona
>>> http://chummels.org
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>