[yt-users] Problems using Rockstar in yt

Tue Aug 12 07:21:06 PDT 2014

On Tue, Aug 12, 2014 at 9:15 AM, Cameron Hummels <chummels at gmail.com> wrote:
> Hi Britton, Sam, and David,
>
> Thanks for the insights into this problem.  I've made it farther, but I'm
> still getting problems.  I'll make sure that when we've resolved this, I'll
> write it up and put it in the docs for other people trying to use Rockstar
> with yt, as I think this may be a common occurrance since rockstar is now
> the preferred halo finder with yt above hop and fof.  A few comments:
>
>> Cameron, you are correct, on Mac systems you need to set DYLD_LIBRARY_PATH
>> instead of LD_LIBRARY_PATH.  The fact that this causes vi and other things
>> to fail is apparently an OS X bug (I found some discussions of it on the
>> internet) and has nothing to do with yt.  I went through a similar thing and
>> had to resort to having a separate terminal for running rockstar that I
>> didn't try to do anything else in.  For what it's worth, this behavior seems
>> to have been fixed.  I am on OS X 10.8.5 and no longer have the issue of
>> things crashing in a terminal where I have edited DYLD_LIBRARY_PATH.
>
>
> That's odd, since I'm also using OS X 10.8.5 on both machines where I'm
> getting this vi failure.  And strangely, it appears to be somewhat related
> to yt, according to the failure message when I try to run 'vi' from the
> command-line seen here:
>
> http://paste.yt-project.org/show/5022/

Looks like a conflict between system libpng and the yt stack's libpng.
Might be fixable with an install where libpng is disabled.

>
> Anyway, as you suggest, I can just run rockstar in a new terminal without
> doing anything else, but I thought you might have seen this error before.
>
>>
>> On the second issue that you both reported, the problem is the way in
>> which one tells yt to run in parallel depends on how you import yt.
>>
>> If you do "from yt.mods import *", the parallelism is enabled by adding
>> the --parallel flag on the command line.  However, if you do "import yt",
>> then parallelism is enabled by doing "yt.enable_parallelism()" directly
>> below the import statement within your script.
>>
>> I looked at the latest documentation on parallelism
>> (http://yt-project.org/doc/analyzing/parallel_computation.html) and is
>> discusses the use of yt.enable_parallelism(), but not the old way that works
>> with "from yt.mods import *".  There should probably be a short discussion
>> added of how this has changed.  Additionally, let's make sure that the
>> yt-rockstar docs get any needed additions there as well.
>
>
> OK, so one needs to run rockstar in parallel.  I thought in reading the
> rockstar docs, there was a way of running it with one processor, but OK, I
> can run it in parallel.  When I do that on this particular dataset, Enzo_64,
> it fails because there are particles in the simulation of two separate
> masses (stars and DM).  I know that is a problem with this version of
> rockstar as described in the docs here:
>
> http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_finders.html#rockstar-halo-finding
>
> It also mentions in those docs that in order to deal with simulations with
> multiple particle masses, "In simulations where the highest-resolution
> particles all have the same mass (ie: zoom-in grid based simulations), one
> can set up a particle filter to select the lowest mass particles and perform
> the halo finding only on those."
> I have tried a few things here to make this work.  One, as suggested I
> created a particle filter to just look at one mass of particles, and then I
> applied it to the dataset, but I'm not sure how to pass that particle filter
> to the Halo Catalog object.  Here is my attempt:
>
> http://paste.yt-project.org/show/5023/
>
> And the corresponding failure traceback:
>
> http://paste.yt-project.org/show/4IWCE1hZS1KGpTWztokW
>
> Alternatively, I see that the Halo Catalog object does take a "data_source"
> kwarg, so I made a cut_region of all_data just selecting particles that were
> the most massive, and again, I get the same failure.  So I'm curious how to
> use Rockstar to find halos on simulations that have multiple masses.  I
> guess I'm just missing something stupid here, as I know there are people who
> have done this in the past.  Does anyone have an example script or some hint
> for me here?  Here is my script using the cut region to get only the most
> massive particles of a single mass, but it fails the same way as above:
>
> http://paste.yt-project.org/show/5027/
>
> Alternatively, it seems like the yt docs advocate using rockstar external to
> yt, and then just using the output halos file for doing things (e.g.
> http://yt-project.org/docs/dev/cookbook/cosmological_analysis.html#plotting-halos),
> but as I said, I am running into problems using the version of rockstar that
> comes with yt on enzo (or any hdf5) datasets.  Does anyone know how to use
> this standalone rockstar with an enzo dataset to produce such a halo file
> for later use within yt?

Hm, news to me about the docs suggesting that.  My understanding is
that we're mostly going in the opposite way -- avoiding using
Rockstar's parallelism completely (to avoid the issues you note) and
instead feeding in FOF groups found by yt.  This didn't get finished
in time for the 3.0 release, and I wouldn't recommend trying it right
now.

For multiple masses, I think your first inclination (using particle
filters) is the best.  Seems like it should just be fixable by
allowing the Halo Catalog to accept a particle filter.

>
> Sorry for the long email, but I've tried lots of things.  Thank you very
> much for the help so far!
>
> Cameron
>
>
>>
>> Britton
>>
>>
>> On Tue, Aug 12, 2014 at 11:52 AM, David Sullivan <D.Sullivan at sussex.ac.uk>
>> wrote:
>>>
>>> Hi all,
>>>
>>> I also get the same error as Cameron when trying to run rockstar with
>>> mpi:
>>>
>>> http://paste.yt-project.org/show/5019/
>>>
>>> I’m using a ramses dataset (not sure if this is supported, but couldn’t
>>> see any clear evidence that it isn’t) and the following script:
>>>
>>> mpirun -np 4 --mca btl ^openib python run_rockstar.py --parallel
>>> http://paste.yt-project.org/show/5020/
>>>
>>> I followed the same install instructions as Cameron, but I’m using a
>>> Linux cluster running CentOS. Appreciate any help!
>>>
>>> Cheers,
>>> David
>>>
>>> On 12 Aug 2014, at 10:04, yt-users-request at lists.spacepope.org wrote:
>>>
>>> Message: 3
>>> Date: Tue, 12 Aug 2014 02:04:10 -0700
>>> From: Sam Skillman <samskillman at gmail.com>
>>> To: Discussion of the yt analysis package
>>> <yt-users at lists.spacepope.org>
>>> Subject: Re: [yt-users] Problems using Rockstar in yt
>>> Message-ID:
>>> <CAB4=1s1oHK0uPFo88rkSaEa-W6EASMLoyOiEFeniV-xSm6SPaQ at mail.gmail.com>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>>
>>> Hi Cameron,
>>>
>>> Are you running in MPI?  You need to run with at least 3 MPI tasks for
>>> Rockstar to work properly.
>>>
>>> Sam
>>>
>>>
>>> On Mon, Aug 11, 2014 at 6:41 PM, Cameron Hummels <chummels at gmail.com>
>>> wrote:
>>>
>>> Hello,
>>>
>>> I'm trying to use Rockstar with the new HaloCatalog functionality in yt
>>> 3.0, but I'm encountering some problems.  For reference, this is an OS X
>>> box with the "yt" branch.  I've tested it on two OS X systems to the same
>>> result.
>>>
>>> I modified my install_script.py to have it install Rockstar as described
>>> here:
>>>
>>>
>>>
>>> http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_finders.html#rockstar-installation
>>>
>>> Then, after rerunning my install script for the `yt` branch (ie yt-3.1
>>> dev) I tried to run the script provided here in order to generate a halo
>>> catalog using HOP:
>>>
>>>
>>>
>>> http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_transition.html#getting-halo-information
>>>
>>> It can create the catalog fine with finder_method='hop', but when I
>>> change
>>> it to finder_method='rockstar', I get failures.  This is the traceback:
>>>
>>> http://paste.yt-project.org/show/TE2DW8gc50urchv4JnUk
>>>
>>> In the traceback, it claims that it cannot access librockstar.so, but
>>> this
>>> library exists in the yt-x86_64/lib directory, which is explicitly in
>>> from
>>> my $LD_LIBRARY_PATH.  I can seemingly get it to work, if I add that path
>>> to
>>> my $DYLD_LIBRARY_PATH environment variable, but then a bunch of other
>>> things break (like vi).  Not sure what to do here.
>>>
>>> But assuming there is no vi problem here, even then, when I run the
>>> script
>>> above, I get failures.  Here is the traceback:
>>>
>>> http://paste.yt-project.org/show/rCOT1eVT1szR1T6DEUDb
>>>
>>> I'm not sure what to do here.  I've also tried running rockstar external
>>> to yt on an enzo dataset, but it doesn't appear that the version we
>>> install
>>> as part of yt allows it to work on HDF5 datasets.
>>>
>>> Has anyone run into any of these problems before?  I'm at a bit of a loss
>>> as to how to proceed on any of these fronts.
>>>
>>> Cameron
>>>
>>>
>>> --
>>> Cameron Hummels
>>> Postdoctoral Researcher
>>> Steward Observatory
>>> University of Arizona
>>> http://chummels.org
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
>
> --
> Cameron Hummels
> Postdoctoral Researcher
> Steward Observatory
> University of Arizona
> http://chummels.org
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>