[yt-users] Problems using Rockstar in yt

Tue Aug 12 18:46:47 PDT 2014

Hey everyone,

Thanks for all of the tips for solving this.  To follow up,

--OS X needs the $DYLD_LIBRARY_PATH set to the same things that are set by
the activation script for $LD_LIBRARY_PATH in order to run Rockstar
internal to yt.  Setting $DYLD_LIBRARY_PATH to yt's lib dir caused some
problems for me initially when I tried to use things like the vi editor,
but by reinstalling yt without libpng support, this was resolved.

--Rockstar as it is run in yt *requires* parallel operations to work with
at least 3 processors.

--Running rockstar through yt can be done on datasets using particles with
different masses by first applying a filter to isolate the particles of one
mass.  Use the script that Sam suggested:
http://paste.yt-project.org/show/5029/ .  Thanks, Sam!

I'm going to tweak the docs to take into account these issues, but I wanted
to summarize for people reading this at home.

Thanks for the help, everyone.

Cameron

On Tue, Aug 12, 2014 at 8:37 AM, Nathan Goldbaum <nathan12343 at gmail.com>
wrote:

>
>
> On Tuesday, August 12, 2014, Britton Smith <brittonsmith at gmail.com> wrote:
>
>> Hi Cameron, David,
>>
>> Cameron, you are correct, on Mac systems you need to set
>> DYLD_LIBRARY_PATH instead of LD_LIBRARY_PATH.  The fact that this causes vi
>> and other things to fail is apparently an OS X bug (I found some
>> discussions of it on the internet) and has nothing to do with yt.  I went
>> through a similar thing and had to resort to having a separate terminal for
>> running rockstar that I didn't try to do anything else in.  For what it's
>> worth, this behavior seems to have been fixed.  I am on OS X 10.8.5 and no
>> longer have the issue of things crashing in a terminal where I have edited
>> DYLD_LIBRARY_PATH.
>>
>> On the second issue that you both reported, the problem is the way in
>> which one tells yt to run in parallel depends on how you import yt.
>>
>> If you do "from yt.mods import *", the parallelism is enabled by adding
>> the --parallel flag on the command line.  However, if you do "import yt",
>> then parallelism is enabled by doing "yt.enable_parallelism()" directly
>> below the import statement within your script.
>>
>> I looked at the latest documentation on parallelism (
>> http://yt-project.org/doc/analyzing/parallel_computation.html) and is
>> discusses the use of yt.enable_parallelism(), but not the old way that
>> works with "from yt.mods import *".  There should probably be a short
>> discussion added of how this has changed.  Additionally, let's make sure
>> that the yt-rockstar docs get any needed additions there as well.
>>
>
> This is mentioned on the yt3 differences page.  You're right that there
> should be some discussion in the parallelism docs, though
>
>
>>
>> Britton
>>
>>
>> On Tue, Aug 12, 2014 at 11:52 AM, David Sullivan <D.Sullivan at sussex.ac.uk
>> > wrote:
>>
>>> Hi all,
>>>
>>> I also get the same error as Cameron when trying to run rockstar with
>>> mpi:
>>>
>>> http://paste.yt-project.org/show/5019/
>>>
>>> I’m using a ramses dataset (not sure if this is supported, but couldn’t
>>> see any clear evidence that it isn’t) and the following script:
>>>
>>> mpirun -np 4 --mca btl ^openib python run_rockstar.py --parallel
>>> http://paste.yt-project.org/show/5020/
>>>
>>> I followed the same install instructions as Cameron, but I’m using a
>>> Linux cluster running CentOS. Appreciate any help!
>>>
>>> Cheers,
>>> David
>>>
>>> On 12 Aug 2014, at 10:04, yt-users-request at lists.spacepope.org wrote:
>>>
>>> Message: 3
>>> Date: Tue, 12 Aug 2014 02:04:10 -0700
>>> From: Sam Skillman <samskillman at gmail.com>
>>> To: Discussion of the yt analysis package
>>>  <yt-users at lists.spacepope.org>
>>> Subject: Re: [yt-users] Problems using Rockstar in yt
>>> Message-ID:
>>>  <CAB4=1s1oHK0uPFo88rkSaEa-W6EASMLoyOiEFeniV-xSm6SPaQ at mail.gmail.com>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>>
>>> Hi Cameron,
>>>
>>> Are you running in MPI?  You need to run with at least 3 MPI tasks for
>>> Rockstar to work properly.
>>>
>>> Sam
>>>
>>>
>>> On Mon, Aug 11, 2014 at 6:41 PM, Cameron Hummels <chummels at gmail.com>
>>> wrote:
>>>
>>>  Hello,
>>>
>>> I'm trying to use Rockstar with the new HaloCatalog functionality in yt
>>> 3.0, but I'm encountering some problems.  For reference, this is an OS X
>>> box with the "yt" branch.  I've tested it on two OS X systems to the same
>>> result.
>>>
>>> I modified my install_script.py to have it install Rockstar as described
>>> here:
>>>
>>>
>>>
>>> http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_finders.html#rockstar-installation
>>>
>>> Then, after rerunning my install script for the `yt` branch (ie yt-3.1
>>> dev) I tried to run the script provided here in order to generate a halo
>>> catalog using HOP:
>>>
>>>
>>>
>>> http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_transition.html#getting-halo-information
>>>
>>> It can create the catalog fine with finder_method='hop', but when I
>>> change
>>> it to finder_method='rockstar', I get failures.  This is the traceback:
>>>
>>> http://paste.yt-project.org/show/TE2DW8gc50urchv4JnUk
>>>
>>> In the traceback, it claims that it cannot access librockstar.so, but
>>> this
>>> library exists in the yt-x86_64/lib directory, which is explicitly in
>>> from
>>> my $LD_LIBRARY_PATH.  I can seemingly get it to work, if I add that path
>>> to
>>> my $DYLD_LIBRARY_PATH environment variable, but then a bunch of other
>>> things break (like vi).  Not sure what to do here.
>>>
>>> But assuming there is no vi problem here, even then, when I run the
>>> script
>>> above, I get failures.  Here is the traceback:
>>>
>>> http://paste.yt-project.org/show/rCOT1eVT1szR1T6DEUDb
>>>
>>> I'm not sure what to do here.  I've also tried running rockstar external
>>> to yt on an enzo dataset, but it doesn't appear that the version we
>>> install
>>> as part of yt allows it to work on HDF5 datasets.
>>>
>>> Has anyone run into any of these problems before?  I'm at a bit of a loss
>>> as to how to proceed on any of these fronts.
>>>
>>> Cameron
>>>
>>>
>>> --
>>> Cameron Hummels
>>> Postdoctoral Researcher
>>> Steward Observatory
>>> University of Arizona
>>> http://chummels.org
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>

-- 
Cameron Hummels
Postdoctoral Researcher
Steward Observatory
University of Arizona
http://chummels.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20140812/8441e184/attachment.htm>