[yt-users] YT: Rockstar Halo Finder

[Devin Silvia]

To follow up on this, Jacob has been working with me on this dataset and
since Blue Waters doesn't have any nodes with huge quantities of memory
we've been trying just about everything we can think of, including running
on a huge number of nodes or trying a very low processor count per node.
Unfortunately, none of the configurations we've tried has worked.

Thanks for the information Hilary.  Unfortunately, we don't have access to
a shared memory machine at the moment :(  If anyone else has thoughts for
how to make this go, we'd love to hear them.

Thanks!
Devin

On Thu, Jun 11, 2015 at 4:14 PM, Hilary Egan <hilaryye at gmail.com> wrote:

> Hi Jacob,
>
> While I'm not confident in exact numbers in terms of nodes/ppn settings
> for your run, I think I can point you in the right direction.
>
> * When I used rockstar to find halos for a 1024^3 unigrid enzo I ran into
> similar OOM errors. I eventually got it to work by running on a shared
> memory machine (PSC Blacklight to be exact). I'm not sure you're ever going
> to be able to run it without a machine that can share memory between nodes,
> because of the way rockstar handles keeping the particles in memory
> separately from yt.
>
> * You also don't need anywhere near that number of processes. If I can
> recall, I think I only used ~16 processes in the end, once I could request
> enough memory.
>
> Hope that helps a bit! Sorry the current implementation of rockstar in yt
> is a bit of a pain to work with. It's something we've been working on
> fixing, so we can keep you posted on when the new infrastructure is ready
> for users.
>
> -Hilary
>
> On Thu, Jun 11, 2015 at 1:22 PM, Jacob Kneibel <kneibelj at msu.edu> wrote:
>
>> Hello,
>>
>> I've been trying to run the Rockstar Halo finder on a 1536^3 unigrid Enzo
>> simulation and have been running into a few issues. I am using the Blue
>> waters machine and when I run the finder with multiple readers, the process
>> is killed by the OOM killer, and when I run it with one reader there is a
>> Network IO error.
>>
>> My yt version is 3.2-dev, changeset= 8ad62b624390
>>
>> Here is the python script I am running:
>> http://paste.yt-project.org/show/5631/
>>
>> and here is the job script I'm submitting to blue waters:
>> http://paste.yt-project.org/show/5632/
>>
>> I've tried different node and ppn settings, with no luck.
>>
>> Does anyone have an idea on how to get rockstar to run on the blue waters
>> system with this large simulation? Or is there a recommended number of
>> readers based on the number of nodes I set?
>>
>> Thank you for any help you can give,
>>
>> Jacob Kneibel
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>


-- 
Devin W. Silvia
NSF Astronomy and Astrophysics Postdoctoral Fellow
Department of Physics and Astronomy
Michigan State University
www.devinsilvia.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20150611/0c211d96/attachment.html>