<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Oct 29, 2013 at 12:30 PM, Hilary Egan <span dir="ltr"><<a href="mailto:hilaryye@gmail.com" target="_blank">hilaryye@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hi all,<br><br>I'm quite confused on a number of points related to running the rockstar halo finder, so I hope its alright that I put all these questions into this one email!<br>
<br>1. I can't seem to run the rockstar halo finder at all without getting this error followed by a segmentation fault and crash.<br>
<br>[Warning] Network IO Failure (PID XXXXXX): Connection reset by peer<br>[Network] Packet receive retry count at: 1<br></div></blockquote><div><br></div><div>I know this is an old thread so this all might be futile.</div>
<div><br></div><div>I've had this before and it can be a number if network related issues. Primarily <span style="font-family:arial,sans-serif;font-size:13px">it may be that the default ports that the clients are using to communicate with the server are blocked, taken, or invalid for some reason. If everything is running on the same machine, you may be able to try using</span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">PARALLEL_IO_SERVER_INTERFACE = lo</div><div style="font-family:arial,sans-serif;font-size:13px"><br>
</div><div style="font-family:arial,sans-serif;font-size:13px">This will force everything to use the local loopback address (127.0.0.1). Often just waiting a few minutes for other instances to die often solves the problem. You could add 'killall rockstar' to your submission script in case there are zombie rockstar processes still running causing server issues.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I believe kraken uses SLURM in which case the following submission script might be helpful (using srun). e.g. start 128 instances and change FORK_PROCESSORS_PER_MACHINE to 1 in your cfg file.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br>You'll have to check the hdf5 module and change a few other things but here is a template. </div><div style="font-family:arial,sans-serif;font-size:13px"><br>
</div><div><div><font face="arial, sans-serif">#!/bin/bash</font></div><div><font face="arial, sans-serif">#SBATCH -n 128</font></div><div><font face="arial, sans-serif">#SBATCH -o job.o%j</font></div><div><font face="arial, sans-serif">#SBATCH -e job.e%j</font></div>
<div><font face="arial, sans-serif">#SBATCH -t 5000</font></div><div><font face="arial, sans-serif">#SBATCH -p queue_name</font></div><div><font face="arial, sans-serif">#SBATCH --mem=32gb</font></div><div><font face="arial, sans-serif">#SBATCH -J rockstarjob --exclusive</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">module load -S centos6/hdf5-1.8.11_gcc-4.8.0</font></div><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">rsdir=/path/to/rockstar/code/</font></div>
<div><font face="arial, sans-serif">exe=/path/to/rockstar/executable</font></div><div><font face="arial, sans-serif">cd $rsdir</font></div><div><font face="arial, sans-serif">outdir=/path/to/output/directory/</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">$exe -c $rsdir/cfgs/config.cfg &</font></div><div><font face="arial, sans-serif">#uncomment below and comment above for restarts.</font></div>
<div><font face="arial, sans-serif">#$exe -c $outdir/restart.cfg &</font></div><div><font face="arial, sans-serif">cd $outdir</font></div><div><font face="arial, sans-serif">perl -e 'sleep 1 while (!(-e "auto-rockstar.cfg"))'</font></div>
<div>
<p class="">srun -n 128 $exe -c auto-rockstar.cfg</p></div></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">You might have already moved on by now. Hope this helps if not.<br>
</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Brendan</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><br>It sort of seems like this issue (<a href="http://lists.spacepope.org/htdig.cgi/yt-dev-spacepope.org/2012-November/002681.html" target="_blank">http://lists.spacepope.org/htdig.cgi/yt-dev-spacepope.org/2012-November/002681.html</a>) but I couldn't really figure out what the resolution was from the thread. Im attempting to run this on kraken and it doesn't matter if I use a single compute node or multiple, I get the same error. (I hope this isn't the infiniband issue the docs warned about, I couldn't figure out if that is how kraken is connected and I got an error that the suggested flag doesn't exist so I didn't press the issue.)<br>
<br>2. Whenever I finally do get the halo finder to work, I need the results to be in a form that the merger tree can use. It seems as though the MergerTree needs the results in the same form as the other halo finders give, so would getting the halo list and then dumping it as usual be the appropriate strategy? Ie:<div>
<br></div><div> rh.run()</div><div><div> halo_list = rh.halo_list()</div><div><span style="white-space:pre-wrap"> </span>halo_list.dump('MergerHalos')</div><br>2.5. The docs sort of give mixed messages on whether or not I could just be calling MergerTree with the argument halo_finder_function = RockstarHaloFinder. At this point I've pretty thoroughly convinced myself that I can't, but it would be nice if that was clarified. (Just a thoroughly overwhelmed new user's perspective!)</div>
<div><br></div><div>3. I'm a little confused as to whether or not I have to use a TimeSeriesData object rather than the usual single time output when instantiating the halo finder. Under "Rockstar Halo Finding" it uses TimeSeriesData, unlike the rest of the examples, but under the subheading "Output Analysis" it just uses pf. The "Output Analysis" example also doesn't call the run() method, which leads me to believe something else entirely is going on, but its not quite clear. <br>
<div><br></div><div>Thanks!</div><span class=""><font color="#888888"><div>-Hilary</div></font></span></div></div>
<br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div></div>