<div dir="ltr"><span style="font-family:arial,sans-serif;font-size:13px">Hi Rick,</span><br style="font-family:arial,sans-serif;font-size:13px"><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">You need to launch the notebook in parallel using an MPI parallel IPython cluster.  In addition, you will need to configure the notebook to hook into IPython's built-in parallelism.</span><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I've uploaded a notebook that uses parallel IPython here:</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px"><a href="http://nbviewer.ipython.org/gist/ngoldbaum/32ca07cf4f5b3dd06add" target="_blank">http://nbviewer.ipython.org/gist/ngoldbaum/32ca07cf4f5b3dd06add</a></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Note that all the yt operations aren't really important, but the first few cells where I check to make sure IPython's parallelism works and that MPI parllelism is working are quite important.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Note also that you will need to launch the IPython cluster separately from the notebook server using the "ipcluster" command.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">If you want to learn more about parallel IPython, I'd encourage you to take a look at Min Ranger-Kelley's tutorial from scipy 2014: <a href="http://pyvideo.org/video/2738/interactive-parallel-computing-with-ipython-part" target="_blank">http://pyvideo.org/video/2738/interactive-parallel-computing-with-ipython-part</a></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">All that said, it does add a significant amount of semantic overhead to use the IPython notebook in parallel.  It's generally much more straightforward to work with yt in parallel using regular python scripts.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Hope that helps,</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Nathan<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 14, 2015 at 6:49 PM, Rick Sarmento <span dir="ltr"><<a href="mailto:rsarment@asu.edu" target="_blank">rsarment@asu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Hello,<div><br></div><div>I’m pretty new to yt… I’m trying to speed things up significantly since I have rather large RAMSES datasets (running 3Mpc sims with 2^9^3 particles). I’m using STAMPEDE at TACC, <a href="https://vis.tacc.utexas.edu/#" target="_blank">https://vis.tacc.utexas.edu/#</a> , to startup a notebook on their visualization nodes (each has 16 cores). I’m trying to use</div><div><br></div><div><div><font face="Courier New"><span style="font-size:12px">import yt</span></font></div><div><font face="Courier New"><span style="font-size:12px">yt.enable_parallelism()</span></font></div><div><font face="Courier New"><span style="font-size:12px">ds = yt.load("./output_00011/info_00011.txt”)</span></font></div></div><div><br></div><div>and then</div><div><br></div><div><div><font face="Courier New"><span style="font-size:12px"># Looking through the "z" axis... Plot temp weighted by density</span></font></div><div><font face="Courier New"><span style="font-size:12px">p = yt.ProjectionPlot(ds, "z", "temperature", weight_field="density")</span></font></div><div><font face="Courier New"><span style="font-size:12px">p.annotate_title(z)</span></font></div><div><font face="Courier New"><span style="font-size:12px">p.show()</span></font></div></div><div><br></div><div>but I only see one python process via top … and NO speedup. What do I need to do to take advantage of the 16 cores?</div><div><br></div><div><div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div><i><font size="3">Cheers!</font></i></div><span class="HOEnZb"><font color="#888888"><div><i><font size="3"><br></font></i></div><div><i><font size="3">Rick Sarmento</font></i></div><div style="color:rgb(0,0,0);font-size:13px">SESE Astronomy/Astrophysics Grad Student</div><div style="color:rgb(0,0,0);font-size:13px"><a href="mailto:rsarment@asu.edu" target="_blank">rsarment@asu.edu</a></div></font></span></div></div></div></div></div>
</div>
<br></div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div></div>