<meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div dir="ltr">Hi Stuart,<div><br>I ran into exactly this problem, and ultimately managed to get yt-dev installed (and working in parallel) using Anaconda. Instructions are attached; hopefully they'll work for you.</div><div><br></div><div>--Brian</div><div><br></div><div><div style="font-size:12.8px">-----------------------------------------------------------------------</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Getting <span class="">yt</span>-3 to work on <span class="">Blue</span> <span class="">Waters</span>:</div><div style="font-size:12.8px"><div><br></div><div>0) Get the GNU software environment set up: “module swap PrgEnv-cray PrgEnv-gnu” . This will be necessary later.</div><div><br></div><div>1) Install the <span class="">Anaconda</span> python distribution (<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__store.continuum.io_cshop_anaconda_&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=EUYYHwwJrc6IxhTGtc7VhkFPMG1SILmwLcp_ulKGRE0&e=" target="_blank">https://store.continuum.io/cshop/<span class="">anaconda</span>/</a>) for 64-bit Linux systems. Refresh your path so the <span class="">Anaconda</span> python’s in your path.</div><div><br></div><div>2) Download mpi4py from <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__bitbucket.org_mpi4py_mpi4py_&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=Qyo3kUXsc5soNjSuKNWW6RN0UK22ZsdtycUVpdZe6Ik&e=" target="_blank">https://bitbucket.org/mpi4py/mpi4py/</a>, or grab it from my home directory on <span class="">Blue</span> <span class="">Waters</span> (~bwoshea/mpi4py-1.3.1.tar.gz).</div><div><br></div><div>3) Once you’ve unpacked it in your home directory, edit the file mpi.cfg, which is in the top-level mpi4py directory, and add the following chunk of text at the end of the file:</div><div><br></div><div>[cray]</div><div>mpicc = cc</div><div>mpicxx = CC</div><div>extra_link_args = -shared</div><div><br></div><div>then, in the top-level mpi4py directory, type:</div><div><br></div><div>python setup.py build --mpi=cray</div><div>python setp.py install</div><div><br></div><div>At this point, you should have a functional version of mpi4py that uses the cray MPI libraries. </div><div><br></div><div>4) You can then install <span class="">yt</span> via source in your home directory:</div><div><br></div><div>hg clone <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__bitbucket.org_yt-5Fanalysis_yt&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=8dL-5hwP3gUAGmsEP_QRR32KF9UMlij8XID1DJ4dNIk&e=" target="_blank">https://bitbucket.org/yt_analysis/<span class="">yt</span></a></div><div>hg update <span class="">yt</span></div><div>python setup.py develop</div><div><br></div><div>And it should work, though you’re going to need to modify your LD_LIBRARY_PATH to point to your <span class="">anaconda</span> /lib directory. You can verify that <span class="">yt</span> and mpi4py are playing nicely together by running the scripts hw_MPI.py and yt_parallel_test.py, both of which are sitting in my home directory. (Note that you can do this via batch script or through the interactive queue, both with aprun - you can’t just do it from the login node.)</div><div><br></div><div>5) and, optionally, if you want to install Rockstar using <span class="">yt</span> from source, you do the following:</div><div><br></div><div>hg clone <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__bitbucket.org_MatthewTurk_rockstar&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=m1_DdtH_4ZVkn0cLBnf5qVMwDOjGm-lpugq8dKJTXh0&e=" target="_blank">http://bitbucket.org/MatthewTurk/rockstar</a></div><div>cd rockstar</div><div>hg pull</div><div>hg up -C tip</div><div>make lib</div><div><br></div><div>and *then* make a file called rockstar.cfg that has a single line of text: the path of the rockstar source code directory. At that point, you need to run “python setup.py develop” again, and <span class="">yt</span> will recognize that rockstar has been installed and play nicely with it. You will likely also have to modify your LD_LIBRARY_PATH to point at your rockstar directory so it can find the rockstar library (librockstar.so).</div><div><br></div><div><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 8, 2016 at 9:30 PM, Levy, Stuart A <span dir="ltr"><<a href="mailto:salevy@illinois.edu" target="_blank">salevy@illinois.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">Hello yt people,<br>
<br>
I'm trying to install current yt-3.3dev on Blue Waters at NCSA, using the install_script.sh route.<br>
<br>
After hacking around the fact that it has an obsolete & unsafe openssl library as the default (I needed to pass LDFLAGS/CPPFLAGS to the python build to set the -L and -rpath to the non-default usable openssl), I'm having lots of trouble getting other component
installs to work. The yt_install.log has things like<br>
<br>
<blockquote><font face="Arial" size="2">Processing dependencies for matplotlib==1.4.3<br>
Searching for pytz<br>
Reading <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.python.org_simple_pytz_&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=7Y1Vxqt7OnKo7uGmnmvcCmaAZewIUBjvB7wg5B4TASQ&e=" target="_blank">https://pypi.python.org/simple/pytz/</a><br>
Download error on <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.python.org_simple_pytz_&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=7Y1Vxqt7OnKo7uGmnmvcCmaAZewIUBjvB7wg5B4TASQ&e=" target="_blank">https://pypi.python.org/simple/pytz/</a>: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) -- Some packages may not be found!<br>
Couldn't find index page for 'pytz' (maybe misspelled?)<br>
Scanning index of all packages (this may take a while)<br>
Reading <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.python.org_simple_&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=ydoA_ZlRi6TSl-EnPvwqKosavjAz5sZk4ToMcYha5O0&e=" target="_blank">https://pypi.python.org/simple/</a><br>
Download error on <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.python.org_simple_&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=ydoA_ZlRi6TSl-EnPvwqKosavjAz5sZk4ToMcYha5O0&e=" target="_blank">https://pypi.python.org/simple/</a>: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) -- Some packages may not be found!<br>
No local packages or download links found for pytz<br>
error: Could not find suitable distribution for Requirement.parse('pytz')<br>
</font></blockquote>
<br>
<font size="2"><font face="Arial">Yet when I run what seems like the same install by hand ($DIR/bin/pip -v install pytz) it works fine, without error.<br>
<br>
I can then re-run install_script.sh, until it reaches the next dependency and fails, etc. Each cycle takes 5-10 minutes so it's pretty tedious.<br>
<br>
***=> I don't understand why the internal-to-matplotlib-setup call to pip fails, while it succeeds when I run it by hand.<br>
<br>
<br>
A suspicion: could the CERTIFICATE_VERIFY_FAILED errors be hinting that Blue Waters has an https gateway to the outside world that rewrites SSL certificates? I don't know whether that's true. This page suggested a way to grab the needed root certificate
and point env var PIP_CERT at it...<br>
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__stackoverflow.com_questions_25981703_pip-2Dinstall-2Dfails-2Dwith-2Dconnection-2Derror-2Dssl-2Dcertificate-2Dverify-2Dfailed-2Dcerti&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=QrNB0odxD8-IB_Vzbh0dTb8-dzhb0qAM0IUc1om7H2Y&e=" target="_blank">
http://stackoverflow.com/questions/25981703/pip-install-fails-with-connection-error-ssl-certificate-verify-failed-certi</a><br>
but trying that didn't help in this case. The yt-installed "pip" works fine when run by hand, but whatever matplotlib's setup.py is running still fails.<br>
<br>
<br>
<br>
<br>
</font></font></div>
</div>
<br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spacepope.org_listinfo.cgi_yt-2Dusers-2Dspacepope.org&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=EzMGokFGT5ytCWcVoPuJR4IQh-cNa28KmlOUAam1kqY&s=ZNbDpuK-aqPAuCCheUfVqJ2ThZe-J9_-buZb6kxj39A&e=" rel="noreferrer" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>