<div dir="ltr"><div><div>I think I found the problem. The install script first builds the ZeroMQ library, and then tries to build PyZMQ bindings for it. The problem was that the script was not passing in the location of the newly-built ZeroMQ when it ran setup.py for PyZMQ. Instead, PyZMQ tries to use a bundled zmq, which doesn't work correctly. The fix was to manually re-build PyZMQ, passing in the location of the new libraries. The install script picked up from there normally.<br>
<br></div>Why didn't the script work? Looking at the function that calls setup for each package:<span class=""> <br></span><pre><span class="">function </span>do_setup_py
<a name="cl-491"></a><span class="">{</span>
<a name="cl-492"></a> <span class="">[</span> -e <span class="">$1</span>/done <span class="">]</span> <span class="">&&</span> <span class="">return</span>
<a name="cl-493"></a><span class=""> </span><span class="">LIB</span><span class="">=</span><span class="">$1</span>
<a name="cl-494"></a> <span class="">shift</span>
<a name="cl-495"></a><span class=""> </span><span class="">if</span> <span class="">[</span> -z <span class="">"$@"</span> <span class="">]</span>
<a name="cl-496"></a> <span class="">then</span>
<a name="cl-497"></a><span class=""> </span><span class="">echo</span> <span class="">"Installing $LIB"</span>
<a name="cl-498"></a> <span class="">else</span>
<a name="cl-499"></a><span class=""> </span><span class="">echo</span> <span class="">"Installing $LIB (arguments: '$@')"</span>
<a name="cl-500"></a> <span class="">fi</span>
<a name="cl-501"></a> <span class="">[</span> ! -e <span class="">$LIB</span>/extracted <span class="">]</span> <span class="">&&</span> tar xfz <span class="">$LIB</span>.tar.gz
<a name="cl-502"></a> touch <span class="">$LIB</span>/extracted
<a name="cl-503"></a> <span class="">cd</span> <span class="">$LIB</span>
<a name="cl-504"></a> <span class="">if</span> <span class="">[</span> ! -z <span class="">`</span><span class="">echo</span> <span class="">$LIB</span> <span class="">|</span> grep h5py<span class="">`</span> <span class="">]</span>
<a name="cl-505"></a> <span class="">then</span>
<a name="cl-506"></a><span class=""> </span><span class="">shift</span>
<a name="cl-507"></a> <span class="">(</span> <span class="">${</span><span class="">DEST_DIR</span><span class="">}</span>/bin/python2.7 setup.py build --hdf5<span class="">=</span><span class="">${</span><span class="">HDF5_DIR</span><span class="">}</span> <span class="">$*</span> 2><span class="">&</span>1 <span class="">)</span> 1>> <span class="">${</span><span class="">LOG_FILE</span><span class="">}</span> <span class="">||</span> do_exit
<a name="cl-508"></a> <span class="">else</span>
<a name="cl-509"></a><span class=""> </span><span class="">shift</span>
<a name="cl-510"></a> <span class="">(</span> <span class="">${</span><span class="">DEST_DIR</span><span class="">}</span>/bin/python2.7 setup.py build <span class="">$*</span> 2><span class="">&</span>1 <span class="">)</span> 1>> <span class="">${</span><span class="">LOG_FILE</span><span class="">}</span> <span class="">||</span> do_exit
<a name="cl-511"></a> <span class="">fi</span>
<a name="cl-512"></a> <span class="">(</span> <span class="">${</span><span class="">DEST_DIR</span><span class="">}</span>/bin/python2.7 setup.py install 2><span class="">&</span>1 <span class="">)</span> 1>> <span class="">${</span><span class="">LOG_FILE</span><span class="">}</span> <span class="">||</span> do_exit
<a name="cl-513"></a> touch <span class="">done</span>
<a name="cl-514"></a><span class=""> </span><span class="">cd</span> ..
<a name="cl-515"></a><span class="">}</span></pre><span class="">When this function is called, the name of the script is at $0, the library name is at $1, and the rest of the variables hold arguments that should be passed to setup.py. After the first call to shift, the additional args start at $1. The script then calls shift once more, whichever way it branches, so that the first additional argument is no longer contained in $*. So, this function loses the first arg. For PyZMQ, the first arg is the location of the zmq library. If I remove the second and third shifts and re-run the install script from the beginning, everything works for me. <br>
<br></span></div><span class="">-Andrew<br></span><div><div><div><br> <br></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Feb 1, 2014 at 3:16 PM, Andrew Myers <span dir="ltr"><<a href="mailto:atmyers2@gmail.com" target="_blank">atmyers2@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Hi Hilary,<br><br>Thanks for the suggestion. Unfortunately, it didn't work for me - if I install ipython manually, it still fails with the same error message. <br>
<br></div>I think I'll try to ask this at StackOverflow as well, as per Nathan's suggestion.<br>
<br></div>Thanks,<br>Andrew<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Feb 1, 2014 at 10:09 AM, Hilary Egan <span dir="ltr"><<a href="mailto:hilaryye@gmail.com" target="_blank">hilaryye@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I saw the same problem when I was installing on CU's janus machine. I'm pretty sure that I got around it by building just ipython by hand and making sure I was on the compile node when I tried it- no idea if there is an equivalent for Hopper.<div>
<br><div>-download and untar<span style="font-size:13px;font-family:Arial"> ipython.tar.gz (</span><font face="Arial"><a href="http://archive.ipython.org/release/" target="_blank">http://archive.ipython.org/release/</a>)</font></div>
<div>
<span style="font-size:13px;font-family:Arial">-cd ipython</span><br>
<span style="font-size:13px;font-family:Arial">-python setup.py install</span><br>
<span style="font-size:13px;font-family:Arial">-touch done</span></div></div><div><span style="font-size:13px;font-family:Arial"><br></span></div><div><span style="font-size:13px;font-family:Arial">Caveat: I have no idea why this worked when the install script didn't, and I'm just basing my answer on the note I jotted down a few months ago. There's definitely some sort of discussion out there already about this being a problem in some versions in pyzmq (</span><font face="Arial"><a href="https://github.com/zeromq/pyzmq/pull/218" target="_blank">https://github.com/zeromq/pyzmq/pull/218</a>). </font><span style="font-family:Arial;font-size:13px"> Hope that helps!</span></div>
<div><span style="font-family:Arial;font-size:13px"><br></span></div><div><span style="font-family:Arial;font-size:13px">-Hilary</span></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>
On Sat, Feb 1, 2014 at 10:49 AM, Nathan Goldbaum <span dir="ltr"><<a href="mailto:nathan12343@gmail.com" target="_blank">nathan12343@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>Hi Andrew,<div><br></div><div>Unfortunately I'm not sure what the issue is.</div>
<div><br></div><div>You might try asking this question on StackOverflow (tagging it with IPython and ZMQ) <span></span>or the IPython-dev mailing list, some of the ZMQ/IPy devs will be more likely to see it.</div>
<span><font color="#888888">
<div><br></div></font></span><div><span><font color="#888888">Nathan</font></span><div><div><br><div><br>On Saturday, February 1, 2014, Andrew Myers <<a href="mailto:atmyers2@gmail.com" target="_blank">atmyers2@gmail.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><span style="font-family:arial,sans-serif;font-size:13px">Hi Folks,</span><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I'm trying to install the development version of yt-2.7 on NERSC's Hopper machine using the install script. I get to the part where it tries to build IPython, and then the script fails with the following error message:</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><blockquote style="font-family:arial,sans-serif;font-size:13px;margin:0px 0px 0px 40px;border:none;padding:0px"><div>============================================================================</div>
<div>BUILDING IPYTHON</div><div> python: 2.7.6 (default, Jan 31 2014, 21:52:04) [GCC 4.8.1</div><div> 20130531 (Cray Inc.)]</div><div> platform: linux2</div><div><br></div>
<div>OPTIONAL DEPENDENCIES</div><div> sphinx: Not found (required for docs and nbconvert)</div><div> pygments: Not found (required for docs and nbconvert)</div><div> nose: 1.3.0</div>
<div> pexpect: no (required for running standalone doctests)</div><div>Traceback (most recent call last):</div><div> File "setup.py", line 342, in <module></div><div> check_for_dependencies()</div>
<div> File "/global/u2/a/atmyers/yt-x86_64/src/ipython-1.1.0/setupbase.py", line 376, in check_for_dependencies</div><div> check_for_pyzmq()</div><div> File "/global/u2/a/atmyers/yt-x86_64/src/ipython-1.1.0/setupext/setupext.py", line 118, in check_for_pyzmq</div>
<div> import zmq</div><div> File "/global/homes/a/atmyers/yt-x86_64/lib/python2.7/site-packages/zmq/__init__.py", line 35, in <module></div><div> _libzmq = ctypes.CDLL(bundled[0], mode=ctypes.RTLD_GLOBAL)</div>
<div> File "/global/homes/a/atmyers/yt-x86_64/lib/python2.7/ctypes/__init__.py", line 365, in __init__</div><div> self._handle = _dlopen(self._name, mode)</div><div>OSError: /global/homes/a/atmyers/yt-x86_64/lib/python2.7/site-packages/zmq/libzmq.so: undefined symbol: clock_gettime</div>
<div><br></div></blockquote><span style="font-family:arial,sans-serif;font-size:13px">Both ZeroMQ and pyzmq appear to have been installed without errors, but then ZeroMQ can't find some of the definitions it needs. Googling around, it looks like clock_gettime is supposed to be found in librt. </span><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">Has anyone seen this before? Or, relatedly, has anyone been able to build yt on the NERSC machines? I know that ZeroMQ isn't strictly necessary, but I'd like to be able to use the notebooks. </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">My complete yt_install.log is <a href="https://www.dropbox.com/s/pmdq3a9vfrbtwj9/yt_install.log" target="_blank">here</a>.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Thanks,</div><div style="font-family:arial,sans-serif;font-size:13px">Andrew </div></div>
</blockquote></div></div></div></div>
<br></div></div>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>