[yt-users] Building yt on Hopper
Andrew Myers
atmyers2 at gmail.com
Sun Feb 2 11:12:21 PST 2014
I think I found the problem. The install script first builds the ZeroMQ
library, and then tries to build PyZMQ bindings for it. The problem was
that the script was not passing in the location of the newly-built ZeroMQ
when it ran setup.py for PyZMQ. Instead, PyZMQ tries to use a bundled zmq,
which doesn't work correctly. The fix was to manually re-build PyZMQ,
passing in the location of the new libraries. The install script picked up
from there normally.
Why didn't the script work? Looking at the function that calls setup for
each package:
function do_setup_py{ [ -e $1/done ] && return LIB=$1 shift
if [ -z "$@" ] then echo "Installing $LIB" else
echo "Installing $LIB (arguments: '$@')" fi [ ! -e
$LIB/extracted ] && tar xfz $LIB.tar.gz touch $LIB/extracted cd
$LIB if [ ! -z `echo $LIB | grep h5py` ] then shift (
${DEST_DIR}/bin/python2.7 setup.py build --hdf5=${HDF5_DIR} $* 2>&1 )
1>> ${LOG_FILE} || do_exit else shift (
${DEST_DIR}/bin/python2.7 setup.py build $* 2>&1 ) 1>> ${LOG_FILE}
|| do_exit fi ( ${DEST_DIR}/bin/python2.7 setup.py install
2>&1 ) 1>> ${LOG_FILE} || do_exit touch done cd ..}
When this function is called, the name of the script is at $0, the library
name is at $1, and the rest of the variables hold arguments that should be
passed to setup.py. After the first call to shift, the additional args
start at $1. The script then calls shift once more, whichever way it
branches, so that the first additional argument is no longer contained in
$*. So, this function loses the first arg. For PyZMQ, the first arg is the
location of the zmq library. If I remove the second and third shifts and
re-run the install script from the beginning, everything works for me.
-Andrew
On Sat, Feb 1, 2014 at 3:16 PM, Andrew Myers <atmyers2 at gmail.com> wrote:
> Hi Hilary,
>
> Thanks for the suggestion. Unfortunately, it didn't work for me - if I
> install ipython manually, it still fails with the same error message.
>
> I think I'll try to ask this at StackOverflow as well, as per Nathan's
> suggestion.
>
> Thanks,
> Andrew
>
>
> On Sat, Feb 1, 2014 at 10:09 AM, Hilary Egan <hilaryye at gmail.com> wrote:
>
>> I saw the same problem when I was installing on CU's janus machine. I'm
>> pretty sure that I got around it by building just ipython by hand and
>> making sure I was on the compile node when I tried it- no idea if there is
>> an equivalent for Hopper.
>>
>> -download and untar ipython.tar.gz (http://archive.ipython.org/release/)
>> -cd ipython
>> -python setup.py install
>> -touch done
>>
>> Caveat: I have no idea why this worked when the install script didn't,
>> and I'm just basing my answer on the note I jotted down a few months ago.
>> There's definitely some sort of discussion out there already about this
>> being a problem in some versions in pyzmq (
>> https://github.com/zeromq/pyzmq/pull/218). Hope that helps!
>>
>> -Hilary
>>
>>
>> On Sat, Feb 1, 2014 at 10:49 AM, Nathan Goldbaum <nathan12343 at gmail.com>wrote:
>>
>>> Hi Andrew,
>>>
>>> Unfortunately I'm not sure what the issue is.
>>>
>>> You might try asking this question on StackOverflow (tagging it with
>>> IPython and ZMQ) or the IPython-dev mailing list, some of the ZMQ/IPy
>>> devs will be more likely to see it.
>>>
>>> Nathan
>>>
>>>
>>> On Saturday, February 1, 2014, Andrew Myers <atmyers2 at gmail.com> wrote:
>>>
>>>> Hi Folks,
>>>>
>>>> I'm trying to install the development version of yt-2.7 on NERSC's
>>>> Hopper machine using the install script. I get to the part where it tries
>>>> to build IPython, and then the script fails with the following error
>>>> message:
>>>>
>>>>
>>>> ============================================================================
>>>> BUILDING IPYTHON
>>>> python: 2.7.6 (default, Jan 31 2014, 21:52:04) [GCC
>>>> 4.8.1
>>>> 20130531 (Cray Inc.)]
>>>> platform: linux2
>>>>
>>>> OPTIONAL DEPENDENCIES
>>>> sphinx: Not found (required for docs and nbconvert)
>>>> pygments: Not found (required for docs and nbconvert)
>>>> nose: 1.3.0
>>>> pexpect: no (required for running standalone doctests)
>>>> Traceback (most recent call last):
>>>> File "setup.py", line 342, in <module>
>>>> check_for_dependencies()
>>>> File "/global/u2/a/atmyers/yt-x86_64/src/ipython-1.1.0/setupbase.py",
>>>> line 376, in check_for_dependencies
>>>> check_for_pyzmq()
>>>> File
>>>> "/global/u2/a/atmyers/yt-x86_64/src/ipython-1.1.0/setupext/setupext.py",
>>>> line 118, in check_for_pyzmq
>>>> import zmq
>>>> File
>>>> "/global/homes/a/atmyers/yt-x86_64/lib/python2.7/site-packages/zmq/__init__.py",
>>>> line 35, in <module>
>>>> _libzmq = ctypes.CDLL(bundled[0], mode=ctypes.RTLD_GLOBAL)
>>>> File
>>>> "/global/homes/a/atmyers/yt-x86_64/lib/python2.7/ctypes/__init__.py", line
>>>> 365, in __init__
>>>> self._handle = _dlopen(self._name, mode)
>>>> OSError:
>>>> /global/homes/a/atmyers/yt-x86_64/lib/python2.7/site-packages/zmq/libzmq.so:
>>>> undefined symbol: clock_gettime
>>>>
>>>> Both ZeroMQ and pyzmq appear to have been installed without errors, but
>>>> then ZeroMQ can't find some of the definitions it needs. Googling around,
>>>> it looks like clock_gettime is supposed to be found in librt.
>>>>
>>>> Has anyone seen this before? Or, relatedly, has anyone been able to
>>>> build yt on the NERSC machines? I know that ZeroMQ isn't strictly
>>>> necessary, but I'd like to be able to use the notebooks.
>>>>
>>>> My complete yt_install.log is here<https://www.dropbox.com/s/pmdq3a9vfrbtwj9/yt_install.log>
>>>> .
>>>>
>>>> Thanks,
>>>> Andrew
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20140202/f3aedce3/attachment.html>
More information about the yt-users
mailing list