[yt-users] Installing yt on Xsede Smic and Comet

Fri Jun 17 10:10:10 PDT 2016

On Fri, Jun 17, 2016 at 11:51 AM, Yuan Li <bear0980 at gmail.com> wrote:

> Hi Nathan,
>
> Aha! "yt" executable did not exist! The installation went smoothly so I
> did not even think about checking the log, but there were actually some
> error messages related to Cython. I realized that a system python module
> was automatically loaded upon login, so I unloaded it and reinstalled yt.
> It is working now. Thank you!
>

Ah, that's valuable feedback. Perhaps the install script should do some
rudimentary checking to verify that yt is installed correctly.

>
> Best,
> Yuan
>
> On Thu, Jun 16, 2016 at 4:49 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
>
>> Does yt's bin directory show up in your PATH environment variable? If so,
>> does a "yt" executable exist in that directory? If not, how are you setting
>> your PATH? Does it work if you prepend yt's bin directory to your PATH
>> manually in your current ssh session?
>>
>> On Thu, Jun 16, 2016 at 3:43 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>
>>> Yes I did that but it did not seem to do anything.
>>>
>>> Maybe I should email Smic people for help?
>>>
>>> Yuan
>>>
>>> On Thu, Jun 16, 2016 at 4:39 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Jun 16, 2016 at 3:34 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>>>
>>>>> Hi Nathan,
>>>>>
>>>>> On Comet:
>>>>> I set PYTHONPATH in bash_profile, but maybe that was not enough. Now
>>>>> in my script, I do "mpirun -np 96 /home/yuan/source/yt-conda/bin/python
>>>>> **.py", and it seems to be working. Yay!
>>>>>
>>>>>
>>>>> On Smic:
>>>>> I reinstalled yt with INST_YT_SOURCE=1. I set PATH & YT_DEST in my
>>>>> bash_profile. However, the system does not recognize yt.
>>>>>  -bash: yt: command not found
>>>>> even after I do "source ~/source/yt-conda/bin/activate". I thought I
>>>>> set up everything just the same as Comet...
>>>>>
>>>>
>>>> With the new install script, the activate script is unnecessary. Simply
>>>> prepend the location of the yt installation's bin directory to your PATH.
>>>> The install script should print out explicit code that you can copy/paste
>>>> into your .bashrc when the script finishes.
>>>>
>>>>
>>>>>
>>>>>
>>>>> Yuan
>>>>>
>>>>> On Wed, Jun 15, 2016 at 4:50 PM, Nathan Goldbaum <
>>>>> nathan12343 at gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 15, 2016 at 3:41 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Nathan,
>>>>>>>
>>>>>>> Thank you for the quick response!
>>>>>>>
>>>>>>> On Smic: Where is this "clean.sh" file? In the old yt3, it is under
>>>>>>> yt-x86_64/src/yt-hg, but I cannot find it in the new "/yt-conda/".
>>>>>>>
>>>>>>
>>>>>> Ah, so it looks like you used INST_YT_SOURCE=0 here as well. By
>>>>>> default it uses a binary conda package, so no yt-hg folder.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Comet, after reinstalling yt following your instructions, I am
>>>>>>> able to update yt now. However, when I submit a job, I get this error at
>>>>>>> "import yt":
>>>>>>> ...
>>>>>>>   File "/home/yuan/source/yt-conda/src/yt-hg/yt/__init__.py", line
>>>>>>> 78, in <module>
>>>>>>>     import numpy as np # For modern purposes
>>>>>>> ...
>>>>>>> from . import multiarray
>>>>>>> AttributeError: PyCapsule_Import "datetime.datetime_CAPI" is not
>>>>>>> valid
>>>>>>>
>>>>>>>
>>>>>> Are you sure you're using yt's python and numpy here? It looks like
>>>>>> you're importing yt's numpy using an incompatible python interpreter (e.g.
>>>>>> the comet system python).
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Any suggestions on how to fix this?
>>>>>>>
>>>>>>> Yuan
>>>>>>>
>>>>>>> On Wed, Jun 15, 2016 at 2:50 PM, Nathan Goldbaum <
>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 15, 2016 at 1:42 PM, Yuan Li <yuan at astro.columbia.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I am having trouble installing/running the latest yt on Xsede Smic
>>>>>>>>> and Comet clusters. I used to have yt3 on both machines and they were
>>>>>>>>> working fine. Recently, after failing to update yt, I decided to delete the
>>>>>>>>> old one and install the latest version. I am using the unmodified file from
>>>>>>>>> http://bitbucket.org/yt_analysis/yt/raw/yt/doc/install_script.sh
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Here are the problems I have:
>>>>>>>>> *On Smic*
>>>>>>>>> When I do "yt update" on the login node, or when I try to submit a
>>>>>>>>> yt job, I get this error message
>>>>>>>>> ...
>>>>>>>>> ValueError: numpy.dtype has the wrong size, try recompiling
>>>>>>>>>
>>>>>>>>> I have seen this error message before and fixed it somehow, but I
>>>>>>>>> do not remember how...
>>>>>>>>>
>>>>>>>>
>>>>>>>> Navigate to the checkout of the yt mercurial repository and do:
>>>>>>>>
>>>>>>>> $ ./clean.sh
>>>>>>>> $ python setup.py develop
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *On Comet*
>>>>>>>>> When I do "yt update", I get this error message:
>>>>>>>>> ...
>>>>>>>>> YT site-packages not in path, so you must
>>>>>>>>> update this installation manually by committing and
>>>>>>>>> merging your modifications to the code before
>>>>>>>>> updating to the newest changeset.
>>>>>>>>>
>>>>>>>>> I manually set YT_DEST in my bash_profile, but that does not seem
>>>>>>>>> to help. I can however submit a yt job and it will run fine, but not in
>>>>>>>>> parallel. Before I reinstalled yt, I was able to run yt3 in parallel on
>>>>>>>>> Comet after loading mpi4py.
>>>>>>>>>
>>>>>>>>> Also, when I do
>>>>>>>>> "source ~/source/yt-conda/bin/activate" I get this error message:
>>>>>>>>> Error: no environment provided.
>>>>>>>>> and it seems that I do not even need to activate yt anymore (is it
>>>>>>>>> because I have "export PATH=/home/yuanli/source/yt-conda/bin:$PATH" in my
>>>>>>>>> bash_profile?). Maybe I missed something in the doc. I am very confused.
>>>>>>>>>
>>>>>>>>
>>>>>>>> It looks like you ran the install script with INST_YT_SOURCE=0 (the
>>>>>>>> default). Try again with INST_YT_SOURCE=1. You will also likely need to do
>>>>>>>> "pip install mpi4py" to get an mpi4py installation linked against comet's
>>>>>>>> MPI libraries.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks in advance!
>>>>>>>>> Yuan
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160617/d9fbca58/attachment-0001.htm>