[yt-users] Installing yt on Xsede Smic and Comet

Fri Jun 17 11:44:21 PDT 2016

We do this in Trident.  We try to run a very simple functional test to
assure the code works before telling the user that everything is set.  Of
course, Trident is simpler than yt, but I still think it's a good idea.
http://trident.readthedocs.io/en/latest/installation.html#step-3-get-ionization-table-and-verify-installation

Cameron

On Fri, Jun 17, 2016 at 11:30 AM, Yuan Li <bear0980 at gmail.com> wrote:

> The would be nice, especially for people like me who always forget to
> check the log file :).
>
> Yuan
>
> On Fri, Jun 17, 2016 at 1:10 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
>
>>
>>
>> On Fri, Jun 17, 2016 at 11:51 AM, Yuan Li <bear0980 at gmail.com> wrote:
>>
>>> Hi Nathan,
>>>
>>> Aha! "yt" executable did not exist! The installation went smoothly so I
>>> did not even think about checking the log, but there were actually some
>>> error messages related to Cython. I realized that a system python module
>>> was automatically loaded upon login, so I unloaded it and reinstalled yt.
>>> It is working now. Thank you!
>>>
>>
>> Ah, that's valuable feedback. Perhaps the install script should do some
>> rudimentary checking to verify that yt is installed correctly.
>>
>>
>>>
>>> Best,
>>> Yuan
>>>
>>> On Thu, Jun 16, 2016 at 4:49 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>>> wrote:
>>>
>>>> Does yt's bin directory show up in your PATH environment variable? If
>>>> so, does a "yt" executable exist in that directory? If not, how are you
>>>> setting your PATH? Does it work if you prepend yt's bin directory to your
>>>> PATH manually in your current ssh session?
>>>>
>>>> On Thu, Jun 16, 2016 at 3:43 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>>>
>>>>> Yes I did that but it did not seem to do anything.
>>>>>
>>>>> Maybe I should email Smic people for help?
>>>>>
>>>>> Yuan
>>>>>
>>>>> On Thu, Jun 16, 2016 at 4:39 PM, Nathan Goldbaum <
>>>>> nathan12343 at gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 16, 2016 at 3:34 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Nathan,
>>>>>>>
>>>>>>> On Comet:
>>>>>>> I set PYTHONPATH in bash_profile, but maybe that was not enough. Now
>>>>>>> in my script, I do "mpirun -np 96 /home/yuan/source/yt-conda/bin/python
>>>>>>> **.py", and it seems to be working. Yay!
>>>>>>>
>>>>>>>
>>>>>>> On Smic:
>>>>>>> I reinstalled yt with INST_YT_SOURCE=1. I set PATH & YT_DEST in my
>>>>>>> bash_profile. However, the system does not recognize yt.
>>>>>>>  -bash: yt: command not found
>>>>>>> even after I do "source ~/source/yt-conda/bin/activate". I thought I
>>>>>>> set up everything just the same as Comet...
>>>>>>>
>>>>>>
>>>>>> With the new install script, the activate script is unnecessary.
>>>>>> Simply prepend the location of the yt installation's bin directory to your
>>>>>> PATH. The install script should print out explicit code that you can
>>>>>> copy/paste into your .bashrc when the script finishes.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Yuan
>>>>>>>
>>>>>>> On Wed, Jun 15, 2016 at 4:50 PM, Nathan Goldbaum <
>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 15, 2016 at 3:41 PM, Yuan Li <bear0980 at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Nathan,
>>>>>>>>>
>>>>>>>>> Thank you for the quick response!
>>>>>>>>>
>>>>>>>>> On Smic: Where is this "clean.sh" file? In the old yt3, it is
>>>>>>>>> under yt-x86_64/src/yt-hg, but I cannot find it in the new "/yt-conda/".
>>>>>>>>>
>>>>>>>>
>>>>>>>> Ah, so it looks like you used INST_YT_SOURCE=0 here as well. By
>>>>>>>> default it uses a binary conda package, so no yt-hg folder.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Comet, after reinstalling yt following your instructions, I am
>>>>>>>>> able to update yt now. However, when I submit a job, I get this error at
>>>>>>>>> "import yt":
>>>>>>>>> ...
>>>>>>>>>   File "/home/yuan/source/yt-conda/src/yt-hg/yt/__init__.py", line
>>>>>>>>> 78, in <module>
>>>>>>>>>     import numpy as np # For modern purposes
>>>>>>>>> ...
>>>>>>>>> from . import multiarray
>>>>>>>>> AttributeError: PyCapsule_Import "datetime.datetime_CAPI" is not
>>>>>>>>> valid
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Are you sure you're using yt's python and numpy here? It looks like
>>>>>>>> you're importing yt's numpy using an incompatible python interpreter (e.g.
>>>>>>>> the comet system python).
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Any suggestions on how to fix this?
>>>>>>>>>
>>>>>>>>> Yuan
>>>>>>>>>
>>>>>>>>> On Wed, Jun 15, 2016 at 2:50 PM, Nathan Goldbaum <
>>>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 15, 2016 at 1:42 PM, Yuan Li <yuan at astro.columbia.edu
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I am having trouble installing/running the latest yt on Xsede
>>>>>>>>>>> Smic and Comet clusters. I used to have yt3 on both machines and they were
>>>>>>>>>>> working fine. Recently, after failing to update yt, I decided to delete the
>>>>>>>>>>> old one and install the latest version. I am using the unmodified file from
>>>>>>>>>>> http://bitbucket.org/yt_analysis/yt/raw/yt/doc/install_script.sh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Here are the problems I have:
>>>>>>>>>>> *On Smic*
>>>>>>>>>>> When I do "yt update" on the login node, or when I try to submit
>>>>>>>>>>> a yt job, I get this error message
>>>>>>>>>>> ...
>>>>>>>>>>> ValueError: numpy.dtype has the wrong size, try recompiling
>>>>>>>>>>>
>>>>>>>>>>> I have seen this error message before and fixed it somehow, but
>>>>>>>>>>> I do not remember how...
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Navigate to the checkout of the yt mercurial repository and do:
>>>>>>>>>>
>>>>>>>>>> $ ./clean.sh
>>>>>>>>>> $ python setup.py develop
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *On Comet*
>>>>>>>>>>> When I do "yt update", I get this error message:
>>>>>>>>>>> ...
>>>>>>>>>>> YT site-packages not in path, so you must
>>>>>>>>>>> update this installation manually by committing and
>>>>>>>>>>> merging your modifications to the code before
>>>>>>>>>>> updating to the newest changeset.
>>>>>>>>>>>
>>>>>>>>>>> I manually set YT_DEST in my bash_profile, but that does not
>>>>>>>>>>> seem to help. I can however submit a yt job and it will run fine, but not
>>>>>>>>>>> in parallel. Before I reinstalled yt, I was able to run yt3 in parallel on
>>>>>>>>>>> Comet after loading mpi4py.
>>>>>>>>>>>
>>>>>>>>>>> Also, when I do
>>>>>>>>>>> "source ~/source/yt-conda/bin/activate" I get this error message:
>>>>>>>>>>> Error: no environment provided.
>>>>>>>>>>> and it seems that I do not even need to activate yt anymore (is
>>>>>>>>>>> it because I have "export PATH=/home/yuanli/source/yt-conda/bin:$PATH" in
>>>>>>>>>>> my bash_profile?). Maybe I missed something in the doc. I am very confused.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It looks like you ran the install script with INST_YT_SOURCE=0
>>>>>>>>>> (the default). Try again with INST_YT_SOURCE=1. You will also likely need
>>>>>>>>>> to do "pip install mpi4py" to get an mpi4py installation linked against
>>>>>>>>>> comet's MPI libraries.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks in advance!
>>>>>>>>>>> Yuan
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> yt-users mailing list
>>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>

-- 
Cameron Hummels
NSF Postdoctoral Fellow
Department of Astronomy
California Institute of Technology
http://chummels.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160617/6115663b/attachment.html>