[yt-users] Installing yt on Xsede Smic and Comet

Fri Jun 17 11:30:33 PDT 2016

The would be nice, especially for people like me who always forget to check
the log file :).

Yuan

On Fri, Jun 17, 2016 at 1:10 PM, Nathan Goldbaum <nathan12343 at gmail.com>
wrote:

>
>
> On Fri, Jun 17, 2016 at 11:51 AM, Yuan Li <bear0980 at gmail.com> wrote:
>
>> Hi Nathan,
>>
>> Aha! "yt" executable did not exist! The installation went smoothly so I
>> did not even think about checking the log, but there were actually some
>> error messages related to Cython. I realized that a system python module
>> was automatically loaded upon login, so I unloaded it and reinstalled yt.
>> It is working now. Thank you!
>>
>
> Ah, that's valuable feedback. Perhaps the install script should do some
> rudimentary checking to verify that yt is installed correctly.
>
>
>>
>> Best,
>> Yuan
>>
>> On Thu, Jun 16, 2016 at 4:49 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>> wrote:
>>
>>> Does yt's bin directory show up in your PATH environment variable? If
>>> so, does a "yt" executable exist in that directory? If not, how are you
>>> setting your PATH? Does it work if you prepend yt's bin directory to your
>>> PATH manually in your current ssh session?
>>>
>>> On Thu, Jun 16, 2016 at 3:43 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>>
>>>> Yes I did that but it did not seem to do anything.
>>>>
>>>> Maybe I should email Smic people for help?
>>>>
>>>> Yuan
>>>>
>>>> On Thu, Jun 16, 2016 at 4:39 PM, Nathan Goldbaum <nathan12343 at gmail.com
>>>> > wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Jun 16, 2016 at 3:34 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>>>>
>>>>>> Hi Nathan,
>>>>>>
>>>>>> On Comet:
>>>>>> I set PYTHONPATH in bash_profile, but maybe that was not enough. Now
>>>>>> in my script, I do "mpirun -np 96 /home/yuan/source/yt-conda/bin/python
>>>>>> **.py", and it seems to be working. Yay!
>>>>>>
>>>>>>
>>>>>> On Smic:
>>>>>> I reinstalled yt with INST_YT_SOURCE=1. I set PATH & YT_DEST in my
>>>>>> bash_profile. However, the system does not recognize yt.
>>>>>>  -bash: yt: command not found
>>>>>> even after I do "source ~/source/yt-conda/bin/activate". I thought I
>>>>>> set up everything just the same as Comet...
>>>>>>
>>>>>
>>>>> With the new install script, the activate script is unnecessary.
>>>>> Simply prepend the location of the yt installation's bin directory to your
>>>>> PATH. The install script should print out explicit code that you can
>>>>> copy/paste into your .bashrc when the script finishes.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Yuan
>>>>>>
>>>>>> On Wed, Jun 15, 2016 at 4:50 PM, Nathan Goldbaum <
>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 15, 2016 at 3:41 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Nathan,
>>>>>>>>
>>>>>>>> Thank you for the quick response!
>>>>>>>>
>>>>>>>> On Smic: Where is this "clean.sh" file? In the old yt3, it is under
>>>>>>>> yt-x86_64/src/yt-hg, but I cannot find it in the new "/yt-conda/".
>>>>>>>>
>>>>>>>
>>>>>>> Ah, so it looks like you used INST_YT_SOURCE=0 here as well. By
>>>>>>> default it uses a binary conda package, so no yt-hg folder.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Comet, after reinstalling yt following your instructions, I am
>>>>>>>> able to update yt now. However, when I submit a job, I get this error at
>>>>>>>> "import yt":
>>>>>>>> ...
>>>>>>>>   File "/home/yuan/source/yt-conda/src/yt-hg/yt/__init__.py", line
>>>>>>>> 78, in <module>
>>>>>>>>     import numpy as np # For modern purposes
>>>>>>>> ...
>>>>>>>> from . import multiarray
>>>>>>>> AttributeError: PyCapsule_Import "datetime.datetime_CAPI" is not
>>>>>>>> valid
>>>>>>>>
>>>>>>>>
>>>>>>> Are you sure you're using yt's python and numpy here? It looks like
>>>>>>> you're importing yt's numpy using an incompatible python interpreter (e.g.
>>>>>>> the comet system python).
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Any suggestions on how to fix this?
>>>>>>>>
>>>>>>>> Yuan
>>>>>>>>
>>>>>>>> On Wed, Jun 15, 2016 at 2:50 PM, Nathan Goldbaum <
>>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 15, 2016 at 1:42 PM, Yuan Li <yuan at astro.columbia.edu>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I am having trouble installing/running the latest yt on Xsede
>>>>>>>>>> Smic and Comet clusters. I used to have yt3 on both machines and they were
>>>>>>>>>> working fine. Recently, after failing to update yt, I decided to delete the
>>>>>>>>>> old one and install the latest version. I am using the unmodified file from
>>>>>>>>>> http://bitbucket.org/yt_analysis/yt/raw/yt/doc/install_script.sh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Here are the problems I have:
>>>>>>>>>> *On Smic*
>>>>>>>>>> When I do "yt update" on the login node, or when I try to submit
>>>>>>>>>> a yt job, I get this error message
>>>>>>>>>> ...
>>>>>>>>>> ValueError: numpy.dtype has the wrong size, try recompiling
>>>>>>>>>>
>>>>>>>>>> I have seen this error message before and fixed it somehow, but I
>>>>>>>>>> do not remember how...
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Navigate to the checkout of the yt mercurial repository and do:
>>>>>>>>>
>>>>>>>>> $ ./clean.sh
>>>>>>>>> $ python setup.py develop
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *On Comet*
>>>>>>>>>> When I do "yt update", I get this error message:
>>>>>>>>>> ...
>>>>>>>>>> YT site-packages not in path, so you must
>>>>>>>>>> update this installation manually by committing and
>>>>>>>>>> merging your modifications to the code before
>>>>>>>>>> updating to the newest changeset.
>>>>>>>>>>
>>>>>>>>>> I manually set YT_DEST in my bash_profile, but that does not seem
>>>>>>>>>> to help. I can however submit a yt job and it will run fine, but not in
>>>>>>>>>> parallel. Before I reinstalled yt, I was able to run yt3 in parallel on
>>>>>>>>>> Comet after loading mpi4py.
>>>>>>>>>>
>>>>>>>>>> Also, when I do
>>>>>>>>>> "source ~/source/yt-conda/bin/activate" I get this error message:
>>>>>>>>>> Error: no environment provided.
>>>>>>>>>> and it seems that I do not even need to activate yt anymore (is
>>>>>>>>>> it because I have "export PATH=/home/yuanli/source/yt-conda/bin:$PATH" in
>>>>>>>>>> my bash_profile?). Maybe I missed something in the doc. I am very confused.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It looks like you ran the install script with INST_YT_SOURCE=0
>>>>>>>>> (the default). Try again with INST_YT_SOURCE=1. You will also likely need
>>>>>>>>> to do "pip install mpi4py" to get an mpi4py installation linked against
>>>>>>>>> comet's MPI libraries.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks in advance!
>>>>>>>>>> Yuan
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160617/3cdb213b/attachment.htm>