[yt-users] yt on Pleiades
Nathan Goldbaum
nathan12343 at gmail.com
Mon Oct 19 20:48:51 PDT 2015
On Mon, Oct 19, 2015 at 8:43 PM, Mateusz Ruszkowski <mateuszr at umich.edu>
wrote:
>
>
> Good call. It appears that cython was not included in miniconda3 (and was
> not present under miniconda3/bin/):
>
> % conda update cython
> Error: package 'cython' is not installed in
> /nobackupnfs2/mruszkow/miniconda3
> % python test.py
> Traceback (most recent call last):
> File "test.py", line 1, in <module>
> import cython
> ImportError: No module named ‘python'
>
>
> So I ran:
>
> conda install cython [ now miniconda3/bin/ does include cython ]
> % python test.py
> 0.23.4
> /nobackupnfs2/mruszkow/miniconda3/lib/python3.4/site-packages/cython.py
>
> cd yt
> python setup.py develop
>
> [ appears to have ran properly and, after a long list of warnings, I got: ]
>
> [….]
> Adding yt 3.3.dev0 to easy-install.pth file
> Installing yt script to /nobackupnfs2/mruszkow/miniconda3/bin
> Installing iyt script to /nobackupnfs2/mruszkow/miniconda3/bin
>
> Installed /nobackupnfs2/mruszkow/yt
> Processing dependencies for yt==3.3.dev0
> Finished processing dependencies for yt==3.3.dev0
> %
>
>
> So, how do I activate this installation?
No need to activate, just make sure miniconda's python is in your path. You
should be able to "import yt" in miniconda's python right now.
>
>
>
>
>
>
>
> > On Oct 19, 2015, at 10:06 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> >
> >
> >
> > On Mon, Oct 19, 2015 at 6:52 PM, Mateusz Ruszkowski <mateuszr at umich.edu>
> wrote:
> >
> > You were right that I didn’t have python from miniconda in my PATH and
> so I was picking up the (old) system version of python. I prepended PATH
> variable and “which python” now points to the right version of python from
> miniconda. Nevertheless, the problem remains as illustrated below. I am
> puzzled why this is happening ...
> >
> > % pwd
> > /nobackupnfs2/mruszkow
> > % ls
> > Miniconda3-latest-Linux-x86_64.sh
> > % bash Miniconda3-latest-Linux-x86_64.sh
> > % setenv PATH /nobackupnfs2/mruszkow/miniconda3/bin:${PATH}
> > % echo $PATH
> > % which python
> > % conda install yt
> > % module load pkgsrc/2015Q2
> > % hg clone https://bitbucket.org/yt_analysis/yt
> > % ls
> > Miniconda3-latest-Linux-x86_64.sh miniconda3 yt
> > % pwd
> > /nobackupnfs2/mruszkow
> > % cd yt/
> > % hg up yt
> > % which python
> > /nobackupnfs2/mruszkow/miniconda3/bin/python
> > % python setup.py develop
> > Cython is a build-time requirement for the source tree of yt.
> > Please either install yt from a provided, release tarball,
> > or install Cython (version 0.22 or higher).
> > You may be able to accomplish this by typing:
> > pip install -U Cython
> >
> > Does it persist if you do "conda update cython"? There should be builds
> of Cython 0.23.4 on linux.
> >
> > If it does persist, can you paste the output of
> >
> > import cython
> > print(cython.__version__)
> > print(cython.__file__)
> >
> > Using the miniconda python.
> >
> > Sorry this is so complicated, unfortunately this is wrapped up with the
> general awfulness of packaging in python and one of several things could be
> going wrong.
> >
> >
> >
> >
> >> On Oct 19, 2015, at 7:53 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> >>
> >> You don't have the python from miniconda in your PATH and are picking
> up a version of python 2.6 or older from the operating system on Pleiades.
> >>
> >> What's the output of "which python"? If the path returned by that
> command doesn't have "miniconda" in it, it's the wrong python executable.
> You might need to manually add miniconda's bin directory to your PATH
> environment variable.
> >>
> >> On Mon, Oct 19, 2015 at 4:51 PM, Mateusz Ruszkowski <mateuszr at umich.edu>
> wrote:
> >>
> >>
> >>
> >> The reason I originally loaded python_2.7.2 was to avoid the error
> message below. Now that python_2.7.2 is unloaded this message is back ...
> >>
> >> % module unload python_2.7.2
> >> % python setup.py develop
> >> yt currently requires Python version 2.7
> >> certain features may fail unexpectedly and silently with older versions.
> >> %
> >>
> >> As the next step I will try to repeat the whole process from scratch
> w/o using module load pkgsrc/2015Q2. Instead, I will just use standalone
> mercurial installation to see if this helps.
> >>
> >>
> >>
> >>
> >>
> >>
> >>> On Oct 19, 2015, at 7:31 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> >>>
> >>>
> >>>
> >>> On Mon, Oct 19, 2015 at 4:11 PM, Mateusz Ruszkowski <
> mateuszr at umich.edu> wrote:
> >>>
> >>>
> >>> Hi Nathan/Desika,
> >>>
> >>> Thanks! Following your suggestions, I replaced "conda build
> conda-recipes/yt" with "conda install yt.” I am also able to do "conda
> install pip" w/o problem. But I nevertheless get the same error message
> when I execute the last command in this series:
> >>>
> >>> bash Miniconda3-latest-Linux-x86_64.sh
> >>> miniconda3/bin/conda install yt [instead of:
> miniconda3/bin/conda build conda-recipes/yt]
> >>> miniconda3/bin/conda install pip
> >>> hg clone https://bitbucket.org/yt_analysis/yt
> >>> cd yt/
> >>> hg up yt
> >>> python setup.py develop
> >>>
> >>> Cython is a build-time requirement for the source tree of yt.
> >>> Please either install yt from a provided, release tarball,
> >>> or install Cython (version 0.22 or higher).
> >>> You may be able to accomplish this by typing:
> >>> pip install -U Cython
> >>>
> >>> Is there anything obvious in the above command sequence that is wrong?
> I am wondering if this problem has anything to do with how you setup
> various modules on Pleiades. For example, in order to use mercuria,l I do
> "module load pkgsrc/2015Q2” I also do “module load python_2.7.2” The
> pkgsrc/2015Q2 package loads a number of obsolete modules and perhaps they
> are messing things up?
> >>>
> >>> Yes, that is possible.
> >>>
> >>> You want to make sure that you are running the miniconda python and
> not the python loaded by the module. You can check by doing "which python".
> If you're using miniconda, you want to unload the python_2.7.2 module.
> >>>
> >>>
> >>> Thanks,
> >>> Mateusz
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>> On Oct 19, 2015, at 5:05 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Mon, Oct 19, 2015 at 1:42 PM, Mateusz Ruszkowski <
> mateuszr at umich.edu> wrote:
> >>>>
> >>>>
> >>>> Hi Nathan,
> >>>>
> >>>> Thanks for your help. I think also this almost works ... I
> downloaded the latest miniconda and followed this procedure:
> >>>>
> >>>>
> >>>> bash Miniconda3-latest-Linux-x86_64.sh
> >>>> miniconda3/bin/conda build conda-recipes/yt
> >>>>
> >>>> Here you want to do "conda install yt" to get all of yt's
> dependencies. From there you can install the development version of yt.
> You can "conda install" must python packages.
> >>>>
> >>>> As desika mentioned, you can also install pip with "conda install
> pip".
> >>>>
> >>>> I'd caution against using the full anaconda distribution since
> anaconda's MPI will shadow the system MPI installation unless you
> explicitly remove it.
> >>>>
> >>>> hg clone https://bitbucket.org/yt_analysis/yt
> >>>> cd yt/
> >>>> hy up yt
> >>>> python setup.py develop
> >>>>
> >>>> This resulted in:
> >>>>
> >>>> Cython is a build-time requirement for the source tree of yt.
> >>>> Please either install yt from a provided, release tarball,
> >>>> or install Cython (version 0.22 or higher).
> >>>> You may be able to accomplish this by typing:
> >>>> pip install -U Cython
> >>>>
> >>>> which I am not able to install via "pip install -U Cython." So this
> is essentially the same error message I got when I attempted to install yt
> using the method described in my previous e-mail. Any idea what may still
> be wrong?
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Mateusz
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> On Oct 19, 2015, at 2:26 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> >>>>>
> >>>>> Hi Mateusz
> >>>>>
> >>>>> Rather than using the python module on pleiades, I'd suggest setting
> up your python stack on your own in your home folder.
> >>>>>
> >>>>> By far the quickest way to do this is to install miniconda and then
> install yt into the miniconda environment using "conda install yt". You
> could also use the yt install script, although you might have issues with
> that if Pleiades has an older (insecure) version of OpenSSL installed,
> which is sadly quite common on supercomputers these days...
> >>>>>
> >>>>> Personally, I set up my python environment on Pleiades using
> miniconda.
> >>>>>
> >>>>> -Nathan
> >>>>>
> >>>>> On Mon, Oct 19, 2015 at 11:23 AM, Mateusz Ruszkowski <
> mateuszr at umich.edu> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I am trying to install yt on Pleiades. I was able to complete a
> number installation steps but I eventually got an error message related to
> Cython (Cython seems to be installed though). The error message is attached
> below. Does anybody have an idea what needs to be done to complete this
> installation?
> >>>>>
> >>>>> Thanks,
> >>>>> Mateusz
> >>>>>
> >>>>> ------------------------------------------------
> >>>>> Dr. Mateusz Ruszkowski
> >>>>> Associate Professor
> >>>>>
> >>>>> University of Michigan
> >>>>> Department of Astronomy
> >>>>> 301C West Hall
> >>>>> 1085 South University Ave.
> >>>>> Ann Arbor, MI 48109-1107, USA
> >>>>>
> >>>>> e-mail: mateuszr at umich.edu
> >>>>>
> >>>>>
> >>>>> ————————————————————————
> >>>>>
> >>>>> % module load pkgsrc/2014Q3
> >>>>> % module load python/2.7.9
> >>>>> % hg clone https://bitbucket.org/yt_analysis/yt
> >>>>> warning: bitbucket.org certificate with fingerprint
> 46:de:34:e7:9b:18:cd:7f:ae:fd:8b:e3:bc:f4:1a:5e:38:d7:ac:24 not verified
> (check hostfingerprints or web.cacerts config setting)
> >>>>> destination directory: yt
> >>>>> requesting all changes
> >>>>> adding changesets
> >>>>> adding manifests
> >>>>> adding file changes
> >>>>> added 17568 changesets with 36510 changes to 3053 files (+3 heads)
> >>>>> updating to bookmark @ on branch stable
> >>>>> 1209 files updated, 0 files merged, 0 files removed, 0 files
> unresolved
> >>>>> % cd yt
> >>>>> % hg update yt
> >>>>> 317 files updated, 0 files merged, 148 files removed, 0 files
> unresolved
> >>>>> (leaving bookmark @)
> >>>>> % python setup.py install --user --prefix=/nobackupnfs2/mruszkow
> >>>>> Cython is a build-time requirement for the source tree of yt.
> >>>>> Please either install yt from a provided, release tarball,
> >>>>> or install Cython (version 0.22 or higher).
> >>>>> You may be able to accomplish this by typing:
> >>>>> pip install -U Cython
> >>>>> % pip install -U Cython
> >>>>> Downloading/unpacking Cython from
> https://pypi.python.org/packages/source/C/Cython/Cython-0.23.4.tar.gz#md5=157df1f69bcec6b56fd97e0f2e057f6e
> >>>>> Downloading Cython-0.23.4.tar.gz (1.6MB): 1.6MB downloaded
> >>>>> Running setup.py (path:/tmp/pip_build_mruszkow/Cython/setup.py)
> egg_info for package Cython
> >>>>> Unable to find pgen, not compiling formal grammar.
> >>>>>
> >>>>> warning: no files found matching '*.pyx' under directory
> 'Cython/Debugger/Tests'
> >>>>> warning: no files found matching '*.pxd' under directory
> 'Cython/Debugger/Tests'
> >>>>> warning: no files found matching '*.h' under directory
> 'Cython/Debugger/Tests'
> >>>>> warning: no files found matching '*.pxd' under directory
> 'Cython/Utility'
> >>>>> Installing collected packages: Cython
> >>>>> Found existing installation: Cython 0.21.1
> >>>>> Uninstalling Cython:
> >>>>> Cleaning up...
> >>>>> Exception:
> >>>>> Traceback (most recent call last):
> >>>>> File
> "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/basecommand.py", line
> 122, in main
> >>>>> status = self.run(options, args)
> >>>>> File
> "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/commands/install.py",
> line 283, in run
> >>>>> requirement_set.install(install_options, global_options,
> root=options.root_path)
> >>>>> File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/req.py",
> line 1431, in install
> >>>>> requirement.uninstall(auto_confirm=True)
> >>>>> File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/req.py",
> line 598, in uninstall
> >>>>> paths_to_remove.remove(auto_confirm)
> >>>>> File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/req.py",
> line 1836, in remove
> >>>>> renames(path, new_path)
> >>>>> File
> "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/util.py", line 295, in
> renames
> >>>>> shutil.move(old, new)
> >>>>> File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 300, in
> move
> >>>>> rmtree(src)
> >>>>> File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 247, in
> rmtree
> >>>>> rmtree(fullname, ignore_errors, onerror)
> >>>>> File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 252, in
> rmtree
> >>>>> onerror(os.remove, fullname, sys.exc_info())
> >>>>> File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 250, in
> rmtree
> >>>>> os.remove(fullname)
> >>>>> OSError: [Errno 30] Read-only file system:
> '/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/Cython/Build/BuildExecutable.py'
> >>>>>
> >>>>> Storing debug log for failure in /u/mruszkow/.pip/pip.log
> >>>>> %
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> yt-users mailing list
> >>>>> yt-users at lists.spacepope.org
> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> yt-users mailing list
> >>>>> yt-users at lists.spacepope.org
> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> yt-users mailing list
> >>>> yt-users at lists.spacepope.org
> >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> yt-users mailing list
> >>>> yt-users at lists.spacepope.org
> >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>>
> >>>
> >>> _______________________________________________
> >>> yt-users mailing list
> >>> yt-users at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>>
> >>>
> >>> _______________________________________________
> >>> yt-users mailing list
> >>> yt-users at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>
> >>
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>
> >>
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >
> >
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >
> >
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20151019/b7198ac9/attachment.html>
More information about the yt-users
mailing list