[yt-users] installing YT without internet connection
Luigi Iapichino
luigi at uni-heidelberg.de
Mon Feb 3 06:50:43 PST 2014
Dear Philipp, dear yt users,
I attach to this email the modified version of my install script, used to
install yt 2.6 on a machine without internet access. The procedure for the
installation is also attached as a separate txt file, but I copy it below, for
sake of simplicity, in case of later discussion.
As I wrote a few hours ago, this is not straightforward, but it works for me,
and I haven't come to any better solution. If anyone else has better
suggestions, I would be glad to hear from them.
Cheers,
Luigi
---
Here I describe the installation of yt 2.6. For other versions, please check
the version numbers of the software that I mention below.
Let's start with a bit of syntax:
<remote-machine>: the remote machine without Internet access (a high-
performance supercomputer, most of the times);
<remote-username>: your username on that machine;
<remote-directory>: a directory containing the yt source, in the remote
machine (create it, if not already existing);
<local-machine>: a PC with Internet access (your local PC, for example)
<local-directory>: on your local machine, this is an initially empty directory
where you will mount the content of <remote-directory>.
Moreover, every command given below will be preceded by <remote-machine> or
<local-machine>, indicating on which machine it should be executed.
First of all, in order to install yt, or to update it by using Mercurial on a
machine with no Internet access, you have to remotely mount the source
directory in your local PC.
So, in order to remotely mount <remote-directory> on <local-machine>, the
command is:
(<local-machine>) sshfs -o workaround=rename <remote-username>@<remote-
machine>:<remote-directory> <local-directory>
After this command, on <local-directory> you will find the content of <remote-
directory>. After working with it, you can unmount with
(<local-machine>) fusermount -u <local-directory>
Beware: both sshfs and fusermount have to be installed on <local-machine>, and
in order to use them you should be inserted in some special user group (I
don't remember which one exactly, but if you ask your local administrator it
will be fixed for you).
Now, the yt part: I attach here a modified install script. The most outstanding
changes with respect to the standard script are:
- two paths for DEST_DIR, one for <local-machine> and one for <remote-
machine>. You must switch between the two of them, because the script will be
executed partly on <local-machine>, and partly on <remote-machine>.
- INST_HG=0 : you don't need Mercurial on <remote-machine>, but you have it
already on <local-machine>. Beware: it's somehow funny, but although you don't
need hg on <remote-machine>, it should be installed there. If not, please
substite the lines 803-810 with the following ones:
# We assume that hg can be found in the path.
# if type -P hg &>/dev/null
# then
export HG_EXEC=hg
# else
# echo "Cannot find mercurial. Please set INST_HG=1."
# do_exit
# fi
which simply disable a check that would raise an error.
- Comment the installation of pip (l. 844-845: it does not work on SuperMUC,
with this installation procedure)
- Installation of distribute: before starting the script, uncomment the lines
841-842. After the install script reaches that point, comment them again.
- l. 882 to 885: I moved there the installation of some required software, not
included in the initial download step.
Here is finally the installation procedure:
1) (<local-machine>) Perform the remote mount, as described above.
2) (<local-machine>) Create a directory called yt-... and copy there the
modified installation script. Make the script executable.
3) (<local-machine>) Execute the script, up to the point where the sources are
downloaded. The file distribute-0.6.32.tar.gz is not taken together with the
other ones, It must be added to src manually (important!). Moreover, you need
two additional source files copied in src: pyparsing-1.5.6.tar.gz and python-
dateutil-1.5.tar.gz (look for them on the Web).
4) (<remote-machine>) Execute again the install script, which now compiles the
tools until Python and then fails.
5) (<local-machine>) For going on, the script executes the hg-related commands
for the yt source. After this part it crashes, and I go again to <remote-
machine>.
6) (<remote-machine>) It will try to update yt from here, failing. Stop again
and do it from <local-machine>, then go back to (<remote-machine>).
7) (<remote-machine>) Go on with the script, which (hopefully!) now proceeds
to the end without further problems.
---
On Monday 03 February 2014 09:27:31 Luigi Iapichino wrote:
> Hi Philipp,
>
> I did exactly what you are looking for, on SuperMUC in Garching. I will send
> my modified version of the install script, with instructions for the
> installation, in the course of the day. It's a rather cumbersome procedure
> so, if anyone else has met and solved the same problem, please join the
> discussion, so that we can compare and improve together.
>
> Cheers,
>
> Luigi
>
> On Monday 03 February 2014 08:47:12 Philipp Girichidis wrote:
> > Dear YT users,
> >
> > I have a problem during the installation of yt. On the supercomputer,
> > where
> > all our simulations are stored, we do not have internet connection because
> > of 'security reasons'. So, I tried to download the data on another
> > machine,
> > then copy the files to this supercomputer and tried to install everything
> > from the packages. But there are quite a few other points in the install
> > script, where the setup process needs access to the internet. I also tried
> > to set up a virtual machine with a linux system similar to the one in the
> > computing center, but no success. Did anyone run into a similar problem
> > and
> > got a solution or has a guess how to first download everything and then
> > installing yt without further internet connection?
> > Best regards,
> > Philipp
--
---------------------------------------------------------------
Luigi Iapichino
Universität Heidelberg, Zentrum für Astronomie
Institut für Theoretische Astrophysik
Philosophenweg 12, D-69120 Heidelberg, Germany
Tel: +49 6221 548983, Fax: +49 6221 544221
e-mail: luigi at uni-heidelberg.de
URL: http://www.ita.uni-heidelberg.de/~luigi/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: install_script-remote.sh
Type: application/x-shellscript
Size: 39687 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20140203/bcb4527c/attachment-0001.bin>
-------------- next part --------------
Here I describe the installation of yt 2.6. For other versions, please check the version numbers of the software that I mention below.
Let's start with a bit of syntax:
<remote-machine>: the remote machine without Internet access (a high-performance supercomputer, most of the times);
<remote-username>: your username on that machine;
<remote-directory>: a directory containing the yt source, in the remote machine (create it, if not already existing);
<local-machine>: a PC with Internet access (your local PC, for example)
<local-directory>: on your local machine, this is an initially empty directory where you will mount the content of <remote-directory>.
Moreover, every command given below will be preceded by <remote-machine> or <local-machine>, indicating on which machine it should be executed.
First of all, in order to install yt, or to update it by using Mercurial on a machine with no Internet access, you have to remotely mount the source directory in your local PC.
So, in order to remotely mount <remote-directory> on <local-machine>, the command is:
(<local-machine>) sshfs -o workaround=rename <remote-username>@<remote-machine>:<remote-directory> <local-directory>
After this command, on <local-directory> you will find the content of <remote-directory>. After working with it, you can unmount with
(<local-machine>) fusermount -u <local-directory>
Beware: both sshfs and fusermount have to be installed on <local-machine>, and in order to use them you should be inserted in some special user group (I don't remember which one exactly, but if you ask your local administrator it will be fixed for you).
Now, the yt part: I attach here a modified install script. The most outstanding changes with respect to the standard script are:
- two paths for DEST_DIR, one for <local-machine> and one for <remote-machine>. You must switch between the two of them, because the script will be executed partly on <local-machine>, and partly on <remote-machine>.
- INST_HG=0 : you don't need Mercurial on <remote-machine>, but you have it already on <local-machine>. Beware: it's somehow funny, but although you don't need hg on <remote-machine>, it should be installed there. If not, please substite the lines 803-810 with the following ones:
# We assume that hg can be found in the path.
# if type -P hg &>/dev/null
# then
export HG_EXEC=hg
# else
# echo "Cannot find mercurial. Please set INST_HG=1."
# do_exit
# fi
which simply disable a check that would raise an error.
- Comment the installation of pip (l. 844-845: it does not work on SuperMUC, with this installation procedure)
- Installation of distribute: before starting the script, uncomment the lines 841-842. After the install script reaches that point, comment them again.
- l. 882 to 885: I moved there the installation of some required software, not included in the initial download step.
Here is finally the installation procedure:
1) (<local-machine>) Perform the remote mount, as described above.
2) (<local-machine>) Create a directory called yt-... and copy there the modified installation script. Make the script executable.
3) (<local-machine>) Execute the script, up to the point where the sources are downloaded. The file distribute-0.6.32.tar.gz is not taken together with the other ones, It must be added to src manually (important!). Moreover, you need two additional source files copied in src: pyparsing-1.5.6.tar.gz and python-dateutil-1.5.tar.gz (look for them on the Web).
4) (<remote-machine>) Execute again the install script, which now compiles the tools until Python and then fails.
5) (<local-machine>) For going on, the script executes the hg-related commands for the yt source. After this part it crashes, and I go again to <remote-machine>.
6) (<remote-machine>) It will try to update yt from here, failing. Stop again and do it from <local-machine>, then go back to (<remote-machine>.
7) (<remote-machine>) Go on with the script, which (hopefully!) now proceeds to the end without further problems.
More information about the yt-users
mailing list