[Yt-dev] [yt-users] install problems

[Matthew Turk]

Hi Britton and Stephen,

Sorry for the delay in replying, as I was away for the weekend.  I did
have a chance to think over a number of these issues, which I want to
address.

On Sat, Mar 12, 2011 at 1:22 PM, Stephen Skory <stephenskory at yahoo.com> wrote:
> Matt & Britton,
>
>>OS 10.6 comes with python 2.6.  Would it be better if they just used that python and didn't try to build another one?  Can we add another flag to the install script to optionally build python?
>
>
> Along the same lines, we could write some instructions on using yt with enthought python. It comes with many handy things...

Well, there are a couple things in here.  I'm not sure Enthought's
goals align with ours is the first and primary one.  However, the more
pragmatic one that I can put my finger on is that I don't think that
the EPD provides an HDF5 library that we can both *identify* and link
against, and certainly it does not provide one that can be used by
Enzo.  Other than the issue of HDF5, this solution would work.  If you
can verify that yt and Enzo can (find and) build against the Enthought
HDF5, this would probably be okay.  I added a few checks to the yt
build procedure to try to identify the correct HDF5.  (Note that
because Enthought provides framework builds it's a .dylib.

>
>>>>>Can you check the options in ./configure, and if none of those are
>
> There is no preference for that in ./configure AFAICT.

I see --without-included-gettext, but I believe that might just be
autotools cruft.

>
>>>>>obvious, can you investigate how much work getting a localize gettext
>>>>>to install is?
>
> I installed gettext - it didn't help.

Okay.

>
>>  Or, better yet, maybe this is improved/fixed in python
>>>>>2.7, and you could test that?
>
>
> I might try that out.
>

My guess now is that it won't.  I think the issue is simply an issue
of a broken gettext install coming from something else like ports or
fink or homebrew or who knows what.  It would also not surprise me if
the Apple compiler -- which has a history of "helpfully" adding
#include statements, non-intuitively linking against .dylibs/.sos that
are incorrect and generally not being as helpful as it thinks it is --
is doing something unhelpful.

Here's the chain of events as I see them:

0) The user has installed something that also installs a gettext
without the correct set of symbols in libintl.
1) The user attempts to install Python (2.6)
2) The _locale module, which acts as the C-bridge for gettext to
python, fails to compile
3) Later on, another module (hg in this case) attempts to import
_locale and fails

Looking over the Python 2.7.1 codebase, it appears that they have made
a number of changes to the _localemodule.c source that are consistent
with fixing this bug.  Could you give it a shot?  I have uploaded a
new install script here:

http://yt.enzotools.org/files/27_install_script.txt

(Server issues for the file extension.)

So as to Britton's suggestions, I wanted to start out with my broad
feelings and a brief delving into not-too-distant-history.

I think if we don't provide an installation script for OSX, people
simply won't install and use it.  This is how it used to be; back in
the day, it was basically impossible to get yt running anywhere.
(Anybody remember when a tertiary dependency required boost 1.31.1,
but not any other version?  Yowza.)

We used to provide not an installation script, but a bunch of binaries
I built myself that were included in a script.  These were all linked
against a "Framework" build of Python.  This became much, much harder
to keep up to date with OSX 10.6, because of the mixed mode
compilation for 64-bit/32-bit.  It got to the point where it was
unmaintainable, and on the suggestion of some people here (I think
Rick, actually, was the one who suggested it?  Maybe?) we moved to
trying to provide a unified installation script which did *not*
attempt to provide framework builds.  The utility of a framework build
is much less when you don't use GUIs, so a single, isolated stack
seemed to work better, and to be much less intrusive.  It also didn't
require sudo.

So if we were to return to not providing a Python but using the system
Python, there are some upsides and downsides.  The (apparent) upside
is that things would work better.  (I am skeptical.)  But, we would
then have to provide one of the following things:

1) A from-source build that installed the dependencies (Apple provides
a broken NumPy) using sudo into a systemwide framework build location
2) A from-source build that utilized the per-user site-packages
directory provided by distutils; this means that all the packages
would live under a home directory, but executables would not be
installed
3) A set of links to manual installation of binary builds of some
things and source builds of other things.

I think the first two are difficult.  The third one is easier for us,
but much, much harder for other people to do -- go through the steps,
install in the right order, mix between GUI/CLI.  I think the best
option is to fix it.  And I think 2.7 might fix it.  I really, really
don't want to go back to forcing people to install everything
manually; too much of our userbase is on OSX 10.6.  And while I think
if we get around a few of the oddities of linking on OSX (which is
probably unavoidable, considering how hostile to non-iOS developers
Apple is becoming) we can provide this unified stack.

Plus, I think there are a lot of interesting things we can leverage
the install stack to provide, and I'm not really sure we should give
up on it yet.

Thoughts, from OSX users?  Britton, Stephen, Rick, Cameron, Tom,
Brian, everybody?  (Are Jeff and I the only ones who don't use OSX
anymore?)

-Matt