[yt-dev] Packaging.

Britton Smith brittonsmith at gmail.com
Tue Sep 10 12:24:14 PDT 2013 

Why does conda come with libraries built on such an old system?  That seems
a bit strange, especially for Mac, which has evolved quite a bit since
10.5.  Is this sort of thing really not a problem for us?

Britton


On Tue, Sep 10, 2013 at 7:10 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Nathan,
>
> Okay -- but if that's the price we pay, I think it may still be worth it.
>
> -Matt
>
> On Tue, Sep 10, 2013 at 2:09 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> > Not 100% sure.  It certainly won't work on OS X at the moment.
> >
> > -Nathan
> >
> >
> > On Tue, Sep 10, 2013 at 10:59 AM, Matthew Turk <matthewturk at gmail.com>
> > wrote:
> >>
> >> Hi Nathan,
> >>
> >> On Tue, Sep 10, 2013 at 1:16 PM, Nathan Goldbaum <nathan12343 at gmail.com
> >
> >> wrote:
> >> > Hey John,
> >> >
> >> > A few questions:
> >> >
> >> > 1. What did you choose for INST_YT_SOURCE in get_yt.sh? If
> >> > INST_YT_SOURCE=1,
> >> > what happens when you try INST_YT_SOURCE=0?
> >> >
> >> > 2. Again, if you had INST_YT_SOURCE=1, when you manually run `setup.py
> >> > install` in the yt source repository, it should print out a few lines
> >> > showing the root directory for the PNG, Freetype, and HDF5 libraries
> it
> >> > tries to link against.  This should be the very first thing printed
> out
> >> > by
> >> > the setup script.  Is it linking against anaconda's libraries or
> system
> >> > libraries?
> >> >
> >> > 3. What's the output of `ldd
> >> >
> >> >
> /u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/lib/png_writer.so`?
> >> >
> >> > I'm beginning to suspect that we cannot link against the libraries
> >> > provided
> >> > by anaconda unless we tailor our build environment to match the sort
> of
> >> > gymnastics that `conda build` does to sanitize the build environment.
> >> > Just
> >> > yesterday I was having no end of grief trying to link against
> anaconda's
> >> > hdf5 library.  yt is able to link against this library inside a conda
> >> > build
> >> > environment, but when I try to link in my normal shell environment I
> run
> >> > into issues - mostly because the OS X hdf5 library on anaconda was
> >> > compiled
> >> > on an OS X 10.5 machine.
> >>
> >> If I read you correctly, you are saying that, practically speaking,
> >> this would mainly mean not using the yt-conda HDF5 for Enzo/FLASH/etc
> >> outside of yt.  Right?
> >>
> >> >
> >> >
> >> > On Tue, Sep 10, 2013 at 9:54 AM, John ZuHone <jzuhone at gmail.com>
> wrote:
> >> >>
> >> >> Hi Nathan and all,
> >> >>
> >> >> Here's a little more info on that error you asked about regarding
> >> >> glibc.
> >> >> It came up again for me on Pleiades, this time with yt itself.
> >> >>
> >> >> Traceback (most recent call last):
> >> >>   File "/u/jzuhone/anaconda/bin/yt", line 4, in <module>
> >> >>     from yt.utilities.command_line import run_main
> >> >>   File
> >> >>
> >> >>
> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/command_line.py",
> >> >> line 29, in <module>
> >> >>     from yt.mods import *
> >> >>   File "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/mods.py",
> >> >> line
> >> >> 60, in <module>
> >> >>     from yt.data_objects.api import \
> >> >>   File
> >> >>
> >> >>
> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/api.py",
> >> >> line 31, in <module>
> >> >>     from grid_patch import \
> >> >>   File
> >> >>
> >> >>
> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/grid_patch.py",
> >> >> line 35, in <module>
> >> >>     from yt.data_objects.data_containers import YTFieldData
> >> >>   File
> >> >>
> >> >>
> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/data_containers.py",
> >> >> line 45, in <module>
> >> >>     from yt.data_objects.derived_quantities import
> GridChildMaskWrapper
> >> >>   File
> >> >>
> >> >>
> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/derived_quantities.py",
> >> >> line 36, in <module>
> >> >>     from yt.utilities.parallel_tools.parallel_analysis_interface
> import
> >> >> \
> >> >>   File
> >> >>
> >> >>
> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> >> >> line 39, in <module>
> >> >>     from yt.utilities.lib import \
> >> >>   File
> >> >>
> >> >>
> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/lib/__init__.py",
> >> >> line 35, in <module>
> >> >>     from .png_writer import *
> >> >> ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found
> (required
> >> >> by
> >> >>
> >> >>
> /u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/lib/png_writer.so)
> >> >>
> >> >> Best,
> >> >>
> >> >> John
> >> >>
> >> >> On Sep 3, 2013, at 3:10 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> >> >> wrote:
> >> >>
> >> >> John, can you give a little more info on that error?  I'm a little
> >> >> concerned that there are binary incompatibilities that conda can't
> >> >> resolve.
> >> >> Perhaps this is something we should report on the conda issue
> tracker.
> >> >>
> >> >> Matt, can't we get the recipes in the same way we get the latest dev
> >> >> install script?  Something like:
> >> >>
> >> >> wget https://bitbucket.org/yt_analysis/yt_conda/raw/yt/meta.yaml
> >> >>
> >> >> This doesn't currently work, although it does work for the yt repo, I
> >> >> think because we use named branches in that repository.
> >> >>
> >> >>
> >> >> On Tue, Sep 3, 2013 at 12:00 PM, John ZuHone <jzuhone at gmail.com>
> wrote:
> >> >>>
> >> >>> Works fine for me on OS X x86_64.
> >> >>>
> >> >>> On my Goddard-controlled Linux x86_64 server, everything worked fine
> >> >>> except Mercurial, which I "conda install"-ed using the yt link, but
> >> >>> the
> >> >>> binary had an incompatibility which the glibc that was installed on
> >> >>> the
> >> >>> machine. Using pip to install Mercurial (which did it from source)
> was
> >> >>> the
> >> >>> workaround.
> >> >>>
> >> >>> On Sep 3, 2013, at 2:54 PM, Matthew Turk <matthewturk at gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>> > Hi all,
> >> >>> >
> >> >>> > Thanks to everybody who has reported back on testing.  After some
> >> >>> > talking both offline and on IRC, as well as here, I think we would
> >> >>> > need to do the following things:
> >> >>> >
> >> >>> > * Make a single script that grabs the appropriate distribution of
> >> >>> > miniconda and installs it.  Right now I have a mechanism for doing
> >> >>> > this, but it's currently tied to an architecture.
> >> >>> > * Create a mechanism for installing all the packages we need.
> >> >>> > Nearly
> >> >>> > all are available inside the Continuum repos.  What we're hung up
> on
> >> >>> > is that source installs require a "recipe", and transmitting the
> >> >>> > recipe is where I don't have an idea of what to do.
> >> >>> > * Test this out lots of places
> >> >>> > * Clean up the edges in the (new) install script
> >> >>> > * Move the old install script to maintenance mode
> >> >>> > * Update all documentation to describe this and mothball other
> >> >>> > methods of installation
> >> >>> >
> >> >>> > What would be nice:
> >> >>> >
> >> >>> > * Make available nightly builds of yt on several architectures
> using
> >> >>> > binstar
> >> >>> > * Utilize more of the packages included in conda elsewhere in yt,
> >> >>> > now
> >> >>> > that we can!
> >> >>> >
> >> >>> > Here's my current recipe for get_yt.sh:
> >> >>> >
> >> >>> > http://paste.yt-project.org/show/3843/
> >> >>> >
> >> >>> > (The config thing may get switched to include the --system
> argument,
> >> >>> > to modify the "yt-conda" condarc.)  The step that I'm most stuck
> on
> >> >>> > is
> >> >>> > getting the yt recipe to people.  If we want to make it possible
> and
> >> >>> > easy to build from source, we need to get the contents of a "conda
> >> >>> > recipe" to people.  They can then run "conda build ." in the
> >> >>> > directory.  Here are the recipes that we've been playing with:
> >> >>> >
> >> >>> > https://bitbucket.org/yt_analysis/yt_conda/src
> >> >>> >
> >> >>> > Basically, in get_yt.sh, to do from source instead of from binary
> we
> >> >>> > need to insert a step at the end that downloads the recipes
> somehow
> >> >>> > and then cd's into the right directory and builds them.  The
> reason
> >> >>> > this is tricky is that we often need to bootstrap ourselves; we
> >> >>> > can't
> >> >>> > assume anything exists.  We can download the .tar.bz2 of the
> current
> >> >>> > tip of the repo, but it includes the hash in the directory name
> that
> >> >>> > it extracts to.
> >> >>> >
> >> >>> > So I think what we need is a mechanism for getting the current
> state
> >> >>> > of the repo, figuring out the name of the repo's directory, moves
> >> >>> > into
> >> >>> > it, and then builds.  I believe that all/most of this becomes
> much,
> >> >>> > much easier if hg gets included in Anaconda, which Nathan has
> >> >>> > submitted a PR for.  So hopefully that will be taken care of, but
> >> >>> > until that time we can possibly figure something out.  I'm not
> sure
> >> >>> > that we have the resources to continually support binary nightly
> >> >>> > builds in perpetuity for all the architectures that people run on,
> >> >>> > so
> >> >>> > having source would be awesome.  Plus, one of the biggest appeals
> of
> >> >>> > how we distribute yt is that the source is included; I would very
> >> >>> > much
> >> >>> > not like to give this up.
> >> >>> >
> >> >>> > Thoughts?  Has anyone else tested any of this out?
> >> >>> >
> >> >>> > -Matt
> >> >>> >
> >> >>> > On Tue, Sep 3, 2013 at 1:04 PM, Britton Smith
> >> >>> > <brittonsmith at gmail.com>
> >> >>> > wrote:
> >> >>> >> Hi everyone,
> >> >>> >>
> >> >>> >> Sorry for chiming in late.  I just moved when this thread began
> and
> >> >>> >> do
> >> >>> >> not
> >> >>> >> have regular internet access.
> >> >>> >>
> >> >>> >> I really like this idea of conda, especially as a package manager
> >> >>> >> that
> >> >>> >> only
> >> >>> >> optionally makes its own edits to your .bashrc.  I have always
> >> >>> >> really
> >> >>> >> liked
> >> >>> >> that the install script creates a clean python stack with
> basically
> >> >>> >> everything a python user needs.  I have on occasion suggested it
> to
> >> >>> >> people
> >> >>> >> just looking to use numpy and matploblib.  It looks like conda
> will
> >> >>> >> continue
> >> >>> >> to provide this nice by-product, so I'm all for it.
> >> >>> >>
> >> >>> >> I won't be in a position to help with testing and such for
> another
> >> >>> >> week or
> >> >>> >> so when I get regular internet access, but I would be glad to do
> so
> >> >>> >> then.
> >> >>> >>
> >> >>> >> Britton
> >> >>> >>
> >> >>> >>
> >> >>> >> On Fri, Aug 30, 2013 at 9:29 PM, Nathan Goldbaum
> >> >>> >> <nathan12343 at gmail.com>
> >> >>> >> wrote:
> >> >>> >>>
> >> >>> >>> Everything should be available now for 64 bit linux and OS X.
> >> >>> >>>
> >> >>> >>>
> >> >>> >>> On Fri, Aug 30, 2013 at 10:52 AM, Chris Malone
> >> >>> >>> <chris.m.malone at gmail.com>
> >> >>> >>> wrote:
> >> >>> >>>>
> >> >>> >>>> Hi Nathan,
> >> >>> >>>>
> >> >>> >>>> That appears to work as it built the environment and `conda
> >> >>> >>>> install
> >> >>> >>>> ...`
> >> >>> >>>> added packages to my environment.
> >> >>> >>>>
> >> >>> >>>> One mistake I made was that I originally downloaded the
> "latest"
> >> >>> >>>> OS
> >> >>> >>>> X
> >> >>> >>>> build of Miniconda, but that happened to be Miniconda3, which
> is
> >> >>> >>>> python 3
> >> >>> >>>> based.  Trying to build the environment with that yields an
> error
> >> >>> >>>> regarding
> >> >>> >>>> incompatibility of yt and python3.3, as it should.
> >> >>> >>>>
> >> >>> >>>> Chris
> >> >>> >>>>
> >> >>> >>>>
> >> >>> >>>> On Fri, Aug 30, 2013 at 10:42 AM, Nathan Goldbaum
> >> >>> >>>> <nathan12343 at gmail.com>
> >> >>> >>>> wrote:
> >> >>> >>>>>
> >> >>> >>>>> Hey Chris,
> >> >>> >>>>>
> >> >>> >>>>> I don't think mercurial is strictly necessary, can you try
> again
> >> >>> >>>>> without
> >> >>> >>>>> it?  I think if Matt uploads a mercurial package for OS X this
> >> >>> >>>>> won't be an
> >> >>> >>>>> issue. I'll send him an updated tarball.
> >> >>> >>>>>
> >> >>> >>>>> I submitted a mercurial recipe to conda-recipes yesterday
> >> >>> >>>>> (https://github.com/ContinuumIO/conda-recipes/pull/14) so
> >> >>> >>>>> hopefully
> >> >>> >>>>> a
> >> >>> >>>>> mercurial build will be included in future anaconda releases.
> >> >>> >>>>>
> >> >>> >>>>>
> >> >>> >>>>> On Fri, Aug 30, 2013 at 10:39 AM, Chris Malone
> >> >>> >>>>> <chris.m.malone at gmail.com> wrote:
> >> >>> >>>>>>
> >> >>> >>>>>> I just tried setting this up on OS X 10.7.5 and failed when
> >> >>> >>>>>> attempting
> >> >>> >>>>>> to create the conda environment due to a missing mercurial
> >> >>> >>>>>> package:
> >> >>> >>>>>>
> >> >>> >>>>>> $ conda create -n ytenv -c
> http://conda.binstar.org/yt_project
> >> >>> >>>>>> yt
> >> >>> >>>>>> mercurial ipython tornado pyzmq pygments jinja2 sphinx
> >> >>> >>>>>> Error: No packages found matching: mercurial
> >> >>> >>>>>>
> >> >>> >>>>>>
> >> >>> >>>>>> On Thu, Aug 29, 2013 at 9:01 PM, Nathan Goldbaum
> >> >>> >>>>>> <nathan12343 at gmail.com> wrote:
> >> >>> >>>>>>>
> >> >>> >>>>>>> Yup, please try on OSX as well.  If you make sure Matt's
> >> >>> >>>>>>> binstar
> >> >>> >>>>>>> is in
> >> >>> >>>>>>> your .condarc, you should be able to get yt by doing 'conda
> >> >>> >>>>>>> install yt'.
> >> >>> >>>>>>>
> >> >>> >>>>>>> I built the OSX binary on my laptop so I'd appreciate
> hearing
> >> >>> >>>>>>> about
> >> >>> >>>>>>> issues, particularly if there are issues on older OS X
> >> >>> >>>>>>> releases.
> >> >>> >>>>>>>
> >> >>> >>>>>>> On Thursday, August 29, 2013, Matthew Turk wrote:
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Hi all,
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Thank you for the feedback -- I am glad there is some
> >> >>> >>>>>>>> agreement
> >> >>> >>>>>>>> about
> >> >>> >>>>>>>> possible ways forward, and so I'm happy to try to use this
> as
> >> >>> >>>>>>>> an
> >> >>> >>>>>>>> opportunity to explore simpler, more reliable methods than
> >> >>> >>>>>>>> the
> >> >>> >>>>>>>> install
> >> >>> >>>>>>>> script.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> This afternoon, I spent a bit of time with Conda, and I
> think
> >> >>> >>>>>>>> it's
> >> >>> >>>>>>>> quite nice.  There are a few rough corners, particularly
> >> >>> >>>>>>>> related
> >> >>> >>>>>>>> to
> >> >>> >>>>>>>> the binstar service, but it's so far pretty great.  With
> >> >>> >>>>>>>> Nathan's
> >> >>> >>>>>>>> help, I was able to upload a yt-2.5.5 package for linux
> >> >>> >>>>>>>> x86_64
> >> >>> >>>>>>>> and
> >> >>> >>>>>>>> then install it.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> The workflow that seems to work:
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> * Get miniconda:
> >> >>> >>>>>>>> http://repo.continuum.io/miniconda/index.html
> >> >>> >>>>>>>> * Run the installer for miniconda
> >> >>> >>>>>>>> * Enter the conda environment and then install yt by doing
> >> >>> >>>>>>>> "conda
> >> >>> >>>>>>>> install yt -c http://conda.binstar.org/yt_project/ ".
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> I think that this can likely all be stuck into a bash
> script.
> >> >>> >>>>>>>> A
> >> >>> >>>>>>>> simple, first pass at this is here:
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> http://paste.yt-project.org/show/3833/
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> This right now only works on Linux x86_64, but getting it
> to
> >> >>> >>>>>>>> work for
> >> >>> >>>>>>>> other machines won't be too hard.  I suspect we will be
> able
> >> >>> >>>>>>>> to
> >> >>> >>>>>>>> do
> >> >>> >>>>>>>> nightlies very easily as well.  If anyone out there has an
> >> >>> >>>>>>>> x86_64
> >> >>> >>>>>>>> machine they wouldn't mind trying it on, that would be very
> >> >>> >>>>>>>> helpful!
> >> >>> >>>>>>>> I did find that once I ran this script, I had to actually
> >> >>> >>>>>>>> prepend the
> >> >>> >>>>>>>> PATH afterwards as well.  This means doing:
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> export LD_LIBRARY_PATH=$HOME/yt-conda:$LD_LIBRARY_PATH
> >> >>> >>>>>>>> export PATH=$HOME/yt-conda:$PATH
> >> >>> >>>>>>>> source activate ytenv
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> At that point, everything was set up and working for me.
>  The
> >> >>> >>>>>>>> miniconda install offers to add paths to .bashrc, but I
> don't
> >> >>> >>>>>>>> think
> >> >>> >>>>>>>> we
> >> >>> >>>>>>>> should go down that route.  That being said, this is also a
> >> >>> >>>>>>>> possible
> >> >>> >>>>>>>> point of friction.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> One nice thing is that this also completely works with the
> >> >>> >>>>>>>> full
> >> >>> >>>>>>>> anaconda; if someone wants everything that is in the
> anaconda
> >> >>> >>>>>>>> install,
> >> >>> >>>>>>>> they can even simply do "conda install anaconda" from the
> >> >>> >>>>>>>> command
> >> >>> >>>>>>>> line
> >> >>> >>>>>>>> to get it.  But the stripped down subset is the default.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> If anyone has a chance to try this out and has feedback,
> I'd
> >> >>> >>>>>>>> greatly
> >> >>> >>>>>>>> appreciate it!  I think Nathan has done something very
> >> >>> >>>>>>>> similar
> >> >>> >>>>>>>> for
> >> >>> >>>>>>>> OSX.  I've also put a couple simple conda recipes here:
> >> >>> >>>>>>>> https://bitbucket.org/yt_analysis/yt_conda which we can
> use
> >> >>> >>>>>>>> as a
> >> >>> >>>>>>>> basis
> >> >>> >>>>>>>> for distributing builds and setting them up on buildbots
> and
> >> >>> >>>>>>>> the
> >> >>> >>>>>>>> like.
> >> >>> >>>>>>>> I'm pretty optimistic about this.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> -Matt
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> On Thu, Aug 29, 2013 at 5:50 PM, Nathan Goldbaum
> >> >>> >>>>>>>> <nathan12343 at gmail.com> wrote:
> >> >>> >>>>>>>>> I think to get everything working in a sustainable
> fashion,
> >> >>> >>>>>>>>> we
> >> >>> >>>>>>>>> would need
> >> >>> >>>>>>>>> buildbots for all platform combinations that we want to
> >> >>> >>>>>>>>> support, so
> >> >>> >>>>>>>>> all
> >> >>> >>>>>>>>> permutations of the (32/64 bit,  linux / OS X / windows,
> >> >>> >>>>>>>>> py27/py3.3) tuple.
> >> >>> >>>>>>>>> At the moment anaconda seems to support 32 and 64 bit
> linux,
> >> >>> >>>>>>>>> 64
> >> >>> >>>>>>>>> bit
> >> >>> >>>>>>>>> OS X
> >> >>> >>>>>>>>> (not totally clear if OS X version matters), and 32 and 64
> >> >>> >>>>>>>>> bit
> >> >>> >>>>>>>>> windows.
> >> >>> >>>>>>>>>
> >> >>> >>>>>>>>> Another option is to rely on conda build, which compiles
> >> >>> >>>>>>>>> everything
> >> >>> >>>>>>>>> from
> >> >>> >>>>>>>>> source.
> >> >>> >>>>>>>>>
> >> >>> >>>>>>>>>
> >> >>> >>>>>>>>> On Thu, Aug 29, 2013 at 2:45 PM, Stephen Skory <
> s at skory.us>
> >> >>> >>>>>>>>> wrote:
> >> >>> >>>>>>>>>>
> >> >>> >>>>>>>>>> Hi all,
> >> >>> >>>>>>>>>>
> >> >>> >>>>>>>>>> I have less of a skin in this than I used to, but I'd
> like
> >> >>> >>>>>>>>>> to
> >> >>> >>>>>>>>>> raise
> >> >>> >>>>>>>>>> the issue of Windows & package managers. For example,
> >> >>> >>>>>>>>>> Anaconda
> >> >>> >>>>>>>>>> is
> >> >>> >>>>>>>>>> available for Windows - would that mean that yt might
> "just
> >> >>> >>>>>>>>>> work"
> >> >>> >>>>>>>>>> on
> >> >>> >>>>>>>>>> Windows? Or the opposite, and it would require a great
> deal
> >> >>> >>>>>>>>>> of
> >> >>> >>>>>>>>>> effort
> >> >>> >>>>>>>>>> to get all the various things we expect to be .so's to
> work
> >> >>> >>>>>>>>>> as
> >> >>> >>>>>>>>>> .dll's
> >> >>> >>>>>>>>>> (such as the Cython helpers or halo-finding stuff)?
> >> >>> >>>>>>>>>>
> >> >>> >>>>>>>>>> I don't know the answers to these questions, but I think
> >> >>> >>>>>>>>>> it's
> >> >>> >>>>>>>>>> worth
> >> >>> >>>>>>>>>> thinking about.
> >> >>> >>>>>>>>>>
> >> >>> >>>>>>>>>> --
> >> >>> >>>>>>>>>> Stephen Skory
> >> >>> >>>>>>>>>> s at skory.us
> >> >>> >>>>>>>>>> http://stephenskory.com/
> >> >>> >>>>>>>>>> 510.621.3687 (google voice)
> >> >>> >>>>>>>>>> _______________________________________________
> >> >>> >>>>>>>>>> yt-dev mailing list
> >> >>> >>>>>>>>>> yt-dev at lists.spacepope.org
> >> >>> >>>>>>>>>>
> >> >>> >>>>>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>>>>>>>
> >> >>> >>>>>>>>>
> >> >>> >>>>>>>>>
> >> >>> >>>>>>>>> _______________________________________________
> >> >>> >>>>>>>>> yt-dev mailing list
> >> >>> >>>>>>>>> yt-dev at lists.spacepope.org
> >> >>> >>>>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>>>>>>>
> >> >>> >>>>>>>> _______________________________________________
> >> >>> >>>>>>>> yt-dev mailing list
> >> >>> >>>>>>>> yt-dev at lists.spacepope.org
> >> >>> >>>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>>>>>
> >> >>> >>>>>>>
> >> >>> >>>>>>> _______________________________________________
> >> >>> >>>>>>> yt-dev mailing list
> >> >>> >>>>>>> yt-dev at lists.spacepope.org
> >> >>> >>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>>>>>
> >> >>> >>>>>>
> >> >>> >>>>>>
> >> >>> >>>>>> _______________________________________________
> >> >>> >>>>>> yt-dev mailing list
> >> >>> >>>>>> yt-dev at lists.spacepope.org
> >> >>> >>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>>>>
> >> >>> >>>>>
> >> >>> >>>>>
> >> >>> >>>>> _______________________________________________
> >> >>> >>>>> yt-dev mailing list
> >> >>> >>>>> yt-dev at lists.spacepope.org
> >> >>> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>>>
> >> >>> >>>>
> >> >>> >>>>
> >> >>> >>>> _______________________________________________
> >> >>> >>>> yt-dev mailing list
> >> >>> >>>> yt-dev at lists.spacepope.org
> >> >>> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>>
> >> >>> >>>
> >> >>> >>>
> >> >>> >>> _______________________________________________
> >> >>> >>> yt-dev mailing list
> >> >>> >>> yt-dev at lists.spacepope.org
> >> >>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>>
> >> >>> >>
> >> >>> >>
> >> >>> >> _______________________________________________
> >> >>> >> yt-dev mailing list
> >> >>> >> yt-dev at lists.spacepope.org
> >> >>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >>
> >> >>> > _______________________________________________
> >> >>> > yt-dev mailing list
> >> >>> > yt-dev at lists.spacepope.org
> >> >>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>>
> >> >>> _______________________________________________
> >> >>> yt-dev mailing list
> >> >>> yt-dev at lists.spacepope.org
> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> yt-dev mailing list
> >> >> yt-dev at lists.spacepope.org
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> yt-dev mailing list
> >> >> yt-dev at lists.spacepope.org
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > yt-dev mailing list
> >> > yt-dev at lists.spacepope.org
> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130910/e010263d/attachment.html>

More information about the yt-dev mailing list