[yt-dev] Packaging.

Matthew Turk matthewturk at gmail.com
Tue Sep 10 11:10:44 PDT 2013


Hi Nathan,

Okay -- but if that's the price we pay, I think it may still be worth it.

-Matt

On Tue, Sep 10, 2013 at 2:09 PM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
> Not 100% sure.  It certainly won't work on OS X at the moment.
>
> -Nathan
>
>
> On Tue, Sep 10, 2013 at 10:59 AM, Matthew Turk <matthewturk at gmail.com>
> wrote:
>>
>> Hi Nathan,
>>
>> On Tue, Sep 10, 2013 at 1:16 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>> wrote:
>> > Hey John,
>> >
>> > A few questions:
>> >
>> > 1. What did you choose for INST_YT_SOURCE in get_yt.sh? If
>> > INST_YT_SOURCE=1,
>> > what happens when you try INST_YT_SOURCE=0?
>> >
>> > 2. Again, if you had INST_YT_SOURCE=1, when you manually run `setup.py
>> > install` in the yt source repository, it should print out a few lines
>> > showing the root directory for the PNG, Freetype, and HDF5 libraries it
>> > tries to link against.  This should be the very first thing printed out
>> > by
>> > the setup script.  Is it linking against anaconda's libraries or system
>> > libraries?
>> >
>> > 3. What's the output of `ldd
>> >
>> > /u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/lib/png_writer.so`?
>> >
>> > I'm beginning to suspect that we cannot link against the libraries
>> > provided
>> > by anaconda unless we tailor our build environment to match the sort of
>> > gymnastics that `conda build` does to sanitize the build environment.
>> > Just
>> > yesterday I was having no end of grief trying to link against anaconda's
>> > hdf5 library.  yt is able to link against this library inside a conda
>> > build
>> > environment, but when I try to link in my normal shell environment I run
>> > into issues - mostly because the OS X hdf5 library on anaconda was
>> > compiled
>> > on an OS X 10.5 machine.
>>
>> If I read you correctly, you are saying that, practically speaking,
>> this would mainly mean not using the yt-conda HDF5 for Enzo/FLASH/etc
>> outside of yt.  Right?
>>
>> >
>> >
>> > On Tue, Sep 10, 2013 at 9:54 AM, John ZuHone <jzuhone at gmail.com> wrote:
>> >>
>> >> Hi Nathan and all,
>> >>
>> >> Here's a little more info on that error you asked about regarding
>> >> glibc.
>> >> It came up again for me on Pleiades, this time with yt itself.
>> >>
>> >> Traceback (most recent call last):
>> >>   File "/u/jzuhone/anaconda/bin/yt", line 4, in <module>
>> >>     from yt.utilities.command_line import run_main
>> >>   File
>> >>
>> >> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/command_line.py",
>> >> line 29, in <module>
>> >>     from yt.mods import *
>> >>   File "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/mods.py",
>> >> line
>> >> 60, in <module>
>> >>     from yt.data_objects.api import \
>> >>   File
>> >>
>> >> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/api.py",
>> >> line 31, in <module>
>> >>     from grid_patch import \
>> >>   File
>> >>
>> >> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/grid_patch.py",
>> >> line 35, in <module>
>> >>     from yt.data_objects.data_containers import YTFieldData
>> >>   File
>> >>
>> >> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/data_containers.py",
>> >> line 45, in <module>
>> >>     from yt.data_objects.derived_quantities import GridChildMaskWrapper
>> >>   File
>> >>
>> >> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/data_objects/derived_quantities.py",
>> >> line 36, in <module>
>> >>     from yt.utilities.parallel_tools.parallel_analysis_interface import
>> >> \
>> >>   File
>> >>
>> >> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>> >> line 39, in <module>
>> >>     from yt.utilities.lib import \
>> >>   File
>> >>
>> >> "/u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/lib/__init__.py",
>> >> line 35, in <module>
>> >>     from .png_writer import *
>> >> ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required
>> >> by
>> >>
>> >> /u/jzuhone/anaconda/lib/python2.7/site-packages/yt/utilities/lib/png_writer.so)
>> >>
>> >> Best,
>> >>
>> >> John
>> >>
>> >> On Sep 3, 2013, at 3:10 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>> >> wrote:
>> >>
>> >> John, can you give a little more info on that error?  I'm a little
>> >> concerned that there are binary incompatibilities that conda can't
>> >> resolve.
>> >> Perhaps this is something we should report on the conda issue tracker.
>> >>
>> >> Matt, can't we get the recipes in the same way we get the latest dev
>> >> install script?  Something like:
>> >>
>> >> wget https://bitbucket.org/yt_analysis/yt_conda/raw/yt/meta.yaml
>> >>
>> >> This doesn't currently work, although it does work for the yt repo, I
>> >> think because we use named branches in that repository.
>> >>
>> >>
>> >> On Tue, Sep 3, 2013 at 12:00 PM, John ZuHone <jzuhone at gmail.com> wrote:
>> >>>
>> >>> Works fine for me on OS X x86_64.
>> >>>
>> >>> On my Goddard-controlled Linux x86_64 server, everything worked fine
>> >>> except Mercurial, which I "conda install"-ed using the yt link, but
>> >>> the
>> >>> binary had an incompatibility which the glibc that was installed on
>> >>> the
>> >>> machine. Using pip to install Mercurial (which did it from source) was
>> >>> the
>> >>> workaround.
>> >>>
>> >>> On Sep 3, 2013, at 2:54 PM, Matthew Turk <matthewturk at gmail.com>
>> >>> wrote:
>> >>>
>> >>> > Hi all,
>> >>> >
>> >>> > Thanks to everybody who has reported back on testing.  After some
>> >>> > talking both offline and on IRC, as well as here, I think we would
>> >>> > need to do the following things:
>> >>> >
>> >>> > * Make a single script that grabs the appropriate distribution of
>> >>> > miniconda and installs it.  Right now I have a mechanism for doing
>> >>> > this, but it's currently tied to an architecture.
>> >>> > * Create a mechanism for installing all the packages we need.
>> >>> > Nearly
>> >>> > all are available inside the Continuum repos.  What we're hung up on
>> >>> > is that source installs require a "recipe", and transmitting the
>> >>> > recipe is where I don't have an idea of what to do.
>> >>> > * Test this out lots of places
>> >>> > * Clean up the edges in the (new) install script
>> >>> > * Move the old install script to maintenance mode
>> >>> > * Update all documentation to describe this and mothball other
>> >>> > methods of installation
>> >>> >
>> >>> > What would be nice:
>> >>> >
>> >>> > * Make available nightly builds of yt on several architectures using
>> >>> > binstar
>> >>> > * Utilize more of the packages included in conda elsewhere in yt,
>> >>> > now
>> >>> > that we can!
>> >>> >
>> >>> > Here's my current recipe for get_yt.sh:
>> >>> >
>> >>> > http://paste.yt-project.org/show/3843/
>> >>> >
>> >>> > (The config thing may get switched to include the --system argument,
>> >>> > to modify the "yt-conda" condarc.)  The step that I'm most stuck on
>> >>> > is
>> >>> > getting the yt recipe to people.  If we want to make it possible and
>> >>> > easy to build from source, we need to get the contents of a "conda
>> >>> > recipe" to people.  They can then run "conda build ." in the
>> >>> > directory.  Here are the recipes that we've been playing with:
>> >>> >
>> >>> > https://bitbucket.org/yt_analysis/yt_conda/src
>> >>> >
>> >>> > Basically, in get_yt.sh, to do from source instead of from binary we
>> >>> > need to insert a step at the end that downloads the recipes somehow
>> >>> > and then cd's into the right directory and builds them.  The reason
>> >>> > this is tricky is that we often need to bootstrap ourselves; we
>> >>> > can't
>> >>> > assume anything exists.  We can download the .tar.bz2 of the current
>> >>> > tip of the repo, but it includes the hash in the directory name that
>> >>> > it extracts to.
>> >>> >
>> >>> > So I think what we need is a mechanism for getting the current state
>> >>> > of the repo, figuring out the name of the repo's directory, moves
>> >>> > into
>> >>> > it, and then builds.  I believe that all/most of this becomes much,
>> >>> > much easier if hg gets included in Anaconda, which Nathan has
>> >>> > submitted a PR for.  So hopefully that will be taken care of, but
>> >>> > until that time we can possibly figure something out.  I'm not sure
>> >>> > that we have the resources to continually support binary nightly
>> >>> > builds in perpetuity for all the architectures that people run on,
>> >>> > so
>> >>> > having source would be awesome.  Plus, one of the biggest appeals of
>> >>> > how we distribute yt is that the source is included; I would very
>> >>> > much
>> >>> > not like to give this up.
>> >>> >
>> >>> > Thoughts?  Has anyone else tested any of this out?
>> >>> >
>> >>> > -Matt
>> >>> >
>> >>> > On Tue, Sep 3, 2013 at 1:04 PM, Britton Smith
>> >>> > <brittonsmith at gmail.com>
>> >>> > wrote:
>> >>> >> Hi everyone,
>> >>> >>
>> >>> >> Sorry for chiming in late.  I just moved when this thread began and
>> >>> >> do
>> >>> >> not
>> >>> >> have regular internet access.
>> >>> >>
>> >>> >> I really like this idea of conda, especially as a package manager
>> >>> >> that
>> >>> >> only
>> >>> >> optionally makes its own edits to your .bashrc.  I have always
>> >>> >> really
>> >>> >> liked
>> >>> >> that the install script creates a clean python stack with basically
>> >>> >> everything a python user needs.  I have on occasion suggested it to
>> >>> >> people
>> >>> >> just looking to use numpy and matploblib.  It looks like conda will
>> >>> >> continue
>> >>> >> to provide this nice by-product, so I'm all for it.
>> >>> >>
>> >>> >> I won't be in a position to help with testing and such for another
>> >>> >> week or
>> >>> >> so when I get regular internet access, but I would be glad to do so
>> >>> >> then.
>> >>> >>
>> >>> >> Britton
>> >>> >>
>> >>> >>
>> >>> >> On Fri, Aug 30, 2013 at 9:29 PM, Nathan Goldbaum
>> >>> >> <nathan12343 at gmail.com>
>> >>> >> wrote:
>> >>> >>>
>> >>> >>> Everything should be available now for 64 bit linux and OS X.
>> >>> >>>
>> >>> >>>
>> >>> >>> On Fri, Aug 30, 2013 at 10:52 AM, Chris Malone
>> >>> >>> <chris.m.malone at gmail.com>
>> >>> >>> wrote:
>> >>> >>>>
>> >>> >>>> Hi Nathan,
>> >>> >>>>
>> >>> >>>> That appears to work as it built the environment and `conda
>> >>> >>>> install
>> >>> >>>> ...`
>> >>> >>>> added packages to my environment.
>> >>> >>>>
>> >>> >>>> One mistake I made was that I originally downloaded the "latest"
>> >>> >>>> OS
>> >>> >>>> X
>> >>> >>>> build of Miniconda, but that happened to be Miniconda3, which is
>> >>> >>>> python 3
>> >>> >>>> based.  Trying to build the environment with that yields an error
>> >>> >>>> regarding
>> >>> >>>> incompatibility of yt and python3.3, as it should.
>> >>> >>>>
>> >>> >>>> Chris
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> On Fri, Aug 30, 2013 at 10:42 AM, Nathan Goldbaum
>> >>> >>>> <nathan12343 at gmail.com>
>> >>> >>>> wrote:
>> >>> >>>>>
>> >>> >>>>> Hey Chris,
>> >>> >>>>>
>> >>> >>>>> I don't think mercurial is strictly necessary, can you try again
>> >>> >>>>> without
>> >>> >>>>> it?  I think if Matt uploads a mercurial package for OS X this
>> >>> >>>>> won't be an
>> >>> >>>>> issue. I'll send him an updated tarball.
>> >>> >>>>>
>> >>> >>>>> I submitted a mercurial recipe to conda-recipes yesterday
>> >>> >>>>> (https://github.com/ContinuumIO/conda-recipes/pull/14) so
>> >>> >>>>> hopefully
>> >>> >>>>> a
>> >>> >>>>> mercurial build will be included in future anaconda releases.
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> On Fri, Aug 30, 2013 at 10:39 AM, Chris Malone
>> >>> >>>>> <chris.m.malone at gmail.com> wrote:
>> >>> >>>>>>
>> >>> >>>>>> I just tried setting this up on OS X 10.7.5 and failed when
>> >>> >>>>>> attempting
>> >>> >>>>>> to create the conda environment due to a missing mercurial
>> >>> >>>>>> package:
>> >>> >>>>>>
>> >>> >>>>>> $ conda create -n ytenv -c http://conda.binstar.org/yt_project
>> >>> >>>>>> yt
>> >>> >>>>>> mercurial ipython tornado pyzmq pygments jinja2 sphinx
>> >>> >>>>>> Error: No packages found matching: mercurial
>> >>> >>>>>>
>> >>> >>>>>>
>> >>> >>>>>> On Thu, Aug 29, 2013 at 9:01 PM, Nathan Goldbaum
>> >>> >>>>>> <nathan12343 at gmail.com> wrote:
>> >>> >>>>>>>
>> >>> >>>>>>> Yup, please try on OSX as well.  If you make sure Matt's
>> >>> >>>>>>> binstar
>> >>> >>>>>>> is in
>> >>> >>>>>>> your .condarc, you should be able to get yt by doing 'conda
>> >>> >>>>>>> install yt'.
>> >>> >>>>>>>
>> >>> >>>>>>> I built the OSX binary on my laptop so I'd appreciate hearing
>> >>> >>>>>>> about
>> >>> >>>>>>> issues, particularly if there are issues on older OS X
>> >>> >>>>>>> releases.
>> >>> >>>>>>>
>> >>> >>>>>>> On Thursday, August 29, 2013, Matthew Turk wrote:
>> >>> >>>>>>>>
>> >>> >>>>>>>> Hi all,
>> >>> >>>>>>>>
>> >>> >>>>>>>> Thank you for the feedback -- I am glad there is some
>> >>> >>>>>>>> agreement
>> >>> >>>>>>>> about
>> >>> >>>>>>>> possible ways forward, and so I'm happy to try to use this as
>> >>> >>>>>>>> an
>> >>> >>>>>>>> opportunity to explore simpler, more reliable methods than
>> >>> >>>>>>>> the
>> >>> >>>>>>>> install
>> >>> >>>>>>>> script.
>> >>> >>>>>>>>
>> >>> >>>>>>>> This afternoon, I spent a bit of time with Conda, and I think
>> >>> >>>>>>>> it's
>> >>> >>>>>>>> quite nice.  There are a few rough corners, particularly
>> >>> >>>>>>>> related
>> >>> >>>>>>>> to
>> >>> >>>>>>>> the binstar service, but it's so far pretty great.  With
>> >>> >>>>>>>> Nathan's
>> >>> >>>>>>>> help, I was able to upload a yt-2.5.5 package for linux
>> >>> >>>>>>>> x86_64
>> >>> >>>>>>>> and
>> >>> >>>>>>>> then install it.
>> >>> >>>>>>>>
>> >>> >>>>>>>> The workflow that seems to work:
>> >>> >>>>>>>>
>> >>> >>>>>>>> * Get miniconda:
>> >>> >>>>>>>> http://repo.continuum.io/miniconda/index.html
>> >>> >>>>>>>> * Run the installer for miniconda
>> >>> >>>>>>>> * Enter the conda environment and then install yt by doing
>> >>> >>>>>>>> "conda
>> >>> >>>>>>>> install yt -c http://conda.binstar.org/yt_project/ ".
>> >>> >>>>>>>>
>> >>> >>>>>>>> I think that this can likely all be stuck into a bash script.
>> >>> >>>>>>>> A
>> >>> >>>>>>>> simple, first pass at this is here:
>> >>> >>>>>>>>
>> >>> >>>>>>>> http://paste.yt-project.org/show/3833/
>> >>> >>>>>>>>
>> >>> >>>>>>>> This right now only works on Linux x86_64, but getting it to
>> >>> >>>>>>>> work for
>> >>> >>>>>>>> other machines won't be too hard.  I suspect we will be able
>> >>> >>>>>>>> to
>> >>> >>>>>>>> do
>> >>> >>>>>>>> nightlies very easily as well.  If anyone out there has an
>> >>> >>>>>>>> x86_64
>> >>> >>>>>>>> machine they wouldn't mind trying it on, that would be very
>> >>> >>>>>>>> helpful!
>> >>> >>>>>>>> I did find that once I ran this script, I had to actually
>> >>> >>>>>>>> prepend the
>> >>> >>>>>>>> PATH afterwards as well.  This means doing:
>> >>> >>>>>>>>
>> >>> >>>>>>>> export LD_LIBRARY_PATH=$HOME/yt-conda:$LD_LIBRARY_PATH
>> >>> >>>>>>>> export PATH=$HOME/yt-conda:$PATH
>> >>> >>>>>>>> source activate ytenv
>> >>> >>>>>>>>
>> >>> >>>>>>>> At that point, everything was set up and working for me.  The
>> >>> >>>>>>>> miniconda install offers to add paths to .bashrc, but I don't
>> >>> >>>>>>>> think
>> >>> >>>>>>>> we
>> >>> >>>>>>>> should go down that route.  That being said, this is also a
>> >>> >>>>>>>> possible
>> >>> >>>>>>>> point of friction.
>> >>> >>>>>>>>
>> >>> >>>>>>>> One nice thing is that this also completely works with the
>> >>> >>>>>>>> full
>> >>> >>>>>>>> anaconda; if someone wants everything that is in the anaconda
>> >>> >>>>>>>> install,
>> >>> >>>>>>>> they can even simply do "conda install anaconda" from the
>> >>> >>>>>>>> command
>> >>> >>>>>>>> line
>> >>> >>>>>>>> to get it.  But the stripped down subset is the default.
>> >>> >>>>>>>>
>> >>> >>>>>>>> If anyone has a chance to try this out and has feedback, I'd
>> >>> >>>>>>>> greatly
>> >>> >>>>>>>> appreciate it!  I think Nathan has done something very
>> >>> >>>>>>>> similar
>> >>> >>>>>>>> for
>> >>> >>>>>>>> OSX.  I've also put a couple simple conda recipes here:
>> >>> >>>>>>>> https://bitbucket.org/yt_analysis/yt_conda which we can use
>> >>> >>>>>>>> as a
>> >>> >>>>>>>> basis
>> >>> >>>>>>>> for distributing builds and setting them up on buildbots and
>> >>> >>>>>>>> the
>> >>> >>>>>>>> like.
>> >>> >>>>>>>> I'm pretty optimistic about this.
>> >>> >>>>>>>>
>> >>> >>>>>>>> -Matt
>> >>> >>>>>>>>
>> >>> >>>>>>>> On Thu, Aug 29, 2013 at 5:50 PM, Nathan Goldbaum
>> >>> >>>>>>>> <nathan12343 at gmail.com> wrote:
>> >>> >>>>>>>>> I think to get everything working in a sustainable fashion,
>> >>> >>>>>>>>> we
>> >>> >>>>>>>>> would need
>> >>> >>>>>>>>> buildbots for all platform combinations that we want to
>> >>> >>>>>>>>> support, so
>> >>> >>>>>>>>> all
>> >>> >>>>>>>>> permutations of the (32/64 bit,  linux / OS X / windows,
>> >>> >>>>>>>>> py27/py3.3) tuple.
>> >>> >>>>>>>>> At the moment anaconda seems to support 32 and 64 bit linux,
>> >>> >>>>>>>>> 64
>> >>> >>>>>>>>> bit
>> >>> >>>>>>>>> OS X
>> >>> >>>>>>>>> (not totally clear if OS X version matters), and 32 and 64
>> >>> >>>>>>>>> bit
>> >>> >>>>>>>>> windows.
>> >>> >>>>>>>>>
>> >>> >>>>>>>>> Another option is to rely on conda build, which compiles
>> >>> >>>>>>>>> everything
>> >>> >>>>>>>>> from
>> >>> >>>>>>>>> source.
>> >>> >>>>>>>>>
>> >>> >>>>>>>>>
>> >>> >>>>>>>>> On Thu, Aug 29, 2013 at 2:45 PM, Stephen Skory <s at skory.us>
>> >>> >>>>>>>>> wrote:
>> >>> >>>>>>>>>>
>> >>> >>>>>>>>>> Hi all,
>> >>> >>>>>>>>>>
>> >>> >>>>>>>>>> I have less of a skin in this than I used to, but I'd like
>> >>> >>>>>>>>>> to
>> >>> >>>>>>>>>> raise
>> >>> >>>>>>>>>> the issue of Windows & package managers. For example,
>> >>> >>>>>>>>>> Anaconda
>> >>> >>>>>>>>>> is
>> >>> >>>>>>>>>> available for Windows - would that mean that yt might "just
>> >>> >>>>>>>>>> work"
>> >>> >>>>>>>>>> on
>> >>> >>>>>>>>>> Windows? Or the opposite, and it would require a great deal
>> >>> >>>>>>>>>> of
>> >>> >>>>>>>>>> effort
>> >>> >>>>>>>>>> to get all the various things we expect to be .so's to work
>> >>> >>>>>>>>>> as
>> >>> >>>>>>>>>> .dll's
>> >>> >>>>>>>>>> (such as the Cython helpers or halo-finding stuff)?
>> >>> >>>>>>>>>>
>> >>> >>>>>>>>>> I don't know the answers to these questions, but I think
>> >>> >>>>>>>>>> it's
>> >>> >>>>>>>>>> worth
>> >>> >>>>>>>>>> thinking about.
>> >>> >>>>>>>>>>
>> >>> >>>>>>>>>> --
>> >>> >>>>>>>>>> Stephen Skory
>> >>> >>>>>>>>>> s at skory.us
>> >>> >>>>>>>>>> http://stephenskory.com/
>> >>> >>>>>>>>>> 510.621.3687 (google voice)
>> >>> >>>>>>>>>> _______________________________________________
>> >>> >>>>>>>>>> yt-dev mailing list
>> >>> >>>>>>>>>> yt-dev at lists.spacepope.org
>> >>> >>>>>>>>>>
>> >>> >>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>>>>>>>
>> >>> >>>>>>>>>
>> >>> >>>>>>>>>
>> >>> >>>>>>>>> _______________________________________________
>> >>> >>>>>>>>> yt-dev mailing list
>> >>> >>>>>>>>> yt-dev at lists.spacepope.org
>> >>> >>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>>>>>>>
>> >>> >>>>>>>> _______________________________________________
>> >>> >>>>>>>> yt-dev mailing list
>> >>> >>>>>>>> yt-dev at lists.spacepope.org
>> >>> >>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>>>>>
>> >>> >>>>>>>
>> >>> >>>>>>> _______________________________________________
>> >>> >>>>>>> yt-dev mailing list
>> >>> >>>>>>> yt-dev at lists.spacepope.org
>> >>> >>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>>>>>
>> >>> >>>>>>
>> >>> >>>>>>
>> >>> >>>>>> _______________________________________________
>> >>> >>>>>> yt-dev mailing list
>> >>> >>>>>> yt-dev at lists.spacepope.org
>> >>> >>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> _______________________________________________
>> >>> >>>>> yt-dev mailing list
>> >>> >>>>> yt-dev at lists.spacepope.org
>> >>> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> _______________________________________________
>> >>> >>>> yt-dev mailing list
>> >>> >>>> yt-dev at lists.spacepope.org
>> >>> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>>
>> >>> >>>
>> >>> >>>
>> >>> >>> _______________________________________________
>> >>> >>> yt-dev mailing list
>> >>> >>> yt-dev at lists.spacepope.org
>> >>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >> _______________________________________________
>> >>> >> yt-dev mailing list
>> >>> >> yt-dev at lists.spacepope.org
>> >>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >>
>> >>> > _______________________________________________
>> >>> > yt-dev mailing list
>> >>> > yt-dev at lists.spacepope.org
>> >>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>



More information about the yt-dev mailing list