[yt-dev] Packaging.

John ZuHone jzuhone at gmail.com
Tue Sep 10 09:33:17 PDT 2013


I tried installing this script on Pleiades but had the same problem as Matt, with the requirement of the chrpath package. Of course, on Pleiades we don't have the option of installing it ourselves. Thoughts, besides asking them to install it? I guess that might be the only option. 

On Sep 7, 2013, at 1:27 PM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:

> The yt-dev recipe I created would only be good for nightly binary builds.  I think that would only be useful for people who want the latest and greatest but don't want to set up a build environment, which I suspect is a relatively small group of people.  We'll have to decide at some other point if we want to have buildbots and nightly builds.
> 
> Agreed about updating our documentation and website.  I think we should supply both the script and instructions for setting up yt using `pip` and already-installed conda packages.  We should also link to a page that lists the packages needed to manually set up yt.
> 
> 
> On Sat, Sep 7, 2013 at 8:22 AM, Matthew Turk <matthewturk at gmail.com> wrote:
> Hi all,
> 
> I've issued a PR with my first pass at the get_yt.sh script.  I've
> ported over a bunch of functionality from the old install_script.sh
> and it works for me in a clean install on Ubuntu, although it does
> require the "chrpath" package which I oddly did not have.
> 
> https://bitbucket.org/yt_analysis/yt/pull-request/591/first-draft-of-a-new-get_yt-script-based/diff
> 
> What would be really helpful is if a few people could try it out, make
> modifications, commit, push to their fork, and I will then pull in
> those changes until it does everything it needs to.  I think what I
> would like to preserve is that the "get_yt.sh" script should *not* be
> about how to install Conda/Anaconda -- it should get yt installed, and
> the yt environment.  The case of installing yt *into* Conda is what
> I'd like to cover by providing binary packages, and I suspect that our
> audience for that will grow with time.  This script aims along a
> slightly different vector.
> 
> I'm still not entirely sure how to handle source distributions; even
> with the yt-dev recipe Nathan has, it's still building binary
> packages.  I've done my best to include the source as-is in the
> script, but to also use the yt recipe that Nathan has worked on.  I'm
> not sure how the "yt-dev" recipe fits into that.  Kacper, Nathan, any
> ideas about that?  Would it just be the basic nightly package builder?
>  Then we could have it get rebuild and reuploaded all the time; this
> would help with keeping people up to date, but *not* with encouraging
> contributions.
> 
> -Matt
> 
> PS One last thing is that our whole "get yt" documentation should be
> re-done.  We should be much more clear about how and where to get the
> package, and reduce the number of options.
> 
> 
> On Fri, Sep 6, 2013 at 9:08 AM, Matthew Turk <matthewturk at gmail.com> wrote:
> > Hi Nathan,
> >
> > On Wed, Sep 4, 2013 at 2:54 AM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
> >> Here's an updated version of the shell script Matt sent out earlier today
> >> that builds yt and then installs it based on the yt-conda recipe:
> >> http://paste.yt-project.org/show/3848/
> >>
> >> I like this a lot and think it can be improved in several simple ways, most
> >> of which can be readily scraped from the install script:
> >
> > Me too.  This is much improved over mine.
> >
> >>
> >>     * Install in the folder that the script was invoked in rather than $HOME
> >
> > Good idea.
> >
> >>     * Add an option to install anaconda with a warning about needing a
> >> couple hundred megabytes of space
> >
> > Excellent!  I like this a lot.  And, we should point to the Anaconda
> > page for more info about packages and how Anaconda works.
> >
> >>     * Install the miniconda package appropriate for the platform
> >>     * Add warnings on various platforms regarding needed packages and
> >> compilers (i.e. still need build-essentials on ubuntu and Xcode on OSX).
> >>     * Add a warning at the end about modifying .bashrc for PATH and
> >> LD_LIBRARY_PATH or supply an activate script that sets them for us.
> >
> > I think I'd prefer to supply an activate script that both activates
> > Conda *and* the yt environment in it.  That way individuals who want
> > to use Conda directly can do so, but we also make it possible to keep
> > the behavior that people have already.
> >
> >>     * Clone yt-doc, yt-supplemental and the main yt repo and put them in the
> >> yt-conda directory.
> >
> > Agreed.
> >
> >>     * Check if conda is already available and (optionally?) skip the
> >> miniconda step.
> >
> > I'm not sure about this, since I think we will then just tell people
> > to install their own yt.  I think the install script should not be
> > *too* complex in this regard.
> >
> >>     * Ensure the user that yt loves them.
> >
> > Yes.
> >
> >>
> >> As Matt suggested on a Conda github issue
> >> (https://github.com/ContinuumIO/conda/issues/176#issuecomment-23753101) it
> >> would be nice to first try to conda install yt and then if that fails use
> >> conda build.  I think it should be possible to set this up as part of the
> >> bash script.  As asmurer notes in the issue, general support for this sort
> >> of mixed binary/source distribution in conda might be difficult but with yt
> >> it works since our setup script is platform agnostic so long as the
> >> compilation environment is set up correctly and we don't even try to work
> >> correctly on windows.
> >>
> >> So long as we provide binary yt packages of (at least) the stable releases,
> >> this should obviate the need for proper compiler setup, which avoid a
> >> significant source of pain for OS X users as well as linux users on
> >> platforms like ubuntu that don't provide sane build environments out of the
> >> box.
> >>
> >> -Nathan
> >
> > Awesome work, and I am getting more excited about this as time goes
> > on.  I am going to try my hand at updating the new script in the way
> > you mentioned, and then I will try checking it into the repo and issue
> > a PR so we can iterate there.  I hope to get to this by tomorrow
> > morning.
> >
> > -Matt
> >
> >>
> >>
> >> On Tue, Sep 3, 2013 at 12:28 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> >>>
> >>> Hi Nathan,
> >>>
> >>> On Tue, Sep 3, 2013 at 3:10 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> >>> wrote:
> >>> > John, can you give a little more info on that error?  I'm a little
> >>> > concerned
> >>> > that there are binary incompatibilities that conda can't resolve.
> >>> > Perhaps
> >>> > this is something we should report on the conda issue tracker.
> >>> >
> >>> > Matt, can't we get the recipes in the same way we get the latest dev
> >>> > install
> >>> > script?  Something like:
> >>> >
> >>> > wget https://bitbucket.org/yt_analysis/yt_conda/raw/yt/meta.yaml
> >>> >
> >>> > This doesn't currently work, although it does work for the yt repo, I
> >>> > think
> >>> > because we use named branches in that repository.
> >>>
> >>> Yup -- you can do it like this:
> >>>
> >>> wget https://bitbucket.org/yt_analysis/yt_conda/raw/default/yt/meta.yaml
> >>>
> >>> ...although for me right now it's failing from a bad BB certificate.
> >>>
> >>> The only problem with "pip install" is that it no longer lists pip
> >>> installations inside the "conda list" or "conda update" outputs.  This
> >>> has been discussed recently on the mailing list for Anaconda:
> >>>
> >>>
> >>> https://groups.google.com/a/continuum.io/forum/?fromgroups#!topic/anaconda/wr95VN7ezpo
> >>>
> >>> ...but there has not yet been a resolution.
> >>>
> >>> -Matt
> >>>
> >>> >
> >>> >
> >>> > On Tue, Sep 3, 2013 at 12:00 PM, John ZuHone <jzuhone at gmail.com> wrote:
> >>> >>
> >>> >> Works fine for me on OS X x86_64.
> >>> >>
> >>> >> On my Goddard-controlled Linux x86_64 server, everything worked fine
> >>> >> except Mercurial, which I "conda install"-ed using the yt link, but the
> >>> >> binary had an incompatibility which the glibc that was installed on the
> >>> >> machine. Using pip to install Mercurial (which did it from source) was
> >>> >> the
> >>> >> workaround.
> >>> >>
> >>> >> On Sep 3, 2013, at 2:54 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> >>> >>
> >>> >> > Hi all,
> >>> >> >
> >>> >> > Thanks to everybody who has reported back on testing.  After some
> >>> >> > talking both offline and on IRC, as well as here, I think we would
> >>> >> > need to do the following things:
> >>> >> >
> >>> >> > * Make a single script that grabs the appropriate distribution of
> >>> >> > miniconda and installs it.  Right now I have a mechanism for doing
> >>> >> > this, but it's currently tied to an architecture.
> >>> >> > * Create a mechanism for installing all the packages we need.  Nearly
> >>> >> > all are available inside the Continuum repos.  What we're hung up on
> >>> >> > is that source installs require a "recipe", and transmitting the
> >>> >> > recipe is where I don't have an idea of what to do.
> >>> >> > * Test this out lots of places
> >>> >> > * Clean up the edges in the (new) install script
> >>> >> > * Move the old install script to maintenance mode
> >>> >> > * Update all documentation to describe this and mothball other
> >>> >> > methods of installation
> >>> >> >
> >>> >> > What would be nice:
> >>> >> >
> >>> >> > * Make available nightly builds of yt on several architectures using
> >>> >> > binstar
> >>> >> > * Utilize more of the packages included in conda elsewhere in yt, now
> >>> >> > that we can!
> >>> >> >
> >>> >> > Here's my current recipe for get_yt.sh:
> >>> >> >
> >>> >> > http://paste.yt-project.org/show/3843/
> >>> >> >
> >>> >> > (The config thing may get switched to include the --system argument,
> >>> >> > to modify the "yt-conda" condarc.)  The step that I'm most stuck on
> >>> >> > is
> >>> >> > getting the yt recipe to people.  If we want to make it possible and
> >>> >> > easy to build from source, we need to get the contents of a "conda
> >>> >> > recipe" to people.  They can then run "conda build ." in the
> >>> >> > directory.  Here are the recipes that we've been playing with:
> >>> >> >
> >>> >> > https://bitbucket.org/yt_analysis/yt_conda/src
> >>> >> >
> >>> >> > Basically, in get_yt.sh, to do from source instead of from binary we
> >>> >> > need to insert a step at the end that downloads the recipes somehow
> >>> >> > and then cd's into the right directory and builds them.  The reason
> >>> >> > this is tricky is that we often need to bootstrap ourselves; we can't
> >>> >> > assume anything exists.  We can download the .tar.bz2 of the current
> >>> >> > tip of the repo, but it includes the hash in the directory name that
> >>> >> > it extracts to.
> >>> >> >
> >>> >> > So I think what we need is a mechanism for getting the current state
> >>> >> > of the repo, figuring out the name of the repo's directory, moves
> >>> >> > into
> >>> >> > it, and then builds.  I believe that all/most of this becomes much,
> >>> >> > much easier if hg gets included in Anaconda, which Nathan has
> >>> >> > submitted a PR for.  So hopefully that will be taken care of, but
> >>> >> > until that time we can possibly figure something out.  I'm not sure
> >>> >> > that we have the resources to continually support binary nightly
> >>> >> > builds in perpetuity for all the architectures that people run on, so
> >>> >> > having source would be awesome.  Plus, one of the biggest appeals of
> >>> >> > how we distribute yt is that the source is included; I would very
> >>> >> > much
> >>> >> > not like to give this up.
> >>> >> >
> >>> >> > Thoughts?  Has anyone else tested any of this out?
> >>> >> >
> >>> >> > -Matt
> >>> >> >
> >>> >> > On Tue, Sep 3, 2013 at 1:04 PM, Britton Smith
> >>> >> > <brittonsmith at gmail.com>
> >>> >> > wrote:
> >>> >> >> Hi everyone,
> >>> >> >>
> >>> >> >> Sorry for chiming in late.  I just moved when this thread began and
> >>> >> >> do
> >>> >> >> not
> >>> >> >> have regular internet access.
> >>> >> >>
> >>> >> >> I really like this idea of conda, especially as a package manager
> >>> >> >> that
> >>> >> >> only
> >>> >> >> optionally makes its own edits to your .bashrc.  I have always
> >>> >> >> really
> >>> >> >> liked
> >>> >> >> that the install script creates a clean python stack with basically
> >>> >> >> everything a python user needs.  I have on occasion suggested it to
> >>> >> >> people
> >>> >> >> just looking to use numpy and matploblib.  It looks like conda will
> >>> >> >> continue
> >>> >> >> to provide this nice by-product, so I'm all for it.
> >>> >> >>
> >>> >> >> I won't be in a position to help with testing and such for another
> >>> >> >> week
> >>> >> >> or
> >>> >> >> so when I get regular internet access, but I would be glad to do so
> >>> >> >> then.
> >>> >> >>
> >>> >> >> Britton
> >>> >> >>
> >>> >> >>
> >>> >> >> On Fri, Aug 30, 2013 at 9:29 PM, Nathan Goldbaum
> >>> >> >> <nathan12343 at gmail.com>
> >>> >> >> wrote:
> >>> >> >>>
> >>> >> >>> Everything should be available now for 64 bit linux and OS X.
> >>> >> >>>
> >>> >> >>>
> >>> >> >>> On Fri, Aug 30, 2013 at 10:52 AM, Chris Malone
> >>> >> >>> <chris.m.malone at gmail.com>
> >>> >> >>> wrote:
> >>> >> >>>>
> >>> >> >>>> Hi Nathan,
> >>> >> >>>>
> >>> >> >>>> That appears to work as it built the environment and `conda
> >>> >> >>>> install
> >>> >> >>>> ...`
> >>> >> >>>> added packages to my environment.
> >>> >> >>>>
> >>> >> >>>> One mistake I made was that I originally downloaded the "latest"
> >>> >> >>>> OS X
> >>> >> >>>> build of Miniconda, but that happened to be Miniconda3, which is
> >>> >> >>>> python 3
> >>> >> >>>> based.  Trying to build the environment with that yields an error
> >>> >> >>>> regarding
> >>> >> >>>> incompatibility of yt and python3.3, as it should.
> >>> >> >>>>
> >>> >> >>>> Chris
> >>> >> >>>>
> >>> >> >>>>
> >>> >> >>>> On Fri, Aug 30, 2013 at 10:42 AM, Nathan Goldbaum
> >>> >> >>>> <nathan12343 at gmail.com>
> >>> >> >>>> wrote:
> >>> >> >>>>>
> >>> >> >>>>> Hey Chris,
> >>> >> >>>>>
> >>> >> >>>>> I don't think mercurial is strictly necessary, can you try again
> >>> >> >>>>> without
> >>> >> >>>>> it?  I think if Matt uploads a mercurial package for OS X this
> >>> >> >>>>> won't
> >>> >> >>>>> be an
> >>> >> >>>>> issue. I'll send him an updated tarball.
> >>> >> >>>>>
> >>> >> >>>>> I submitted a mercurial recipe to conda-recipes yesterday
> >>> >> >>>>> (https://github.com/ContinuumIO/conda-recipes/pull/14) so
> >>> >> >>>>> hopefully
> >>> >> >>>>> a
> >>> >> >>>>> mercurial build will be included in future anaconda releases.
> >>> >> >>>>>
> >>> >> >>>>>
> >>> >> >>>>> On Fri, Aug 30, 2013 at 10:39 AM, Chris Malone
> >>> >> >>>>> <chris.m.malone at gmail.com> wrote:
> >>> >> >>>>>>
> >>> >> >>>>>> I just tried setting this up on OS X 10.7.5 and failed when
> >>> >> >>>>>> attempting
> >>> >> >>>>>> to create the conda environment due to a missing mercurial
> >>> >> >>>>>> package:
> >>> >> >>>>>>
> >>> >> >>>>>> $ conda create -n ytenv -c http://conda.binstar.org/yt_project
> >>> >> >>>>>> yt
> >>> >> >>>>>> mercurial ipython tornado pyzmq pygments jinja2 sphinx
> >>> >> >>>>>> Error: No packages found matching: mercurial
> >>> >> >>>>>>
> >>> >> >>>>>>
> >>> >> >>>>>> On Thu, Aug 29, 2013 at 9:01 PM, Nathan Goldbaum
> >>> >> >>>>>> <nathan12343 at gmail.com> wrote:
> >>> >> >>>>>>>
> >>> >> >>>>>>> Yup, please try on OSX as well.  If you make sure Matt's
> >>> >> >>>>>>> binstar
> >>> >> >>>>>>> is in
> >>> >> >>>>>>> your .condarc, you should be able to get yt by doing 'conda
> >>> >> >>>>>>> install yt'.
> >>> >> >>>>>>>
> >>> >> >>>>>>> I built the OSX binary on my laptop so I'd appreciate hearing
> >>> >> >>>>>>> about
> >>> >> >>>>>>> issues, particularly if there are issues on older OS X
> >>> >> >>>>>>> releases.
> >>> >> >>>>>>>
> >>> >> >>>>>>> On Thursday, August 29, 2013, Matthew Turk wrote:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> Hi all,
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> Thank you for the feedback -- I am glad there is some
> >>> >> >>>>>>>> agreement
> >>> >> >>>>>>>> about
> >>> >> >>>>>>>> possible ways forward, and so I'm happy to try to use this as
> >>> >> >>>>>>>> an
> >>> >> >>>>>>>> opportunity to explore simpler, more reliable methods than the
> >>> >> >>>>>>>> install
> >>> >> >>>>>>>> script.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> This afternoon, I spent a bit of time with Conda, and I think
> >>> >> >>>>>>>> it's
> >>> >> >>>>>>>> quite nice.  There are a few rough corners, particularly
> >>> >> >>>>>>>> related
> >>> >> >>>>>>>> to
> >>> >> >>>>>>>> the binstar service, but it's so far pretty great.  With
> >>> >> >>>>>>>> Nathan's
> >>> >> >>>>>>>> help, I was able to upload a yt-2.5.5 package for linux x86_64
> >>> >> >>>>>>>> and
> >>> >> >>>>>>>> then install it.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> The workflow that seems to work:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> * Get miniconda: http://repo.continuum.io/miniconda/index.html
> >>> >> >>>>>>>> * Run the installer for miniconda
> >>> >> >>>>>>>> * Enter the conda environment and then install yt by doing
> >>> >> >>>>>>>> "conda
> >>> >> >>>>>>>> install yt -c http://conda.binstar.org/yt_project/ ".
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> I think that this can likely all be stuck into a bash script.
> >>> >> >>>>>>>> A
> >>> >> >>>>>>>> simple, first pass at this is here:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> http://paste.yt-project.org/show/3833/
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> This right now only works on Linux x86_64, but getting it to
> >>> >> >>>>>>>> work
> >>> >> >>>>>>>> for
> >>> >> >>>>>>>> other machines won't be too hard.  I suspect we will be able
> >>> >> >>>>>>>> to
> >>> >> >>>>>>>> do
> >>> >> >>>>>>>> nightlies very easily as well.  If anyone out there has an
> >>> >> >>>>>>>> x86_64
> >>> >> >>>>>>>> machine they wouldn't mind trying it on, that would be very
> >>> >> >>>>>>>> helpful!
> >>> >> >>>>>>>> I did find that once I ran this script, I had to actually
> >>> >> >>>>>>>> prepend
> >>> >> >>>>>>>> the
> >>> >> >>>>>>>> PATH afterwards as well.  This means doing:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> export LD_LIBRARY_PATH=$HOME/yt-conda:$LD_LIBRARY_PATH
> >>> >> >>>>>>>> export PATH=$HOME/yt-conda:$PATH
> >>> >> >>>>>>>> source activate ytenv
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> At that point, everything was set up and working for me.  The
> >>> >> >>>>>>>> miniconda install offers to add paths to .bashrc, but I don't
> >>> >> >>>>>>>> think
> >>> >> >>>>>>>> we
> >>> >> >>>>>>>> should go down that route.  That being said, this is also a
> >>> >> >>>>>>>> possible
> >>> >> >>>>>>>> point of friction.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> One nice thing is that this also completely works with the
> >>> >> >>>>>>>> full
> >>> >> >>>>>>>> anaconda; if someone wants everything that is in the anaconda
> >>> >> >>>>>>>> install,
> >>> >> >>>>>>>> they can even simply do "conda install anaconda" from the
> >>> >> >>>>>>>> command
> >>> >> >>>>>>>> line
> >>> >> >>>>>>>> to get it.  But the stripped down subset is the default.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> If anyone has a chance to try this out and has feedback, I'd
> >>> >> >>>>>>>> greatly
> >>> >> >>>>>>>> appreciate it!  I think Nathan has done something very similar
> >>> >> >>>>>>>> for
> >>> >> >>>>>>>> OSX.  I've also put a couple simple conda recipes here:
> >>> >> >>>>>>>> https://bitbucket.org/yt_analysis/yt_conda which we can use as
> >>> >> >>>>>>>> a
> >>> >> >>>>>>>> basis
> >>> >> >>>>>>>> for distributing builds and setting them up on buildbots and
> >>> >> >>>>>>>> the
> >>> >> >>>>>>>> like.
> >>> >> >>>>>>>> I'm pretty optimistic about this.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> -Matt
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> On Thu, Aug 29, 2013 at 5:50 PM, Nathan Goldbaum
> >>> >> >>>>>>>> <nathan12343 at gmail.com> wrote:
> >>> >> >>>>>>>>> I think to get everything working in a sustainable fashion,
> >>> >> >>>>>>>>> we
> >>> >> >>>>>>>>> would need
> >>> >> >>>>>>>>> buildbots for all platform combinations that we want to
> >>> >> >>>>>>>>> support,
> >>> >> >>>>>>>>> so
> >>> >> >>>>>>>>> all
> >>> >> >>>>>>>>> permutations of the (32/64 bit,  linux / OS X / windows,
> >>> >> >>>>>>>>> py27/py3.3) tuple.
> >>> >> >>>>>>>>> At the moment anaconda seems to support 32 and 64 bit linux,
> >>> >> >>>>>>>>> 64
> >>> >> >>>>>>>>> bit
> >>> >> >>>>>>>>> OS X
> >>> >> >>>>>>>>> (not totally clear if OS X version matters), and 32 and 64
> >>> >> >>>>>>>>> bit
> >>> >> >>>>>>>>> windows.
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>> Another option is to rely on conda build, which compiles
> >>> >> >>>>>>>>> everything
> >>> >> >>>>>>>>> from
> >>> >> >>>>>>>>> source.
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>> On Thu, Aug 29, 2013 at 2:45 PM, Stephen Skory <s at skory.us>
> >>> >> >>>>>>>>> wrote:
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> Hi all,
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> I have less of a skin in this than I used to, but I'd like
> >>> >> >>>>>>>>>> to
> >>> >> >>>>>>>>>> raise
> >>> >> >>>>>>>>>> the issue of Windows & package managers. For example,
> >>> >> >>>>>>>>>> Anaconda
> >>> >> >>>>>>>>>> is
> >>> >> >>>>>>>>>> available for Windows - would that mean that yt might "just
> >>> >> >>>>>>>>>> work"
> >>> >> >>>>>>>>>> on
> >>> >> >>>>>>>>>> Windows? Or the opposite, and it would require a great deal
> >>> >> >>>>>>>>>> of
> >>> >> >>>>>>>>>> effort
> >>> >> >>>>>>>>>> to get all the various things we expect to be .so's to work
> >>> >> >>>>>>>>>> as
> >>> >> >>>>>>>>>> .dll's
> >>> >> >>>>>>>>>> (such as the Cython helpers or halo-finding stuff)?
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> I don't know the answers to these questions, but I think
> >>> >> >>>>>>>>>> it's
> >>> >> >>>>>>>>>> worth
> >>> >> >>>>>>>>>> thinking about.
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> --
> >>> >> >>>>>>>>>> Stephen Skory
> >>> >> >>>>>>>>>> s at skory.us
> >>> >> >>>>>>>>>> http://stephenskory.com/
> >>> >> >>>>>>>>>> 510.621.3687 (google voice)
> >>> >> >>>>>>>>>> _______________________________________________
> >>> >> >>>>>>>>>> yt-dev mailing list
> >>> >> >>>>>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>> _______________________________________________
> >>> >> >>>>>>>>> yt-dev mailing list
> >>> >> >>>>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>> _______________________________________________
> >>> >> >>>>>>>> yt-dev mailing list
> >>> >> >>>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>
> >>> >> >>>>>>>
> >>> >> >>>>>>> _______________________________________________
> >>> >> >>>>>>> yt-dev mailing list
> >>> >> >>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>
> >>> >> >>>>>>
> >>> >> >>>>>>
> >>> >> >>>>>> _______________________________________________
> >>> >> >>>>>> yt-dev mailing list
> >>> >> >>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>
> >>> >> >>>>>
> >>> >> >>>>>
> >>> >> >>>>> _______________________________________________
> >>> >> >>>>> yt-dev mailing list
> >>> >> >>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>
> >>> >> >>>>
> >>> >> >>>>
> >>> >> >>>> _______________________________________________
> >>> >> >>>> yt-dev mailing list
> >>> >> >>>> yt-dev at lists.spacepope.org
> >>> >> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>
> >>> >> >>>
> >>> >> >>>
> >>> >> >>> _______________________________________________
> >>> >> >>> yt-dev mailing list
> >>> >> >>> yt-dev at lists.spacepope.org
> >>> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>
> >>> >> >>
> >>> >> >>
> >>> >> >> _______________________________________________
> >>> >> >> yt-dev mailing list
> >>> >> >> yt-dev at lists.spacepope.org
> >>> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>
> >>> >> > _______________________________________________
> >>> >> > yt-dev mailing list
> >>> >> > yt-dev at lists.spacepope.org
> >>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >>
> >>> >> _______________________________________________
> >>> >> yt-dev mailing list
> >>> >> yt-dev at lists.spacepope.org
> >>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > yt-dev mailing list
> >>> > yt-dev at lists.spacepope.org
> >>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> _______________________________________________
> >>> yt-dev mailing list
> >>> yt-dev at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> >>
> >>
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> 
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130910/b3a8b632/attachment.htm>


More information about the yt-dev mailing list