[yt-dev] Packaging.

Nathan Goldbaum nathan12343 at gmail.com
Sat Sep 7 10:27:14 PDT 2013 

The yt-dev recipe I created would only be good for nightly binary builds.
 I think that would only be useful for people who want the latest and
greatest but don't want to set up a build environment, which I suspect is a
relatively small group of people.  We'll have to decide at some other point
if we want to have buildbots and nightly builds.

Agreed about updating our documentation and website.  I think we should
supply both the script and instructions for setting up yt using `pip` and
already-installed conda packages.  We should also link to a page that lists
the packages needed to manually set up yt.


On Sat, Sep 7, 2013 at 8:22 AM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi all,
>
> I've issued a PR with my first pass at the get_yt.sh script.  I've
> ported over a bunch of functionality from the old install_script.sh
> and it works for me in a clean install on Ubuntu, although it does
> require the "chrpath" package which I oddly did not have.
>
>
> https://bitbucket.org/yt_analysis/yt/pull-request/591/first-draft-of-a-new-get_yt-script-based/diff
>
> What would be really helpful is if a few people could try it out, make
> modifications, commit, push to their fork, and I will then pull in
> those changes until it does everything it needs to.  I think what I
> would like to preserve is that the "get_yt.sh" script should *not* be
> about how to install Conda/Anaconda -- it should get yt installed, and
> the yt environment.  The case of installing yt *into* Conda is what
> I'd like to cover by providing binary packages, and I suspect that our
> audience for that will grow with time.  This script aims along a
> slightly different vector.
>
> I'm still not entirely sure how to handle source distributions; even
> with the yt-dev recipe Nathan has, it's still building binary
> packages.  I've done my best to include the source as-is in the
> script, but to also use the yt recipe that Nathan has worked on.  I'm
> not sure how the "yt-dev" recipe fits into that.  Kacper, Nathan, any
> ideas about that?  Would it just be the basic nightly package builder?
>  Then we could have it get rebuild and reuploaded all the time; this
> would help with keeping people up to date, but *not* with encouraging
> contributions.
>
> -Matt
>
> PS One last thing is that our whole "get yt" documentation should be
> re-done.  We should be much more clear about how and where to get the
> package, and reduce the number of options.
>
>
> On Fri, Sep 6, 2013 at 9:08 AM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> > Hi Nathan,
> >
> > On Wed, Sep 4, 2013 at 2:54 AM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> >> Here's an updated version of the shell script Matt sent out earlier
> today
> >> that builds yt and then installs it based on the yt-conda recipe:
> >> http://paste.yt-project.org/show/3848/
> >>
> >> I like this a lot and think it can be improved in several simple ways,
> most
> >> of which can be readily scraped from the install script:
> >
> > Me too.  This is much improved over mine.
> >
> >>
> >>     * Install in the folder that the script was invoked in rather than
> $HOME
> >
> > Good idea.
> >
> >>     * Add an option to install anaconda with a warning about needing a
> >> couple hundred megabytes of space
> >
> > Excellent!  I like this a lot.  And, we should point to the Anaconda
> > page for more info about packages and how Anaconda works.
> >
> >>     * Install the miniconda package appropriate for the platform
> >>     * Add warnings on various platforms regarding needed packages and
> >> compilers (i.e. still need build-essentials on ubuntu and Xcode on OSX).
> >>     * Add a warning at the end about modifying .bashrc for PATH and
> >> LD_LIBRARY_PATH or supply an activate script that sets them for us.
> >
> > I think I'd prefer to supply an activate script that both activates
> > Conda *and* the yt environment in it.  That way individuals who want
> > to use Conda directly can do so, but we also make it possible to keep
> > the behavior that people have already.
> >
> >>     * Clone yt-doc, yt-supplemental and the main yt repo and put them
> in the
> >> yt-conda directory.
> >
> > Agreed.
> >
> >>     * Check if conda is already available and (optionally?) skip the
> >> miniconda step.
> >
> > I'm not sure about this, since I think we will then just tell people
> > to install their own yt.  I think the install script should not be
> > *too* complex in this regard.
> >
> >>     * Ensure the user that yt loves them.
> >
> > Yes.
> >
> >>
> >> As Matt suggested on a Conda github issue
> >> (https://github.com/ContinuumIO/conda/issues/176#issuecomment-23753101)
> it
> >> would be nice to first try to conda install yt and then if that fails
> use
> >> conda build.  I think it should be possible to set this up as part of
> the
> >> bash script.  As asmurer notes in the issue, general support for this
> sort
> >> of mixed binary/source distribution in conda might be difficult but
> with yt
> >> it works since our setup script is platform agnostic so long as the
> >> compilation environment is set up correctly and we don't even try to
> work
> >> correctly on windows.
> >>
> >> So long as we provide binary yt packages of (at least) the stable
> releases,
> >> this should obviate the need for proper compiler setup, which avoid a
> >> significant source of pain for OS X users as well as linux users on
> >> platforms like ubuntu that don't provide sane build environments out of
> the
> >> box.
> >>
> >> -Nathan
> >
> > Awesome work, and I am getting more excited about this as time goes
> > on.  I am going to try my hand at updating the new script in the way
> > you mentioned, and then I will try checking it into the repo and issue
> > a PR so we can iterate there.  I hope to get to this by tomorrow
> > morning.
> >
> > -Matt
> >
> >>
> >>
> >> On Tue, Sep 3, 2013 at 12:28 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>>
> >>> Hi Nathan,
> >>>
> >>> On Tue, Sep 3, 2013 at 3:10 PM, Nathan Goldbaum <nathan12343 at gmail.com
> >
> >>> wrote:
> >>> > John, can you give a little more info on that error?  I'm a little
> >>> > concerned
> >>> > that there are binary incompatibilities that conda can't resolve.
> >>> > Perhaps
> >>> > this is something we should report on the conda issue tracker.
> >>> >
> >>> > Matt, can't we get the recipes in the same way we get the latest dev
> >>> > install
> >>> > script?  Something like:
> >>> >
> >>> > wget https://bitbucket.org/yt_analysis/yt_conda/raw/yt/meta.yaml
> >>> >
> >>> > This doesn't currently work, although it does work for the yt repo, I
> >>> > think
> >>> > because we use named branches in that repository.
> >>>
> >>> Yup -- you can do it like this:
> >>>
> >>> wget
> https://bitbucket.org/yt_analysis/yt_conda/raw/default/yt/meta.yaml
> >>>
> >>> ...although for me right now it's failing from a bad BB certificate.
> >>>
> >>> The only problem with "pip install" is that it no longer lists pip
> >>> installations inside the "conda list" or "conda update" outputs.  This
> >>> has been discussed recently on the mailing list for Anaconda:
> >>>
> >>>
> >>>
> https://groups.google.com/a/continuum.io/forum/?fromgroups#!topic/anaconda/wr95VN7ezpo
> >>>
> >>> ...but there has not yet been a resolution.
> >>>
> >>> -Matt
> >>>
> >>> >
> >>> >
> >>> > On Tue, Sep 3, 2013 at 12:00 PM, John ZuHone <jzuhone at gmail.com>
> wrote:
> >>> >>
> >>> >> Works fine for me on OS X x86_64.
> >>> >>
> >>> >> On my Goddard-controlled Linux x86_64 server, everything worked fine
> >>> >> except Mercurial, which I "conda install"-ed using the yt link, but
> the
> >>> >> binary had an incompatibility which the glibc that was installed on
> the
> >>> >> machine. Using pip to install Mercurial (which did it from source)
> was
> >>> >> the
> >>> >> workaround.
> >>> >>
> >>> >> On Sep 3, 2013, at 2:54 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>> >>
> >>> >> > Hi all,
> >>> >> >
> >>> >> > Thanks to everybody who has reported back on testing.  After some
> >>> >> > talking both offline and on IRC, as well as here, I think we would
> >>> >> > need to do the following things:
> >>> >> >
> >>> >> > * Make a single script that grabs the appropriate distribution of
> >>> >> > miniconda and installs it.  Right now I have a mechanism for doing
> >>> >> > this, but it's currently tied to an architecture.
> >>> >> > * Create a mechanism for installing all the packages we need.
>  Nearly
> >>> >> > all are available inside the Continuum repos.  What we're hung up
> on
> >>> >> > is that source installs require a "recipe", and transmitting the
> >>> >> > recipe is where I don't have an idea of what to do.
> >>> >> > * Test this out lots of places
> >>> >> > * Clean up the edges in the (new) install script
> >>> >> > * Move the old install script to maintenance mode
> >>> >> > * Update all documentation to describe this and mothball other
> >>> >> > methods of installation
> >>> >> >
> >>> >> > What would be nice:
> >>> >> >
> >>> >> > * Make available nightly builds of yt on several architectures
> using
> >>> >> > binstar
> >>> >> > * Utilize more of the packages included in conda elsewhere in yt,
> now
> >>> >> > that we can!
> >>> >> >
> >>> >> > Here's my current recipe for get_yt.sh:
> >>> >> >
> >>> >> > http://paste.yt-project.org/show/3843/
> >>> >> >
> >>> >> > (The config thing may get switched to include the --system
> argument,
> >>> >> > to modify the "yt-conda" condarc.)  The step that I'm most stuck
> on
> >>> >> > is
> >>> >> > getting the yt recipe to people.  If we want to make it possible
> and
> >>> >> > easy to build from source, we need to get the contents of a "conda
> >>> >> > recipe" to people.  They can then run "conda build ." in the
> >>> >> > directory.  Here are the recipes that we've been playing with:
> >>> >> >
> >>> >> > https://bitbucket.org/yt_analysis/yt_conda/src
> >>> >> >
> >>> >> > Basically, in get_yt.sh, to do from source instead of from binary
> we
> >>> >> > need to insert a step at the end that downloads the recipes
> somehow
> >>> >> > and then cd's into the right directory and builds them.  The
> reason
> >>> >> > this is tricky is that we often need to bootstrap ourselves; we
> can't
> >>> >> > assume anything exists.  We can download the .tar.bz2 of the
> current
> >>> >> > tip of the repo, but it includes the hash in the directory name
> that
> >>> >> > it extracts to.
> >>> >> >
> >>> >> > So I think what we need is a mechanism for getting the current
> state
> >>> >> > of the repo, figuring out the name of the repo's directory, moves
> >>> >> > into
> >>> >> > it, and then builds.  I believe that all/most of this becomes
> much,
> >>> >> > much easier if hg gets included in Anaconda, which Nathan has
> >>> >> > submitted a PR for.  So hopefully that will be taken care of, but
> >>> >> > until that time we can possibly figure something out.  I'm not
> sure
> >>> >> > that we have the resources to continually support binary nightly
> >>> >> > builds in perpetuity for all the architectures that people run
> on, so
> >>> >> > having source would be awesome.  Plus, one of the biggest appeals
> of
> >>> >> > how we distribute yt is that the source is included; I would very
> >>> >> > much
> >>> >> > not like to give this up.
> >>> >> >
> >>> >> > Thoughts?  Has anyone else tested any of this out?
> >>> >> >
> >>> >> > -Matt
> >>> >> >
> >>> >> > On Tue, Sep 3, 2013 at 1:04 PM, Britton Smith
> >>> >> > <brittonsmith at gmail.com>
> >>> >> > wrote:
> >>> >> >> Hi everyone,
> >>> >> >>
> >>> >> >> Sorry for chiming in late.  I just moved when this thread began
> and
> >>> >> >> do
> >>> >> >> not
> >>> >> >> have regular internet access.
> >>> >> >>
> >>> >> >> I really like this idea of conda, especially as a package manager
> >>> >> >> that
> >>> >> >> only
> >>> >> >> optionally makes its own edits to your .bashrc.  I have always
> >>> >> >> really
> >>> >> >> liked
> >>> >> >> that the install script creates a clean python stack with
> basically
> >>> >> >> everything a python user needs.  I have on occasion suggested it
> to
> >>> >> >> people
> >>> >> >> just looking to use numpy and matploblib.  It looks like conda
> will
> >>> >> >> continue
> >>> >> >> to provide this nice by-product, so I'm all for it.
> >>> >> >>
> >>> >> >> I won't be in a position to help with testing and such for
> another
> >>> >> >> week
> >>> >> >> or
> >>> >> >> so when I get regular internet access, but I would be glad to do
> so
> >>> >> >> then.
> >>> >> >>
> >>> >> >> Britton
> >>> >> >>
> >>> >> >>
> >>> >> >> On Fri, Aug 30, 2013 at 9:29 PM, Nathan Goldbaum
> >>> >> >> <nathan12343 at gmail.com>
> >>> >> >> wrote:
> >>> >> >>>
> >>> >> >>> Everything should be available now for 64 bit linux and OS X.
> >>> >> >>>
> >>> >> >>>
> >>> >> >>> On Fri, Aug 30, 2013 at 10:52 AM, Chris Malone
> >>> >> >>> <chris.m.malone at gmail.com>
> >>> >> >>> wrote:
> >>> >> >>>>
> >>> >> >>>> Hi Nathan,
> >>> >> >>>>
> >>> >> >>>> That appears to work as it built the environment and `conda
> >>> >> >>>> install
> >>> >> >>>> ...`
> >>> >> >>>> added packages to my environment.
> >>> >> >>>>
> >>> >> >>>> One mistake I made was that I originally downloaded the
> "latest"
> >>> >> >>>> OS X
> >>> >> >>>> build of Miniconda, but that happened to be Miniconda3, which
> is
> >>> >> >>>> python 3
> >>> >> >>>> based.  Trying to build the environment with that yields an
> error
> >>> >> >>>> regarding
> >>> >> >>>> incompatibility of yt and python3.3, as it should.
> >>> >> >>>>
> >>> >> >>>> Chris
> >>> >> >>>>
> >>> >> >>>>
> >>> >> >>>> On Fri, Aug 30, 2013 at 10:42 AM, Nathan Goldbaum
> >>> >> >>>> <nathan12343 at gmail.com>
> >>> >> >>>> wrote:
> >>> >> >>>>>
> >>> >> >>>>> Hey Chris,
> >>> >> >>>>>
> >>> >> >>>>> I don't think mercurial is strictly necessary, can you try
> again
> >>> >> >>>>> without
> >>> >> >>>>> it?  I think if Matt uploads a mercurial package for OS X this
> >>> >> >>>>> won't
> >>> >> >>>>> be an
> >>> >> >>>>> issue. I'll send him an updated tarball.
> >>> >> >>>>>
> >>> >> >>>>> I submitted a mercurial recipe to conda-recipes yesterday
> >>> >> >>>>> (https://github.com/ContinuumIO/conda-recipes/pull/14) so
> >>> >> >>>>> hopefully
> >>> >> >>>>> a
> >>> >> >>>>> mercurial build will be included in future anaconda releases.
> >>> >> >>>>>
> >>> >> >>>>>
> >>> >> >>>>> On Fri, Aug 30, 2013 at 10:39 AM, Chris Malone
> >>> >> >>>>> <chris.m.malone at gmail.com> wrote:
> >>> >> >>>>>>
> >>> >> >>>>>> I just tried setting this up on OS X 10.7.5 and failed when
> >>> >> >>>>>> attempting
> >>> >> >>>>>> to create the conda environment due to a missing mercurial
> >>> >> >>>>>> package:
> >>> >> >>>>>>
> >>> >> >>>>>> $ conda create -n ytenv -c
> http://conda.binstar.org/yt_project
> >>> >> >>>>>> yt
> >>> >> >>>>>> mercurial ipython tornado pyzmq pygments jinja2 sphinx
> >>> >> >>>>>> Error: No packages found matching: mercurial
> >>> >> >>>>>>
> >>> >> >>>>>>
> >>> >> >>>>>> On Thu, Aug 29, 2013 at 9:01 PM, Nathan Goldbaum
> >>> >> >>>>>> <nathan12343 at gmail.com> wrote:
> >>> >> >>>>>>>
> >>> >> >>>>>>> Yup, please try on OSX as well.  If you make sure Matt's
> >>> >> >>>>>>> binstar
> >>> >> >>>>>>> is in
> >>> >> >>>>>>> your .condarc, you should be able to get yt by doing 'conda
> >>> >> >>>>>>> install yt'.
> >>> >> >>>>>>>
> >>> >> >>>>>>> I built the OSX binary on my laptop so I'd appreciate
> hearing
> >>> >> >>>>>>> about
> >>> >> >>>>>>> issues, particularly if there are issues on older OS X
> >>> >> >>>>>>> releases.
> >>> >> >>>>>>>
> >>> >> >>>>>>> On Thursday, August 29, 2013, Matthew Turk wrote:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> Hi all,
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> Thank you for the feedback -- I am glad there is some
> >>> >> >>>>>>>> agreement
> >>> >> >>>>>>>> about
> >>> >> >>>>>>>> possible ways forward, and so I'm happy to try to use this
> as
> >>> >> >>>>>>>> an
> >>> >> >>>>>>>> opportunity to explore simpler, more reliable methods than
> the
> >>> >> >>>>>>>> install
> >>> >> >>>>>>>> script.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> This afternoon, I spent a bit of time with Conda, and I
> think
> >>> >> >>>>>>>> it's
> >>> >> >>>>>>>> quite nice.  There are a few rough corners, particularly
> >>> >> >>>>>>>> related
> >>> >> >>>>>>>> to
> >>> >> >>>>>>>> the binstar service, but it's so far pretty great.  With
> >>> >> >>>>>>>> Nathan's
> >>> >> >>>>>>>> help, I was able to upload a yt-2.5.5 package for linux
> x86_64
> >>> >> >>>>>>>> and
> >>> >> >>>>>>>> then install it.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> The workflow that seems to work:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> * Get miniconda:
> http://repo.continuum.io/miniconda/index.html
> >>> >> >>>>>>>> * Run the installer for miniconda
> >>> >> >>>>>>>> * Enter the conda environment and then install yt by doing
> >>> >> >>>>>>>> "conda
> >>> >> >>>>>>>> install yt -c http://conda.binstar.org/yt_project/ ".
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> I think that this can likely all be stuck into a bash
> script.
> >>> >> >>>>>>>> A
> >>> >> >>>>>>>> simple, first pass at this is here:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> http://paste.yt-project.org/show/3833/
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> This right now only works on Linux x86_64, but getting it
> to
> >>> >> >>>>>>>> work
> >>> >> >>>>>>>> for
> >>> >> >>>>>>>> other machines won't be too hard.  I suspect we will be
> able
> >>> >> >>>>>>>> to
> >>> >> >>>>>>>> do
> >>> >> >>>>>>>> nightlies very easily as well.  If anyone out there has an
> >>> >> >>>>>>>> x86_64
> >>> >> >>>>>>>> machine they wouldn't mind trying it on, that would be very
> >>> >> >>>>>>>> helpful!
> >>> >> >>>>>>>> I did find that once I ran this script, I had to actually
> >>> >> >>>>>>>> prepend
> >>> >> >>>>>>>> the
> >>> >> >>>>>>>> PATH afterwards as well.  This means doing:
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> export LD_LIBRARY_PATH=$HOME/yt-conda:$LD_LIBRARY_PATH
> >>> >> >>>>>>>> export PATH=$HOME/yt-conda:$PATH
> >>> >> >>>>>>>> source activate ytenv
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> At that point, everything was set up and working for me.
>  The
> >>> >> >>>>>>>> miniconda install offers to add paths to .bashrc, but I
> don't
> >>> >> >>>>>>>> think
> >>> >> >>>>>>>> we
> >>> >> >>>>>>>> should go down that route.  That being said, this is also a
> >>> >> >>>>>>>> possible
> >>> >> >>>>>>>> point of friction.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> One nice thing is that this also completely works with the
> >>> >> >>>>>>>> full
> >>> >> >>>>>>>> anaconda; if someone wants everything that is in the
> anaconda
> >>> >> >>>>>>>> install,
> >>> >> >>>>>>>> they can even simply do "conda install anaconda" from the
> >>> >> >>>>>>>> command
> >>> >> >>>>>>>> line
> >>> >> >>>>>>>> to get it.  But the stripped down subset is the default.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> If anyone has a chance to try this out and has feedback,
> I'd
> >>> >> >>>>>>>> greatly
> >>> >> >>>>>>>> appreciate it!  I think Nathan has done something very
> similar
> >>> >> >>>>>>>> for
> >>> >> >>>>>>>> OSX.  I've also put a couple simple conda recipes here:
> >>> >> >>>>>>>> https://bitbucket.org/yt_analysis/yt_conda which we can
> use as
> >>> >> >>>>>>>> a
> >>> >> >>>>>>>> basis
> >>> >> >>>>>>>> for distributing builds and setting them up on buildbots
> and
> >>> >> >>>>>>>> the
> >>> >> >>>>>>>> like.
> >>> >> >>>>>>>> I'm pretty optimistic about this.
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> -Matt
> >>> >> >>>>>>>>
> >>> >> >>>>>>>> On Thu, Aug 29, 2013 at 5:50 PM, Nathan Goldbaum
> >>> >> >>>>>>>> <nathan12343 at gmail.com> wrote:
> >>> >> >>>>>>>>> I think to get everything working in a sustainable
> fashion,
> >>> >> >>>>>>>>> we
> >>> >> >>>>>>>>> would need
> >>> >> >>>>>>>>> buildbots for all platform combinations that we want to
> >>> >> >>>>>>>>> support,
> >>> >> >>>>>>>>> so
> >>> >> >>>>>>>>> all
> >>> >> >>>>>>>>> permutations of the (32/64 bit,  linux / OS X / windows,
> >>> >> >>>>>>>>> py27/py3.3) tuple.
> >>> >> >>>>>>>>> At the moment anaconda seems to support 32 and 64 bit
> linux,
> >>> >> >>>>>>>>> 64
> >>> >> >>>>>>>>> bit
> >>> >> >>>>>>>>> OS X
> >>> >> >>>>>>>>> (not totally clear if OS X version matters), and 32 and 64
> >>> >> >>>>>>>>> bit
> >>> >> >>>>>>>>> windows.
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>> Another option is to rely on conda build, which compiles
> >>> >> >>>>>>>>> everything
> >>> >> >>>>>>>>> from
> >>> >> >>>>>>>>> source.
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>> On Thu, Aug 29, 2013 at 2:45 PM, Stephen Skory <
> s at skory.us>
> >>> >> >>>>>>>>> wrote:
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> Hi all,
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> I have less of a skin in this than I used to, but I'd
> like
> >>> >> >>>>>>>>>> to
> >>> >> >>>>>>>>>> raise
> >>> >> >>>>>>>>>> the issue of Windows & package managers. For example,
> >>> >> >>>>>>>>>> Anaconda
> >>> >> >>>>>>>>>> is
> >>> >> >>>>>>>>>> available for Windows - would that mean that yt might
> "just
> >>> >> >>>>>>>>>> work"
> >>> >> >>>>>>>>>> on
> >>> >> >>>>>>>>>> Windows? Or the opposite, and it would require a great
> deal
> >>> >> >>>>>>>>>> of
> >>> >> >>>>>>>>>> effort
> >>> >> >>>>>>>>>> to get all the various things we expect to be .so's to
> work
> >>> >> >>>>>>>>>> as
> >>> >> >>>>>>>>>> .dll's
> >>> >> >>>>>>>>>> (such as the Cython helpers or halo-finding stuff)?
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> I don't know the answers to these questions, but I think
> >>> >> >>>>>>>>>> it's
> >>> >> >>>>>>>>>> worth
> >>> >> >>>>>>>>>> thinking about.
> >>> >> >>>>>>>>>>
> >>> >> >>>>>>>>>> --
> >>> >> >>>>>>>>>> Stephen Skory
> >>> >> >>>>>>>>>> s at skory.us
> >>> >> >>>>>>>>>> http://stephenskory.com/
> >>> >> >>>>>>>>>> 510.621.3687 (google voice)
> >>> >> >>>>>>>>>> _______________________________________________
> >>> >> >>>>>>>>>> yt-dev mailing list
> >>> >> >>>>>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>>> _______________________________________________
> >>> >> >>>>>>>>> yt-dev mailing list
> >>> >> >>>>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>>>
> >>> >> >>>>>>>> _______________________________________________
> >>> >> >>>>>>>> yt-dev mailing list
> >>> >> >>>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>
> >>> >> >>>>>>>
> >>> >> >>>>>>> _______________________________________________
> >>> >> >>>>>>> yt-dev mailing list
> >>> >> >>>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>>
> >>> >> >>>>>>
> >>> >> >>>>>>
> >>> >> >>>>>> _______________________________________________
> >>> >> >>>>>> yt-dev mailing list
> >>> >> >>>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>>
> >>> >> >>>>>
> >>> >> >>>>>
> >>> >> >>>>> _______________________________________________
> >>> >> >>>>> yt-dev mailing list
> >>> >> >>>>> yt-dev at lists.spacepope.org
> >>> >> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>>
> >>> >> >>>>
> >>> >> >>>>
> >>> >> >>>> _______________________________________________
> >>> >> >>>> yt-dev mailing list
> >>> >> >>>> yt-dev at lists.spacepope.org
> >>> >> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>>
> >>> >> >>>
> >>> >> >>>
> >>> >> >>> _______________________________________________
> >>> >> >>> yt-dev mailing list
> >>> >> >>> yt-dev at lists.spacepope.org
> >>> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>
> >>> >> >>
> >>> >> >>
> >>> >> >> _______________________________________________
> >>> >> >> yt-dev mailing list
> >>> >> >> yt-dev at lists.spacepope.org
> >>> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>
> >>> >> > _______________________________________________
> >>> >> > yt-dev mailing list
> >>> >> > yt-dev at lists.spacepope.org
> >>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >>
> >>> >> _______________________________________________
> >>> >> yt-dev mailing list
> >>> >> yt-dev at lists.spacepope.org
> >>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > yt-dev mailing list
> >>> > yt-dev at lists.spacepope.org
> >>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> _______________________________________________
> >>> yt-dev mailing list
> >>> yt-dev at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> >>
> >>
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130907/e0168922/attachment.html>

More information about the yt-dev mailing list