<div dir="ltr">Hey J, <div><br></div><div>Sadly there are no primers on this particular issue because it is not a solved problem! Of course, you can pick up the cmake / pip / autoconf / brew / ports manuals and read through those. This will provide a basis for understanding how people have solves this problem when they have complete control over the environment. What it doesn't provide is the context for users running into issues trying to install shard object libraries on platforms that don't natively support a compiler ;). For that, you need experience and to join mailing lists where this comes up, such as yt and (shameless plug) NumFOCUS.</div>
<div><br></div><div>Be Well</div><div>Anthony</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 14, 2013 at 12:12 PM, j s oishi <span dir="ltr"><<a href="mailto:jsoishi@gmail.com" target="_blank">jsoishi@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I am very, very interested in this discussion, but I am an absolute neophyte when it comes to build systems, installing, and so on. Is there a good primer on these issues that I could look at?<span class="HOEnZb"><font color="#888888"><div>
<br></div>
<div>j</div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 14, 2013 at 11:59 AM, Anthony Scopatz <span dir="ltr"><<a href="mailto:scopatz@gmail.com" target="_blank">scopatz@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Thanks for fwding this along Nathan.</p>
<div class="gmail_quote"><div><div>On Nov 14, 2013 3:25 AM, "Nathan Goldbaum" <<a href="mailto:nathan12343@gmail.com" target="_blank">nathan12343@gmail.com</a>> wrote:<br type="attribution"></div>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
<div style="word-wrap:break-word"><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">There is a very interesting conversation going on over on the numfocus list about a distributed build service for the numfocus ecosystem: <a href="https://groups.google.com/forum/#!topic/numfocus/mVNakFqfpZg" target="_blank">https://groups.google.com/forum/#!topic/numfocus/mVNakFqfpZg</a></div>
<div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
I’m forwarding this message from Anthony since he mentions yt. I’m hopeful that in the end this discussion leads to something concrete and useful for our users.</div> <div></div> <br><p style="color:#a0a0a8">On November 14, 2013 at 12:16:44 AM, Anthony Scopatz (<a href="mailto://scopatz@gmail.com" target="_blank">scopatz@gmail.com</a>) wrote:</p>
<blockquote type="cite"><span><div><div>
<div dir="ltr">Hello All,
<div><br></div>
<div>I'd like to volunteer to be the person to spearhead this
effort, or at least be the manager/organizer of this effort.
Normally I wouldn't do this (I have enough on my plate) but the
build and distribution problems have become <i>the</i> central
issue for <a href="http://pynesim.org/" target="_blank">PyNE</a> in
the past 10 days [1]. On Monday Nov. 4th, Katy hosted a PyNE
workshop at UC Berkeley. Out of the ~15 attendees, 2 could
successfully install PyNE...even using anaconda and/or canopy and
ignoring our optional dependencies. <b>We absolutely have to solve
this problem for the project to survive.</b> The PyNE stack fairly
standard for this community. The fact that we have had such
terrible install problems is really a serious issue. We are
going to dedicate the next release to fixing this problem <i>rather
than</i> doing any nuclear engineering.</div>
<div><br></div>
<div>Furthermore, I don't think that this is a wholly unique
situation as I know other codes, such as yt, have experienced
similar difficulties in this space.</div>
<div><br></div>
<div>I believe that to fix this problem sufficiently we must have
empathy for our users, rather than for ourselves as developers.
We need to be open to installing packages in the way the user
is comfortable with even though it may make more work for us.
</div>
<div><br></div>
<div>Many of the problems I feel stem from wanting to support a
public API for many languages simultaneously
(C/C++/Python/Fortran). PyNE is not unique in this way.
NumPy also does this. However, the dependency pool for
PyNE is much broader and deeper than for NumPy. Additionally
as Aaron brings up, we aren't really just talking about supporting
Python & Cython code. By virtue of our dependencies, we
also have to support packages in these compiled libraries.</div>
<div><br></div>
<div>I have many ideas about how to go about this process, though
it will require help from many people. I believe all of the
pieces are there to do this right, but what we need is a
coordinated strategy, someone to carve up tasks appropriately, and
someone to make sure that this process doesn't take too long.
Again, I am happy to be the cat-wrangler, if for no other
reason than that PyNE will be going through this shortly.</div>
<div><br></div>
<div>Travis-ci, vms, etc, have come up quite a bit so far on this
thread. As others have noted, this is the right idea but
insufficiently executed. Therefore, I'd like to direct folks
to the <a href="https://www.batlab.org/" target="_blank">Build & Test Lab
(BaTLab)</a>. This is an NSF funded organization whose purpose it
is to provide <a href="https://www.batlab.org/?q=platforms" target="_blank">an
integrated suite of platforms</a> for building and testing
scientific software. Even though it is at UW-Madison, it is
open to everyone and free to use. The advantage here is that
BaTLab jobs can take as long as they need to and it already
supports the platforms we need in many flavors: Windows, Mac,
Linux. For another project (cyclus), I am working on <a href="https://github.com/cyclus/polyphemus" target="_blank">a little web app</a> which
exposes Travis-CI-like functionality but is backended by BaTLab.
Hopefully this will be in a working state by this weekend.
<i>Basically, there is no need for us to build the
infrastructure!</i> Or at lease no need to do so in the short- to
medium-term.</div>
<div><br></div>
<div>I think if every project were able to donate a person or two
to work on this for the next month we really could get something
working that truly addressed or routed around some of the
fundamental flaws with the current system.</div>
<div><br></div>
<div>In summary, we (PyNE +/- yt) can run off and solve this for
ourselves -OR- we (NumFOCUS) can help pool our resources and try to
create a quality solution that applies to everyone. Sorry if
this is a bit ranty, but it has become difficult for me to sell
Python & PyNE because of building and distribution. This
is long overdue and I am willing to put my money where my mouth is.
If someone has better ideas, I'd love to hear them!</div>
<div><br></div>
<div>Be Well</div>
<div>Anthony</div>
<div><br></div>
<div>1. <a href="https://groups.google.com/d/topic/pyne-dev/RxNCHvZAAEQ/discussion" target="_blank">https://groups.google.com/d/topic/pyne-dev/RxNCHvZAAEQ/discussion</a></div></div>
</div></div></span></blockquote></div><br></div></div>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div>
<br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div>