Hi Matt,<br><br>Are there resources describing how to set up new tests?<br><br>Britton<br><br><div class="gmail_quote">On Wed, Oct 10, 2012 at 7:26 PM, Matthew Turk <span dir="ltr"><<a href="mailto:matthewturk@gmail.com" target="_blank">matthewturk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hey Britton,<br>
<br>
No worries about asking again -- there are currently two types of<br>
tests. One is nascent, and only a handful exist (although many<br>
libraries have many thousands) and that's unit testing. That's<br>
primarily what Sam and everyone else are talking about, although<br>
"testing" also includes Answer Testing, which is currently running<br>
every 30 minutes or so on my machine. We want to supplement answer<br>
testing with much smaller, more contained unit testing. Answer<br>
testing is the same thing you and I worked on about a year ago, and<br>
it's also what we use for Enzo. (That was the subject of the ticket,<br>
although now it needs to be extended.)<br>
<br>
But, we now want to add on very tiny, unit tests that do not require<br>
either a reference answer that's externally available or a lot of time<br>
to run the tests. So for instance, this would be the new<br>
interpolation tests and so on.<br>
<br>
-Matt<br>
<div class="HOEnZb"><div class="h5"><br>
On Wed, Oct 10, 2012 at 8:27 AM, Britton Smith <<a href="mailto:brittonsmith@gmail.com">brittonsmith@gmail.com</a>> wrote:<br>
> Hi all,<br>
><br>
> I apologize if I'm asking questions on things that have already been<br>
> discussed. What documentation currently exists on the testing<br>
> infrastructure, specifically how to write and contribute the tests? I have<br>
> been long assigned to the issue of "testing documentation" on the issue<br>
> tracker, but I took myself off it this morning because I haven't worked with<br>
> it in so long that I don't remember how it works. I would be willing to<br>
> help out getting the tests together, but it would be helpful if there was<br>
> something I could go look at for how to do them and what remains. I've been<br>
> out of the loop for a while, so sorry if this has been dealt with.<br>
><br>
> Also, I like the idea of having an issue that can be pointed to for each<br>
> enhancement. I would support that as well as quarterly releases.<br>
><br>
> Britton<br>
><br>
><br>
> On Wed, Oct 10, 2012 at 10:48 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
> wrote:<br>
>><br>
>> Hi Sam,<br>
>><br>
>> On Tue, Oct 9, 2012 at 9:42 PM, Sam Skillman <<a href="mailto:samskillman@gmail.com">samskillman@gmail.com</a>><br>
>> wrote:<br>
>> ><br>
>> ><br>
>> > On Tue, Oct 9, 2012 at 10:34 PM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> Hi Mike,<br>
>> >><br>
>> >> On Tue, Oct 9, 2012 at 9:23 PM, Michael Kuhlen <<a href="mailto:mqk@astro.berkeley.edu">mqk@astro.berkeley.edu</a>><br>
>> >> wrote:<br>
>> >> > Hi Jeff<br>
>> >> ><br>
>> >> ><br>
>> >> >> Holy crap, I didn't realize<br>
>> >> >><br>
>> >> >> pip install yt<br>
>> >> >><br>
>> >> >> was a goal! that would be awesome.<br>
>> >> ><br>
>> >> > In that case you may be interested in this ubuntu PPA I made a little<br>
>> >> > while<br>
>> >> > ago, for yt (2.4) and yt-devel (2.5):<br>
>> >> > <a href="https://launchpad.net/~kuhlen/+archive/ppa" target="_blank">https://launchpad.net/~kuhlen/+archive/ppa</a><br>
>> >> ><br>
>> >> > The current version of yt-devel is based on changeset 467f57b (from<br>
>> >> > 08/24).<br>
>> >> > I need to update it...<br>
>> >><br>
>> >> I completely forgot to update the web page! I will do this either<br>
>> >> tomorrow or Thursday (although if anybody wants to issue a pull<br>
>> >> request to the website with the info, it can be redeployed asap.)<br>
>> >> Thank you again for doing this!<br>
>> >><br>
>> >> I think the PPA, Kacper's ebuild, having pip install work, and TomR's<br>
>> >> MacPorts are all really, really good reasons to start focusing on<br>
>> >> reducing the install script overhead, handling things like<br>
>> >> dependencies in a more clear way, and making yt work as an independent<br>
>> >> software package much better. And I think the more we move into this<br>
>> >> area the more we should try to have a rolling, regular release<br>
>> >> schedule. Does that ring true to everybody else, too? The more we<br>
>> >> have the ability to install yt independently of hg, independently of<br>
>> >> the install_script, the more we should try to make a regular release<br>
>> >> schedule with it.<br>
>> ><br>
>> ><br>
>> > Yes! I personally think regular releases should be nearly automated<br>
>> > based<br>
>> > on the passing of tests at regular intervals (i.e. monthly/quarterly).<br>
>> > If<br>
>> > we are diligent about setting up BB issues that track individual<br>
>> > enhancements, even the features changelog could be easily generated.<br>
>><br>
>> I like this idea. We have speculated in the past about moving to<br>
>> quarterly releases. If we were better about managing the issue<br>
>> tracker (or JIRA!) and unit (not just answer) testing new<br>
>> functionality, this would be easier to manage. Furthermore, as you<br>
>> note, the changelog would be easier to write. Should we mandate that<br>
>> any substantial PR also include reference to an issue? Perhaps simply<br>
>> having an issue point to the PR and be closed when the PR is closed is<br>
>> good, to ensure we don't fragment the PR conversations but that we<br>
>> have a unified place where changes are tracked.<br>
>><br>
>> I would support this. But we *need* to have a testing push to make it<br>
>> happen. I've been out of the loop most of this week, but I hope to be<br>
>> back in action next week. So what we're looking at is:<br>
>><br>
>> 1) Issue tracking for enhancements, to allow for changelog writing and so<br>
>> on<br>
>> 2) Regular releases -- I'd push for quarterly -- with a real release<br>
>> coordinator<br>
>> 3) Much higher barrier to entry for testing<br>
>><br>
>> Would contributors be willing to participate in this? I will commit<br>
>> to unit testing new functionality in advance of any push or PR.<br>
>><br>
>> -Matt<br>
>><br>
>> ><br>
>> > Sam<br>
>> ><br>
>> >><br>
>> >> -Matt<br>
>> >><br>
>> >> ><br>
>> >> > Cheers,<br>
>> >> ><br>
>> >> > Mike<br>
>> >> ><br>
>> >> ><br>
>> >> > On Tue, Oct 9, 2012 at 4:03 PM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> >> > wrote:<br>
>> >> >><br>
>> >> >> Hi Casey,<br>
>> >> >><br>
>> >> >> On Tue, Oct 9, 2012 at 3:59 PM, Casey W. Stark<br>
>> >> >> <<a href="mailto:caseywstark@gmail.com">caseywstark@gmail.com</a>><br>
>> >> >> wrote:<br>
>> >> >> > Just so this is clear, if we are working on development that is<br>
>> >> >> > not<br>
>> >> >> > testing,<br>
>> >> >> > should we move over to 3.0 now?<br>
>> >> >><br>
>> >> >> Not yet, but soon. Sorry, I should have been more clear -- I<br>
>> >> >> believe<br>
>> >> >> it's almost ready for primetime, and in a settled state for<br>
>> >> >> rectilinear, patch-based data. I will update the list very, very<br>
>> >> >> soon<br>
>> >> >> on its state. I'll go through the milestone list and take a crack<br>
>> >> >> at<br>
>> >> >> updating the tickets, the scripts, and report back.<br>
>> >> >><br>
>> >> >> -Matt<br>
>> >> >><br>
>> >> >> ><br>
>> >> >> > - Casey<br>
>> >> >> ><br>
>> >> >> ><br>
>> >> >> > On Tue, Oct 9, 2012 at 2:38 PM, Sam Skillman<br>
>> >> >> > <<a href="mailto:samskillman@gmail.com">samskillman@gmail.com</a>><br>
>> >> >> > wrote:<br>
>> >> >> >><br>
>> >> >> >> Sounds good to me. I was actually also holding out a bit to<br>
>> >> >> >> incorporate<br>
>> >> >> >> testing into some of the new rendering capabilities anyways.<br>
>> >> >> >><br>
>> >> >> >><br>
>> >> >> >> On Tue, Oct 9, 2012 at 3:36 PM, Matthew Turk<br>
>> >> >> >> <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> >> >> >> wrote:<br>
>> >> >> >>><br>
>> >> >> >>> Hi Sam,<br>
>> >> >> >>><br>
>> >> >> >>> On Tue, Oct 9, 2012 at 2:30 PM, Sam Skillman<br>
>> >> >> >>> <<a href="mailto:samskillman@gmail.com">samskillman@gmail.com</a>><br>
>> >> >> >>> wrote:<br>
>> >> >> >>> > If the release timeframe is end of year, I will put in the<br>
>> >> >> >>> > alpha<br>
>> >> >> >>> > channel<br>
>> >> >> >>> > rendering, enabling a lot of cool things. It is already<br>
>> >> >> >>> > functional<br>
>> >> >> >>> > in<br>
>> >> >> >>> > one<br>
>> >> >> >>> > of my forks, but it needs to be cleaned up.<br>
>> >> >> >>><br>
>> >> >> >>> What if we said instead that we'd release as soon as unit<br>
>> >> >> >>> testing<br>
>> >> >> >>> is<br>
>> >> >> >>> ready and 3.0 is ready for daily use for patch-based AMR, and<br>
>> >> >> >>> then<br>
>> >> >> >>> if<br>
>> >> >> >>> you have time before that point to get the alpha channel in<br>
>> >> >> >>> good,<br>
>> >> >> >>> but<br>
>> >> >> >>> otherwise toss it into 3.0?<br>
>> >> >> >>><br>
>> >> >> >>> -Matt<br>
>> >> >> >>><br>
>> >> >> >>> ><br>
>> >> >> >>> > Sam<br>
>> >> >> >>> ><br>
>> >> >> >>> ><br>
>> >> >> >>> > On Tue, Oct 9, 2012 at 3:28 PM, Matthew Turk<br>
>> >> >> >>> > <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> >> >> >>> > wrote:<br>
>> >> >> >>> >><br>
>> >> >> >>> >> Hi Jeff,<br>
>> >> >> >>> >><br>
>> >> >> >>> >> On Tue, Oct 9, 2012 at 2:19 PM, j s oishi <<a href="mailto:jsoishi@gmail.com">jsoishi@gmail.com</a>><br>
>> >> >> >>> >> wrote:<br>
>> >> >> >>> >> > Hi,<br>
>> >> >> >>> >> ><br>
>> >> >> >>> >> > Since testing is something that is so high priority for<br>
>> >> >> >>> >> > this,<br>
>> >> >> >>> >> > and<br>
>> >> >> >>> >> > otherwise 2.5 is just a stepping stone to 3.0 (which a<br>
>> >> >> >>> >> > *lot*<br>
>> >> >> >>> >> > of<br>
>> >> >> >>> >> > people<br>
>> >> >> >>> >> > are already diving into), maybe we should *only* include<br>
>> >> >> >>> >> > testing,<br>
>> >> >> >>> >> > unless there are some already done things we could toss in?<br>
>> >> >> >>> >> ><br>
>> >> >> >>> >> > j<br>
>> >> >> >>> >><br>
>> >> >> >>> >> Come to mention it, I *really* like this idea. Perhaps we<br>
>> >> >> >>> >> should<br>
>> >> >> >>> >> identify a threshold for building out the non-core<br>
>> >> >> >>> >> infrastructure<br>
>> >> >> >>> >> fixes (i.e., having "pip install yt" work, having a good set<br>
>> >> >> >>> >> of<br>
>> >> >> >>> >> testing, etc etc) and then any other fixes or improvements<br>
>> >> >> >>> >> that<br>
>> >> >> >>> >> happen<br>
>> >> >> >>> >> along the way are just icing on the cake? I think having<br>
>> >> >> >>> >> better<br>
>> >> >> >>> >> testing should definitely be the focus, particularly as we<br>
>> >> >> >>> >> transition<br>
>> >> >> >>> >> the codebase.<br>
>> >> >> >>> >><br>
>> >> >> >>> >> -Matt<br>
>> >> >> >>> >><br>
>> >> >> >>> >> ><br>
>> >> >> >>> >> > On Tue, Oct 9, 2012 at 5:14 PM, Matthew Turk<br>
>> >> >> >>> >> > <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> >> >> >>> >> > wrote:<br>
>> >> >> >>> >> >> Hi all,<br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >> We should probably try to get a 2.5 release together by<br>
>> >> >> >>> >> >> the<br>
>> >> >> >>> >> >> end<br>
>> >> >> >>> >> >> of<br>
>> >> >> >>> >> >> the<br>
>> >> >> >>> >> >> year. It would be really helpful if you are working on<br>
>> >> >> >>> >> >> something,<br>
>> >> >> >>> >> >> to<br>
>> >> >> >>> >> >> fill it out and target both milestone 2.5 and version 2.5<br>
>> >> >> >>> >> >> as<br>
>> >> >> >>> >> >> an<br>
>> >> >> >>> >> >> issue.<br>
>> >> >> >>> >> >> That way we can identify goals and push to stable.<br>
>> >> >> >>> >> >> Testing<br>
>> >> >> >>> >> >> should<br>
>> >> >> >>> >> >> perhaps be a huge focus of this release. But, once it's<br>
>> >> >> >>> >> >> done, I<br>
>> >> >> >>> >> >> think<br>
>> >> >> >>> >> >> we can try to transition to 3.0 for development.<br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >> Here's the current list, which may need curation a bit as<br>
>> >> >> >>> >> >> some<br>
>> >> >> >>> >> >> seem<br>
>> >> >> >>> >> >> to<br>
>> >> >> >>> >> >> be completed or in progress:<br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >> <a href="https://bitbucket.org/yt_analysis/yt/issues?status=new&status=open&milestone=2.5" target="_blank">https://bitbucket.org/yt_analysis/yt/issues?status=new&status=open&milestone=2.5</a><br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >> If you want to subdivide something, create a new milestone<br>
>> >> >> >>> >> >> and<br>
>> >> >> >>> >> >> target<br>
>> >> >> >>> >> >> *that*, but with *version* 2.5.<br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >> -Matt<br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >> PS The new bitbucket redesign is quite nice!<br>
>> >> >> >>> >> >> _______________________________________________<br>
>> >> >> >>> >> >> yt-dev mailing list<br>
>> >> >> >>> >> >> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> >>> >> >><br>
>> >> >> >>> >> >> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> >> >>> >> > _______________________________________________<br>
>> >> >> >>> >> > yt-dev mailing list<br>
>> >> >> >>> >> > <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> >>> >> ><br>
>> >> >> >>> >> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> >> >>> >> _______________________________________________<br>
>> >> >> >>> >> yt-dev mailing list<br>
>> >> >> >>> >> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> >>> >> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> >> >>> ><br>
>> >> >> >>> ><br>
>> >> >> >>> ><br>
>> >> >> >>> > _______________________________________________<br>
>> >> >> >>> > yt-dev mailing list<br>
>> >> >> >>> > <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> >>> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> >> >>> ><br>
>> >> >> >>> _______________________________________________<br>
>> >> >> >>> yt-dev mailing list<br>
>> >> >> >>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> >>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> >> >><br>
>> >> >> >><br>
>> >> >> >><br>
>> >> >> >> _______________________________________________<br>
>> >> >> >> yt-dev mailing list<br>
>> >> >> >> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> >> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> >> >><br>
>> >> >> ><br>
>> >> >> ><br>
>> >> >> > _______________________________________________<br>
>> >> >> > yt-dev mailing list<br>
>> >> >> > <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> >> ><br>
>> >> >> _______________________________________________<br>
>> >> >> yt-dev mailing list<br>
>> >> >> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> >> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> ><br>
>> >> ><br>
>> >> ><br>
>> >> ><br>
>> >> > --<br>
>> >> > *********************************************************************<br>
>> >> > * *<br>
>> >> > * Dr. Michael Kuhlen Theoretical Astrophysics Center *<br>
>> >> > * email: <a href="mailto:mqk@astro.berkeley.edu">mqk@astro.berkeley.edu</a> UC Berkeley *<br>
>> >> > * cell phone: <a href="tel:%28831%29%20588-1468" value="+18315881468">(831) 588-1468</a> B-116 Hearst Field Annex # 3411 *<br>
>> >> > * skype username: mikekuhlen Berkeley, CA 94720 *<br>
>> >> > * *<br>
>> >> > *********************************************************************<br>
>> >> ><br>
>> >> ><br>
>> >> > _______________________________________________<br>
>> >> > yt-dev mailing list<br>
>> >> > <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> >> ><br>
>> >> _______________________________________________<br>
>> >> yt-dev mailing list<br>
>> >> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> >> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > yt-dev mailing list<br>
>> > <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>> ><br>
>> _______________________________________________<br>
>> yt-dev mailing list<br>
>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> yt-dev mailing list<br>
> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
><br>
_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
</div></div></blockquote></div><br>