[yt-dev] YTArray YTEP

Tue Mar 12 10:58:16 PDT 2013

Hey Casey,

YTEP-0014 looks awesome so far!  (Sidenote, I talked about this work
at our pizza lunch today and several people were interested in it. :)

-Matt

On Tue, Mar 12, 2013 at 1:26 PM, Casey W. Stark <caseywstark at gmail.com> wrote:
> I just pushed a start to YTEP-0014 to my fork btw:
> https://bitbucket.org/caseywstark/ytep.
>
>
> On Tue, Mar 12, 2013 at 10:20 AM, Casey W. Stark <caseywstark at gmail.com>
> wrote:
>>
>> Hi all.
>>
>> I'm writing the YTEP and I came across a technical problem.
>>
>> For the operation overloading, I realized that we might be able/have to
>> remove the __op__ methods and do everything inside the __array_wrap__
>> method. Currently if you call `a + b` (where they're YTArray objects), it
>> will check the units and make sure that's valid. But if you call np.add(a,
>> b), it goes straight to __array_wrap__ without checking units!
>>
>> Maybe we can have two ufunc registries -- one for unit conversion
>> functions and one for unit check functions?
>>
>> - Casey
>>
>>
>> On Mon, Mar 11, 2013 at 11:28 AM, Matthew Turk <matthewturk at gmail.com>
>> wrote:
>>>
>>> Hi Doug,
>>>
>>> On Mon, Mar 11, 2013 at 1:40 PM, Douglas Harvey Rudd <drudd at uchicago.edu>
>>> wrote:
>>> > Hi Matt,
>>> >
>>> > Let us know when you'd like us to push ARTIO changes to the mainline.
>>> > It's still
>>> > being actively worked on, but the main features should work, and we
>>> > really would like
>>> > to get the new code into the first 3.0 alpha so we can push our users
>>> > to start testing.
>>>
>>> Awesome!  I think I will have time to read and test them in detail
>>> probably starting Wednesday, but I suspect that they're probably in a
>>> good state already to go in.  I'll ping you once I've put in a handful
>>> of other changes I'm looking at.
>>>
>>> Incidentally, I'm planning to put all this on a blog post about the
>>> workshop, but I wanted to take this opportunity to say how cool it was
>>> to see over the course of the few days the transition for ARTIO to a
>>> frontend based on opaque data objects.
>>>
>>> -Matt
>>>
>>> >
>>> > Douglas Rudd
>>> > Scientific Computing Consultant
>>> > Research Computing Center
>>> > drudd at uchicago.edu
>>> >
>>> >
>>> >
>>> > On Mar 11, 2013, at 12:21 PM, Matthew Turk <matthewturk at gmail.com>
>>> > wrote:
>>> >
>>> >> On Mon, Mar 11, 2013 at 1:15 PM, Nathan Goldbaum
>>> >> <nathan12343 at gmail.com> wrote:
>>> >>> Hi Matt,
>>> >>>
>>> >>> I think the steps you outline are a good start.  My plan to continue
>>> >>> the
>>> >>> units work was to try to get the unit tests passing and see what
>>> >>> needs to
>>> >>> get done as that process proceeds.  I'm glad to hear you're excited
>>> >>> enough
>>> >>> about this to get started on that work.
>>> >>
>>> >> Absolutely.  On the plane I was able to get it down to 193 fails, out
>>> >> of ~2600 or so.  The biggest issues seem to be a problem with how the
>>> >> broadcasting is working (as evidenced in the radial velocity fields),
>>> >> things like Baroclinic vorticity disagreeing with how we're changing
>>> >> units, and some misc other problems.  I've pushed my changes up to
>>> >> Casey's fork.
>>> >>
>>> >>>
>>> >>> At the moment the work of subclassing NDArray is probably ~75%
>>> >>> complete.  To
>>> >>> finish we'll need to make sure we cover all of the possible ufuncs
>>> >>> and do
>>> >>> appropriate operations in all cases.  Having a formal spec will help
>>> >>> in that
>>> >>> effort.  There are also a couple of odd issues that still need to be
>>> >>> worked
>>> >>> out to make the code cleaner, i.e. the issue referenced in fce1e6c.
>>> >>
>>> >> I definitely think we need to address the ufuncs through design.
>>> >> There are several issues I'm concerned about, specifically those cases
>>> >> where we do *not* want to return a subclass of YTArray.  As an
>>> >> example, any of the ufuncs that return logical operators.  These could
>>> >> in principle return dimensionless (which is what I have done) but I
>>> >> think I'm more comfortable returning just standard arrays . I've also
>>> >> had to put in what I think is something of a hack in returning scalars
>>> >> when a ufunc drops to a single value.
>>> >>
>>> >>>
>>> >>> Right now all of this work is going on in Casey's yt-3.0 fork:
>>> >>> https://bitbucket.org/caseywstark/yt-3.0.  Last week we were just
>>> >>> pushing
>>> >>> directly to the fork but I think now that we're all in different time
>>> >>> zones
>>> >>> it's probably a good idea to process further changes via pull
>>> >>> requests to
>>> >>> that fork.  Once everything is ready, Casey will be able to issue a
>>> >>> pull
>>> >>> request to the yt_analysis/yt-3.0 repository.  Does anyone disagree
>>> >>> with
>>> >>> that idea?
>>> >>
>>> >> Yes, I think it should live separately for a while -- it's
>>> >> *incredibly* invasive, and I'd like to roll out 3.0a1 on the 15th.  I
>>> >> think we should wait until the tagging of 3.0a1 to do this.  I'm also
>>> >> a bit concerned that there's heavy development on the ART and ARTIO
>>> >> fronts that has not yet been pulled back in to mainline, and I want to
>>> >> try to pull that in before this change.  I'm not trying to put the
>>> >> brakes on the development, just that I think we need to take this a
>>> >> bit carefully since it's quite invasive and we now have a set of
>>> >> people using 3.0 for their work.  Maybe we could aim to have the YTEP
>>> >> done, and then see how ready the code base is?
>>> >>
>>> >>>
>>> >>> Cheers,
>>> >>>
>>> >>> Nathan
>>> >>>
>>> >>> On Mar 11, 2013, at 9:09 AM, Matthew Turk <matthewturk at gmail.com>
>>> >>> wrote:
>>> >>>
>>> >>> Hi all,
>>> >>>
>>> >>> Last week the units stuff really took off -- great work on that
>>> >>> front!
>>> >>> I was wondering if I might solicit a YTEP?  As I have been going
>>> >>> through the YTArray changes to get the unit tests passing, I'm
>>> >>> realizing just how invasive it is in some ways.  I think having an
>>> >>> enumeration of the behavior of the YTArray under ufuncs as well as
>>> >>> common operations will be pretty essential.  That is something we
>>> >>> should also probably discuss on this list.
>>> >>>
>>> >>> I'll be submitting a few changes for review by Casey and Nathan as
>>> >>> well.  Do either of you have a feeling for where things stand now and
>>> >>> what is needed next?
>>> >>>
>>> >>> -Matt
>>> >>> _______________________________________________
>>> >>> yt-dev mailing list
>>> >>> yt-dev at lists.spacepope.org
>>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>> >>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> yt-dev mailing list
>>> >>> yt-dev at lists.spacepope.org
>>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>> >>>
>>> >> _______________________________________________
>>> >> yt-dev mailing list
>>> >> yt-dev at lists.spacepope.org
>>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>> >
>>> > _______________________________________________
>>> > yt-dev mailing list
>>> > yt-dev at lists.spacepope.org
>>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>