[yt-dev] YTArray YTEP

Tue Mar 12 10:50:16 PDT 2013

Hi Casey,

On Tue, Mar 12, 2013 at 1:20 PM, Casey W. Stark <caseywstark at gmail.com> wrote:
> Hi all.
>
> I'm writing the YTEP and I came across a technical problem.
>
> For the operation overloading, I realized that we might be able/have to
> remove the __op__ methods and do everything inside the __array_wrap__
> method. Currently if you call `a + b` (where they're YTArray objects), it
> will check the units and make sure that's valid. But if you call np.add(a,
> b), it goes straight to __array_wrap__ without checking units!
>
> Maybe we can have two ufunc registries -- one for unit conversion functions
> and one for unit check functions?

I'll be honest with you: I don't know enough about array wrapping to
have an answer to this, and my *guess* is that you and Nathan are the
best equipped to figure this one out.  This might be a good reason to
look at other implementations of array wrapping (there are, I think,
even units-ful array wrappings) and maybe even query the numpy list.
Sorry not to have the answer, I am just worried we might give the
wrong one.

-Matt

>
> - Casey
>
>
> On Mon, Mar 11, 2013 at 11:28 AM, Matthew Turk <matthewturk at gmail.com>
> wrote:
>>
>> Hi Doug,
>>
>> On Mon, Mar 11, 2013 at 1:40 PM, Douglas Harvey Rudd <drudd at uchicago.edu>
>> wrote:
>> > Hi Matt,
>> >
>> > Let us know when you'd like us to push ARTIO changes to the mainline.
>> > It's still
>> > being actively worked on, but the main features should work, and we
>> > really would like
>> > to get the new code into the first 3.0 alpha so we can push our users to
>> > start testing.
>>
>> Awesome!  I think I will have time to read and test them in detail
>> probably starting Wednesday, but I suspect that they're probably in a
>> good state already to go in.  I'll ping you once I've put in a handful
>> of other changes I'm looking at.
>>
>> Incidentally, I'm planning to put all this on a blog post about the
>> workshop, but I wanted to take this opportunity to say how cool it was
>> to see over the course of the few days the transition for ARTIO to a
>> frontend based on opaque data objects.
>>
>> -Matt
>>
>> >
>> > Douglas Rudd
>> > Scientific Computing Consultant
>> > Research Computing Center
>> > drudd at uchicago.edu
>> >
>> >
>> >
>> > On Mar 11, 2013, at 12:21 PM, Matthew Turk <matthewturk at gmail.com>
>> > wrote:
>> >
>> >> On Mon, Mar 11, 2013 at 1:15 PM, Nathan Goldbaum
>> >> <nathan12343 at gmail.com> wrote:
>> >>> Hi Matt,
>> >>>
>> >>> I think the steps you outline are a good start.  My plan to continue
>> >>> the
>> >>> units work was to try to get the unit tests passing and see what needs
>> >>> to
>> >>> get done as that process proceeds.  I'm glad to hear you're excited
>> >>> enough
>> >>> about this to get started on that work.
>> >>
>> >> Absolutely.  On the plane I was able to get it down to 193 fails, out
>> >> of ~2600 or so.  The biggest issues seem to be a problem with how the
>> >> broadcasting is working (as evidenced in the radial velocity fields),
>> >> things like Baroclinic vorticity disagreeing with how we're changing
>> >> units, and some misc other problems.  I've pushed my changes up to
>> >> Casey's fork.
>> >>
>> >>>
>> >>> At the moment the work of subclassing NDArray is probably ~75%
>> >>> complete.  To
>> >>> finish we'll need to make sure we cover all of the possible ufuncs and
>> >>> do
>> >>> appropriate operations in all cases.  Having a formal spec will help
>> >>> in that
>> >>> effort.  There are also a couple of odd issues that still need to be
>> >>> worked
>> >>> out to make the code cleaner, i.e. the issue referenced in fce1e6c.
>> >>
>> >> I definitely think we need to address the ufuncs through design.
>> >> There are several issues I'm concerned about, specifically those cases
>> >> where we do *not* want to return a subclass of YTArray.  As an
>> >> example, any of the ufuncs that return logical operators.  These could
>> >> in principle return dimensionless (which is what I have done) but I
>> >> think I'm more comfortable returning just standard arrays . I've also
>> >> had to put in what I think is something of a hack in returning scalars
>> >> when a ufunc drops to a single value.
>> >>
>> >>>
>> >>> Right now all of this work is going on in Casey's yt-3.0 fork:
>> >>> https://bitbucket.org/caseywstark/yt-3.0.  Last week we were just
>> >>> pushing
>> >>> directly to the fork but I think now that we're all in different time
>> >>> zones
>> >>> it's probably a good idea to process further changes via pull requests
>> >>> to
>> >>> that fork.  Once everything is ready, Casey will be able to issue a
>> >>> pull
>> >>> request to the yt_analysis/yt-3.0 repository.  Does anyone disagree
>> >>> with
>> >>> that idea?
>> >>
>> >> Yes, I think it should live separately for a while -- it's
>> >> *incredibly* invasive, and I'd like to roll out 3.0a1 on the 15th.  I
>> >> think we should wait until the tagging of 3.0a1 to do this.  I'm also
>> >> a bit concerned that there's heavy development on the ART and ARTIO
>> >> fronts that has not yet been pulled back in to mainline, and I want to
>> >> try to pull that in before this change.  I'm not trying to put the
>> >> brakes on the development, just that I think we need to take this a
>> >> bit carefully since it's quite invasive and we now have a set of
>> >> people using 3.0 for their work.  Maybe we could aim to have the YTEP
>> >> done, and then see how ready the code base is?
>> >>
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Nathan
>> >>>
>> >>> On Mar 11, 2013, at 9:09 AM, Matthew Turk <matthewturk at gmail.com>
>> >>> wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>> Last week the units stuff really took off -- great work on that front!
>> >>> I was wondering if I might solicit a YTEP?  As I have been going
>> >>> through the YTArray changes to get the unit tests passing, I'm
>> >>> realizing just how invasive it is in some ways.  I think having an
>> >>> enumeration of the behavior of the YTArray under ufuncs as well as
>> >>> common operations will be pretty essential.  That is something we
>> >>> should also probably discuss on this list.
>> >>>
>> >>> I'll be submitting a few changes for review by Casey and Nathan as
>> >>> well.  Do either of you have a feeling for where things stand now and
>> >>> what is needed next?
>> >>>
>> >>> -Matt
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>