Hey Matt.<div><br></div><div>I think Britton suggested something like this too. Something like units="unknown" that works like nan, but for units. So whenever you combine a YTArray with valid units and another YTArray with "unknown" units, the valid units get clobbered into this "unknown" state. Is this something we need to implement?</div>
<div><br></div><div>- Casey</div><div><br><br><div class="gmail_quote">On Tue, Mar 12, 2013 at 11:13 AM, Matthew Turk <span dir="ltr"><<a href="mailto:matthewturk@gmail.com" target="_blank">matthewturk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Tue, Mar 12, 2013 at 2:09 PM, Nathan Goldbaum <<a href="mailto:nathan12343@gmail.com">nathan12343@gmail.com</a>> wrote:<br>
>> It's probably also worth noting that the *vast* majority of times this<br>
>> will be used in yt will be from within a call to __getitem__, where I<br>
>> think we wrap with units anyway, right?<br>
><br>
> Right, we already wrap the units to create the YTArray in __getitem__. While most of the YTArrays will be created there, we need to make sure that math operations work as well, leading to the complications Casey is bringing up.<br>
<br>
</div>Yes, agreed. One point I think I didn't make at the workshop that I<br>
wanted to is that there should be a mechanism of noting that a YTArray<br>
is in an "unknown state", or an easy way of stripping the units. This<br>
would enable people to continue to do relatively nasty things, if they<br>
really want to. (And some people, myself included, will want to and<br>
will get frustrated if we are hindered from doing so.)<br>
<div class="HOEnZb"><div class="h5"><br>
><br>
> On Mar 12, 2013, at 11:03 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>> wrote:<br>
><br>
>> On Tue, Mar 12, 2013 at 2:01 PM, Nathan Goldbaum <<a href="mailto:nathan12343@gmail.com">nathan12343@gmail.com</a>> wrote:<br>
>>> Hi Casey,<br>
>>><br>
>>> Hmm, I'm not sure and like Matt suggests it might be worth enquiring on the<br>
>>> numpy list or doing some digging for other subclasses of NDArray.<br>
>>><br>
>>> I worry about doing to much in __array_wrap__ and __array_finalize__ because<br>
>>> of the nonuniform way the context keyword is populated by different numpy<br>
>>> operations. If there is an exhaustive list of operations we can look up or<br>
>>> generate, that might be enough to engineer a solution around that.<br>
>><br>
>> I like that solution -- is there a spec?<br>
>><br>
>> Also, it seems to me that my first reply was perhaps misguided -- I<br>
>> like Britton's point about addressing this in the array_wrap.<br>
>><br>
>> It's probably also worth noting that the *vast* majority of times this<br>
>> will be used in yt will be from within a call to __getitem__, where I<br>
>> think we wrap with units anyway, right?<br>
>><br>
>>><br>
>>> -Nathan<br>
>>><br>
>>> On Mar 12, 2013, at 10:20 AM, "Casey W. Stark" <<a href="mailto:caseywstark@gmail.com">caseywstark@gmail.com</a>><br>
>>> wrote:<br>
>>><br>
>>> Hi all.<br>
>>><br>
>>> I'm writing the YTEP and I came across a technical problem.<br>
>>><br>
>>> For the operation overloading, I realized that we might be able/have to<br>
>>> remove the __op__ methods and do everything inside the __array_wrap__<br>
>>> method. Currently if you call `a + b` (where they're YTArray objects), it<br>
>>> will check the units and make sure that's valid. But if you call np.add(a,<br>
>>> b), it goes straight to __array_wrap__ without checking units!<br>
>>><br>
>>> Maybe we can have two ufunc registries -- one for unit conversion functions<br>
>>> and one for unit check functions?<br>
>>><br>
>>> - Casey<br>
>>><br>
>>><br>
>>> On Mon, Mar 11, 2013 at 11:28 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>>> wrote:<br>
>>>><br>
>>>> Hi Doug,<br>
>>>><br>
>>>> On Mon, Mar 11, 2013 at 1:40 PM, Douglas Harvey Rudd <<a href="mailto:drudd@uchicago.edu">drudd@uchicago.edu</a>><br>
>>>> wrote:<br>
>>>>> Hi Matt,<br>
>>>>><br>
>>>>> Let us know when you'd like us to push ARTIO changes to the mainline.<br>
>>>>> It's still<br>
>>>>> being actively worked on, but the main features should work, and we<br>
>>>>> really would like<br>
>>>>> to get the new code into the first 3.0 alpha so we can push our users to<br>
>>>>> start testing.<br>
>>>><br>
>>>> Awesome! I think I will have time to read and test them in detail<br>
>>>> probably starting Wednesday, but I suspect that they're probably in a<br>
>>>> good state already to go in. I'll ping you once I've put in a handful<br>
>>>> of other changes I'm looking at.<br>
>>>><br>
>>>> Incidentally, I'm planning to put all this on a blog post about the<br>
>>>> workshop, but I wanted to take this opportunity to say how cool it was<br>
>>>> to see over the course of the few days the transition for ARTIO to a<br>
>>>> frontend based on opaque data objects.<br>
>>>><br>
>>>> -Matt<br>
>>>><br>
>>>>><br>
>>>>> Douglas Rudd<br>
>>>>> Scientific Computing Consultant<br>
>>>>> Research Computing Center<br>
>>>>> <a href="mailto:drudd@uchicago.edu">drudd@uchicago.edu</a><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> On Mar 11, 2013, at 12:21 PM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>>>>> wrote:<br>
>>>>><br>
>>>>>> On Mon, Mar 11, 2013 at 1:15 PM, Nathan Goldbaum<br>
>>>>>> <<a href="mailto:nathan12343@gmail.com">nathan12343@gmail.com</a>> wrote:<br>
>>>>>>> Hi Matt,<br>
>>>>>>><br>
>>>>>>> I think the steps you outline are a good start. My plan to continue<br>
>>>>>>> the<br>
>>>>>>> units work was to try to get the unit tests passing and see what needs<br>
>>>>>>> to<br>
>>>>>>> get done as that process proceeds. I'm glad to hear you're excited<br>
>>>>>>> enough<br>
>>>>>>> about this to get started on that work.<br>
>>>>>><br>
>>>>>> Absolutely. On the plane I was able to get it down to 193 fails, out<br>
>>>>>> of ~2600 or so. The biggest issues seem to be a problem with how the<br>
>>>>>> broadcasting is working (as evidenced in the radial velocity fields),<br>
>>>>>> things like Baroclinic vorticity disagreeing with how we're changing<br>
>>>>>> units, and some misc other problems. I've pushed my changes up to<br>
>>>>>> Casey's fork.<br>
>>>>>><br>
>>>>>>><br>
>>>>>>> At the moment the work of subclassing NDArray is probably ~75%<br>
>>>>>>> complete. To<br>
>>>>>>> finish we'll need to make sure we cover all of the possible ufuncs and<br>
>>>>>>> do<br>
>>>>>>> appropriate operations in all cases. Having a formal spec will help<br>
>>>>>>> in that<br>
>>>>>>> effort. There are also a couple of odd issues that still need to be<br>
>>>>>>> worked<br>
>>>>>>> out to make the code cleaner, i.e. the issue referenced in fce1e6c.<br>
>>>>>><br>
>>>>>> I definitely think we need to address the ufuncs through design.<br>
>>>>>> There are several issues I'm concerned about, specifically those cases<br>
>>>>>> where we do *not* want to return a subclass of YTArray. As an<br>
>>>>>> example, any of the ufuncs that return logical operators. These could<br>
>>>>>> in principle return dimensionless (which is what I have done) but I<br>
>>>>>> think I'm more comfortable returning just standard arrays . I've also<br>
>>>>>> had to put in what I think is something of a hack in returning scalars<br>
>>>>>> when a ufunc drops to a single value.<br>
>>>>>><br>
>>>>>>><br>
>>>>>>> Right now all of this work is going on in Casey's yt-3.0 fork:<br>
>>>>>>> <a href="https://bitbucket.org/caseywstark/yt-3.0" target="_blank">https://bitbucket.org/caseywstark/yt-3.0</a>. Last week we were just<br>
>>>>>>> pushing<br>
>>>>>>> directly to the fork but I think now that we're all in different time<br>
>>>>>>> zones<br>
>>>>>>> it's probably a good idea to process further changes via pull requests<br>
>>>>>>> to<br>
>>>>>>> that fork. Once everything is ready, Casey will be able to issue a<br>
>>>>>>> pull<br>
>>>>>>> request to the yt_analysis/yt-3.0 repository. Does anyone disagree<br>
>>>>>>> with<br>
>>>>>>> that idea?<br>
>>>>>><br>
>>>>>> Yes, I think it should live separately for a while -- it's<br>
>>>>>> *incredibly* invasive, and I'd like to roll out 3.0a1 on the 15th. I<br>
>>>>>> think we should wait until the tagging of 3.0a1 to do this. I'm also<br>
>>>>>> a bit concerned that there's heavy development on the ART and ARTIO<br>
>>>>>> fronts that has not yet been pulled back in to mainline, and I want to<br>
>>>>>> try to pull that in before this change. I'm not trying to put the<br>
>>>>>> brakes on the development, just that I think we need to take this a<br>
>>>>>> bit carefully since it's quite invasive and we now have a set of<br>
>>>>>> people using 3.0 for their work. Maybe we could aim to have the YTEP<br>
>>>>>> done, and then see how ready the code base is?<br>
>>>>>><br>
>>>>>>><br>
>>>>>>> Cheers,<br>
>>>>>>><br>
>>>>>>> Nathan<br>
>>>>>>><br>
>>>>>>> On Mar 11, 2013, at 9:09 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>>>>>>> wrote:<br>
>>>>>>><br>
>>>>>>> Hi all,<br>
>>>>>>><br>
>>>>>>> Last week the units stuff really took off -- great work on that front!<br>
>>>>>>> I was wondering if I might solicit a YTEP? As I have been going<br>
>>>>>>> through the YTArray changes to get the unit tests passing, I'm<br>
>>>>>>> realizing just how invasive it is in some ways. I think having an<br>
>>>>>>> enumeration of the behavior of the YTArray under ufuncs as well as<br>
>>>>>>> common operations will be pretty essential. That is something we<br>
>>>>>>> should also probably discuss on this list.<br>
>>>>>>><br>
>>>>>>> I'll be submitting a few changes for review by Casey and Nathan as<br>
>>>>>>> well. Do either of you have a feeling for where things stand now and<br>
>>>>>>> what is needed next?<br>
>>>>>>><br>
>>>>>>> -Matt<br>
>>>>>>> _______________________________________________<br>
>>>>>>> yt-dev mailing list<br>
>>>>>>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>>>>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>>>>>><br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> _______________________________________________<br>
>>>>>>> yt-dev mailing list<br>
>>>>>>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>>>>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>>>>>><br>
>>>>>> _______________________________________________<br>
>>>>>> yt-dev mailing list<br>
>>>>>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>>>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> yt-dev mailing list<br>
>>>>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>>> _______________________________________________<br>
>>>> yt-dev mailing list<br>
>>>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> yt-dev mailing list<br>
>>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>><br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> yt-dev mailing list<br>
>>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>><br>
>> _______________________________________________<br>
>> yt-dev mailing list<br>
>> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
><br>
> _______________________________________________<br>
> yt-dev mailing list<br>
> <a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
</div></div></blockquote></div><br></div>