[yt-users] NetCDF autoloader

[Marianne Corvellec]

Wonderful!  Thanks for sharing the vision, Nathan.

I'll be investigating and reporting issues; hopefully I can spend more
time on this starting next month.  Will there be a yt sprint at SciPy
2017?

For the sake of context, the precipitation data I'm playing with looks
like this:
https://github.com/Ouranosinc/data-visualization/blob/master/year_slider.ipynb

Cheers,
Marianne

On Fri, Mar 10, 2017 at 4:17 PM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
>
>
> On Fri, Mar 10, 2017 at 1:04 PM, Marianne Corvellec
> <marianne.corvellec at gmail.com> wrote:
>>
>> Hi Kacper!
>>
>> Thank you very much.
>>
>> I was able to run the following http://paste.yt-project.org/show/7081/
>> on my dataset, adapting the following from your script:
>>
>> * I took a slice of the data, keeping only 2 days out of the 366 (this
>> is variable 'time' renamed into 'altitude').  If I kept all days, my
>> memory would completely fill up (15.5G), which would kill my kernel.
>> With 2 days, memory fills up to ~5G...  I should study what
>> ProjectionPlot() does exactly.
>>
>> * Note that SlicePlot() yields exactly the same output, i.e., this
>> unexpected projection (plot attached).  Boundaries for x and y don't
>> match `bbox`
>>
>> array([[ 3652.        ,  4017.        ],
>>        [   41.04166794,    83.45832825],
>>        [ -140.95832825,   -52.04167175]], dtype=float32)
>>
>> these values being output again by yt in the stream.
>>
>> * I removed `ds.surface_height = 0 * ds.domain_width.uq` which doesn't
>> seem to make a difference, and set `field = 'pr'` (which stands for
>> precipitation).
>
>
> Cool!
>
> There is still lots of work to do to make working with these sorts of data
> "nicer". We're very interested in hearing about the issues you run into as
> adding first-class support for these sorts of data will become a big focus
> in the next few years.
>
>>
>>
>> Cheers,
>> Marianne
>>
>> PS: I can't run the initial script
>> (http://paste.yt-project.org/show/7068/) readily on "cm1out_000013.nc"
>> (getting KeyError: 'ni' line 26; a quick glance at $ ncdump -h
>> cm1out_000013.nc shows float xh(ni) among the variables, not ni(ni)...
>> not that I'm super familiar with netcdf data myself).
>>
>> On Thu, Mar 9, 2017 at 3:27 PM, Kacper Kowalik <xarthisius.kk at gmail.com>
>> wrote:
>> > On 03/09/2017 12:49 PM, Marianne Corvellec wrote:
>> >> Hi Kacper,
>> >>
>> >> Thank you so much for your reply!!
>> >>
>> >> This script is a great starting point for me.
>> >>
>> >> I'm trying to adapt it to this dataset
>> >>
>> >> http://outarde.crim.ca:8083/thredds/catalog/birdhouse/nrcan/nrcan_canada_daily/catalog.html?dataset=birdhouse/nrcan/nrcan_canada_daily/nrcan_canada_daily_pr_1960.nc
>> >> and I've reached line 47 -- could I find your example file
>> >> ('cm1out_000013.nc') somewhere to help me troubleshoot?
>> >>
>> >> Thanks again,
>> >> Marianne
>> >
>> > Hi Marianne,
>> > the file can be downloaded from here http://use.yt/upload/06f23b0b
>> > I modified the original script slightly:
>> > http://paste.yt-project.org/show/7074/
>> > it should at least show something with your data.
>> > However, it also shows how crude and inefficient the "autodetection"
>> > is...
>> > Cheers,
>> > Kacper
>> >
>> >> On Sat, Mar 4, 2017 at 6:11 PM, Kacper Kowalik
>> >> <xarthisius.kk at gmail.com> wrote:
>> >>> On 03/02/2017 01:19 PM, Marianne Corvellec wrote:
>> >>>> Dear yt community,
>> >>>>
>> >>>> I am relatively new to this list and this is my first time posting.
>> >>>> I'm still getting acquainted with the yt library.  At least now I'm
>> >>>> enjoying a fully functional setup locally (something which had been
>> >>>> on
>> >>>> my to-do list).
>> >>>>
>> >>>> I've had an informal chat with Matt Turk about using yt in
>> >>>> climatology.  He mentioned some folks were working on a NetCDF
>> >>>> autoloader.  I would love to connect with these folks, and possibly
>> >>>> help out.
>> >>>>
>> >>>> Thank you!
>> >>>> Marianne
>> >>>> _______________________________________________
>> >>>> yt-users mailing list
>> >>>> yt-users at lists.spacepope.org
>> >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >>>
>> >>> Hi Marianne,
>> >>> I played a bit with the idea of autoloading NetCDFs into yt. Once upon
>> >>> a
>> >>> time we tried to create an interactive visualization of a climate
>> >>> dataset that was handed to us. In the process we came up with a simple
>> >>> script that could be used to read netcdf and feed yt with the data:
>> >>>
>> >>> http://paste.yt-project.org/show/7068/
>> >>>
>> >>> That's just a simple hack, rather than something "production" ready.
>> >>> However, If you'd like to create a proper frontend for generic
>> >>> climate/geo data I'd love to help.
>> >>> Cheers,
>> >>> Kacper
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> yt-users mailing list
>> >>> yt-users at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >>>
>> >> _______________________________________________
>> >> yt-users mailing list
>> >> yt-users at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >>
>> >
>> >
>> >
>> > _______________________________________________
>> > yt-users mailing list
>> > yt-users at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>