[yt-users] NetCDF autoloader

Nathan Goldbaum nathan12343 at gmail.com
Fri Mar 10 14:53:41 PST 2017 

On Fri, Mar 10, 2017 at 2:48 PM Marianne Corvellec <
marianne.corvellec at gmail.com> wrote:

> Wonderful!  Thanks for sharing the vision, Nathan.
>
> I'll be investigating and reporting issues; hopefully I can spend more
> time on this starting next month.  Will there be a yt sprint at SciPy
> 2017?


Yes, although I'm worried about the transgender bathroom bill in the Texas
legislature.


>
> For the sake of context, the precipitation data I'm playing with looks
> like this:
>
> https://github.com/Ouranosinc/data-visualization/blob/master/year_slider.ipynb
>
> Cheers,
> Marianne
>
> On Fri, Mar 10, 2017 at 4:17 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
> >
> >
> > On Fri, Mar 10, 2017 at 1:04 PM, Marianne Corvellec
> > <marianne.corvellec at gmail.com> wrote:
> >>
> >> Hi Kacper!
> >>
> >> Thank you very much.
> >>
> >> I was able to run the following http://paste.yt-project.org/show/7081/
> >> on my dataset, adapting the following from your script:
> >>
> >> * I took a slice of the data, keeping only 2 days out of the 366 (this
> >> is variable 'time' renamed into 'altitude').  If I kept all days, my
> >> memory would completely fill up (15.5G), which would kill my kernel.
> >> With 2 days, memory fills up to ~5G...  I should study what
> >> ProjectionPlot() does exactly.
> >>
> >> * Note that SlicePlot() yields exactly the same output, i.e., this
> >> unexpected projection (plot attached).  Boundaries for x and y don't
> >> match `bbox`
> >>
> >> array([[ 3652.        ,  4017.        ],
> >>        [   41.04166794,    83.45832825],
> >>        [ -140.95832825,   -52.04167175]], dtype=float32)
> >>
> >> these values being output again by yt in the stream.
> >>
> >> * I removed `ds.surface_height = 0 * ds.domain_width.uq` which doesn't
> >> seem to make a difference, and set `field = 'pr'` (which stands for
> >> precipitation).
> >
> >
> > Cool!
> >
> > There is still lots of work to do to make working with these sorts of
> data
> > "nicer". We're very interested in hearing about the issues you run into
> as
> > adding first-class support for these sorts of data will become a big
> focus
> > in the next few years.
> >
> >>
> >>
> >> Cheers,
> >> Marianne
> >>
> >> PS: I can't run the initial script
> >> (http://paste.yt-project.org/show/7068/) readily on "cm1out_000013.nc"
> >> (getting KeyError: 'ni' line 26; a quick glance at $ ncdump -h
> >> cm1out_000013.nc shows float xh(ni) among the variables, not ni(ni)...
> >> not that I'm super familiar with netcdf data myself).
> >>
> >> On Thu, Mar 9, 2017 at 3:27 PM, Kacper Kowalik <xarthisius.kk at gmail.com
> >
> >> wrote:
> >> > On 03/09/2017 12:49 PM, Marianne Corvellec wrote:
> >> >> Hi Kacper,
> >> >>
> >> >> Thank you so much for your reply!!
> >> >>
> >> >> This script is a great starting point for me.
> >> >>
> >> >> I'm trying to adapt it to this dataset
> >> >>
> >> >>
> http://outarde.crim.ca:8083/thredds/catalog/birdhouse/nrcan/nrcan_canada_daily/catalog.html?dataset=birdhouse/nrcan/nrcan_canada_daily/nrcan_canada_daily_pr_1960.nc
> >> >> and I've reached line 47 -- could I find your example file
> >> >> ('cm1out_000013.nc') somewhere to help me troubleshoot?
> >> >>
> >> >> Thanks again,
> >> >> Marianne
> >> >
> >> > Hi Marianne,
> >> > the file can be downloaded from here http://use.yt/upload/06f23b0b
> >> > I modified the original script slightly:
> >> > http://paste.yt-project.org/show/7074/
> >> > it should at least show something with your data.
> >> > However, it also shows how crude and inefficient the "autodetection"
> >> > is...
> >> > Cheers,
> >> > Kacper
> >> >
> >> >> On Sat, Mar 4, 2017 at 6:11 PM, Kacper Kowalik
> >> >> <xarthisius.kk at gmail.com> wrote:
> >> >>> On 03/02/2017 01:19 PM, Marianne Corvellec wrote:
> >> >>>> Dear yt community,
> >> >>>>
> >> >>>> I am relatively new to this list and this is my first time posting.
> >> >>>> I'm still getting acquainted with the yt library.  At least now I'm
> >> >>>> enjoying a fully functional setup locally (something which had been
> >> >>>> on
> >> >>>> my to-do list).
> >> >>>>
> >> >>>> I've had an informal chat with Matt Turk about using yt in
> >> >>>> climatology.  He mentioned some folks were working on a NetCDF
> >> >>>> autoloader.  I would love to connect with these folks, and possibly
> >> >>>> help out.
> >> >>>>
> >> >>>> Thank you!
> >> >>>> Marianne
> >> >>>> _______________________________________________
> >> >>>> yt-users mailing list
> >> >>>> yt-users at lists.spacepope.org
> >> >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >> >>>
> >> >>> Hi Marianne,
> >> >>> I played a bit with the idea of autoloading NetCDFs into yt. Once
> upon
> >> >>> a
> >> >>> time we tried to create an interactive visualization of a climate
> >> >>> dataset that was handed to us. In the process we came up with a
> simple
> >> >>> script that could be used to read netcdf and feed yt with the data:
> >> >>>
> >> >>> http://paste.yt-project.org/show/7068/
> >> >>>
> >> >>> That's just a simple hack, rather than something "production" ready.
> >> >>> However, If you'd like to create a proper frontend for generic
> >> >>> climate/geo data I'd love to help.
> >> >>> Cheers,
> >> >>> Kacper
> >> >>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> yt-users mailing list
> >> >>> yt-users at lists.spacepope.org
> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >> >>>
> >> >> _______________________________________________
> >> >> yt-users mailing list
> >> >> yt-users at lists.spacepope.org
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >> >>
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > yt-users mailing list
> >> > yt-users at lists.spacepope.org
> >> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >> >
> >>
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>
> >
> >
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20170310/31e6aa4d/attachment.html>

More information about the yt-users mailing list