[yt-dev] Get rid of hdf5_light_reader and freetype

[Britton Smith]

Hi Matt,

I'm definitely for this.  However, won't we will still have hdf5 as a
dependency since we need it to install h5py?

Britton


On Tue, Oct 8, 2013 at 9:44 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Well, I should be clear -- loading a gigantic dataset in yt-2 is
> slower than either forms of the yt-3 loading, with hdf5_light_reader
> or with h5py.  So the order goes:
>
> yt-3.0 + hdf5_light_reader (fastest)
> yt-3.0 + h5py (not bad)
> yt-2.x + hdf5_light_reader (slowest)
>
> (Also, I'd encourage anybody who wants to see "write only code" to
> take a look at hdf5_light_reader.)
>
> -Matt
>
> On Tue, Oct 8, 2013 at 4:30 PM, Cameron Hummels <chummels at gmail.com>
> wrote:
> > I'm all for it if it results in faster IO for large datasets without any
> > significant downsides for medium/small datasets.  Plus if it simplifies
> > things, that's always better too.  Thanks for looking into this, Matt!
> >
> > +1
> >
> > Cameron
> >
> >
> > On Tue, Oct 8, 2013 at 1:20 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>
> >> Hi all,
> >>
> >> One of the biggest pain points in our distribution is linking against
> >> external libraries.  We do this in three places by default:
> >>
> >>  * HDF5, for the hdf5_light_reader
> >>  * libpng, for write_png
> >>  * freetype, for freetype_writer
> >>
> >> hdf5_light_reader was originally written (by me) something like five
> >> years and change ago, because at the time PyTables was slower than I
> >> wanted.  I've been working on improving IO and found that we can get
> >> nearly the same speed from h5py; when combined with overall
> >> performance improvements in yt-3, this still translates to faster
> >> speeds for very large datasets.  On small to medium datasets, the
> >> performance is not noticeably different.  I'd like to propose that we
> >> stop using hdf5_light_reader, which may result in a slight degradation
> >> of performance for (only) Enzo IO.
> >>
> >> Additionally, I don't think *anyone* uses the freetype writer, and I'd
> >> like to remove it.  I'm actually not sure if anyone has ever used it.
> >>
> >> I think with these two changes we can likely improve our packaging
> >> situation by a good amount, as hdf5 specifically is the main problem
> >> with distribution in Conda.  These changes would only be in yt 3.0,
> >> and would not affect 2.x.
> >>
> >> [+-][01]?
> >>
> >> -Matt
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> >
> >
> >
> > --
> > Cameron Hummels
> > Postdoctoral Researcher
> > Steward Observatory
> > University of Arizona
> > http://chummels.org
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20131008/e37b8174/attachment.html>