[yt-dev] Get rid of hdf5_light_reader and freetype

[Matthew Turk]

On Tue, Oct 8, 2013 at 5:00 PM, Britton Smith <brittonsmith at gmail.com> wrote:
> Hi Matt,
>
> I'm definitely for this.  However, won't we will still have hdf5 as a
> dependency since we need it to install h5py?
>

Yup -- but it means it's an indirect dependency, and also a runtime
rather than buildtime dependency, so it means we don't need to worry
about having particular types of the library around and whatnot.  So
it'll still be in the install_script.sh, but when we use Conda we can
simplify things considerably as we can just use h5py on the system.

> Britton
>
>
> On Tue, Oct 8, 2013 at 9:44 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> Well, I should be clear -- loading a gigantic dataset in yt-2 is
>> slower than either forms of the yt-3 loading, with hdf5_light_reader
>> or with h5py.  So the order goes:
>>
>> yt-3.0 + hdf5_light_reader (fastest)
>> yt-3.0 + h5py (not bad)
>> yt-2.x + hdf5_light_reader (slowest)
>>
>> (Also, I'd encourage anybody who wants to see "write only code" to
>> take a look at hdf5_light_reader.)
>>
>> -Matt
>>
>> On Tue, Oct 8, 2013 at 4:30 PM, Cameron Hummels <chummels at gmail.com>
>> wrote:
>> > I'm all for it if it results in faster IO for large datasets without any
>> > significant downsides for medium/small datasets.  Plus if it simplifies
>> > things, that's always better too.  Thanks for looking into this, Matt!
>> >
>> > +1
>> >
>> > Cameron
>> >
>> >
>> > On Tue, Oct 8, 2013 at 1:20 PM, Matthew Turk <matthewturk at gmail.com>
>> > wrote:
>> >>
>> >> Hi all,
>> >>
>> >> One of the biggest pain points in our distribution is linking against
>> >> external libraries.  We do this in three places by default:
>> >>
>> >>  * HDF5, for the hdf5_light_reader
>> >>  * libpng, for write_png
>> >>  * freetype, for freetype_writer
>> >>
>> >> hdf5_light_reader was originally written (by me) something like five
>> >> years and change ago, because at the time PyTables was slower than I
>> >> wanted.  I've been working on improving IO and found that we can get
>> >> nearly the same speed from h5py; when combined with overall
>> >> performance improvements in yt-3, this still translates to faster
>> >> speeds for very large datasets.  On small to medium datasets, the
>> >> performance is not noticeably different.  I'd like to propose that we
>> >> stop using hdf5_light_reader, which may result in a slight degradation
>> >> of performance for (only) Enzo IO.
>> >>
>> >> Additionally, I don't think *anyone* uses the freetype writer, and I'd
>> >> like to remove it.  I'm actually not sure if anyone has ever used it.
>> >>
>> >> I think with these two changes we can likely improve our packaging
>> >> situation by a good amount, as hdf5 specifically is the main problem
>> >> with distribution in Conda.  These changes would only be in yt 3.0,
>> >> and would not affect 2.x.
>> >>
>> >> [+-][01]?
>> >>
>> >> -Matt
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> >
>> >
>> > --
>> > Cameron Hummels
>> > Postdoctoral Researcher
>> > Steward Observatory
>> > University of Arizona
>> > http://chummels.org
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>