[Yt-dev] moving yt from pytables to h5py

[Britton Smith]

Hi everyone,

Currently, the hdf5 operations in yt are handled by pytables.  However,
pytables is not the only python hdf5 module out there.  For my own non-yt
code, I have been using h5py (http://code.google.com/p/h5py/).  For simple
hdf5 i/o, I find h5py to be far more intuitive and flexible than pytables.

I want to propose to people here that we switch the yt trunk over to h5py
before the yt 1.5 release.  Before I enter my sales pitch, I want to make it
known that the work is already done.  I converted a version of the yt-trunk
to h5py this weekend.  Matt and I have been testing it on various machines
and it seems to be working.  None of the yt function calls have changed.
Only the internal calls to tables functionality have been replaced with h5py
calls.

Other than the ease of casual use, some things in h5py's favor include:
 - h5py is faster and relies on fewer python objects than tables
 - tables is not designed for our style of IO
 - better with reloading nodes than pytables (something very relevant to yt,
esp. in parallel)
 - h5py installs easily: I did it from the source, which requires Cython,
but it should be installable with easy_install with these environment
variables:
HDF5_DIR=path to hdf5
HDF5_API=16

I know that switching dependencies is something we want to do as little as
possible, but I think it's worth it in this case.  Matt will alter the
install script so this change should be relatively seemless.  If anyone has
any major objections, please raise them.

Matt, please chime in if I missed anything or presented anything
incorrectly.

Britton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20090505/5ad2b2e2/attachment.htm>