[Yt-dev] precision in io

[Geoffrey So]

I am all for binary, especially since storage is a big problem for some of
the bigger simulations.

I am just curious, though, which binary are the other simulation groups
pushing for?  Is there a consistent way of storing binary data in YT?  I am
asking because currently sometimes I pickle my own data or store them in
hdf5 .h5 binary or in the .yt files, and I didn't realize the different
binaries are not compatible until recently.

Are we looking for a universal YT way of storing binaries?  One big
advantage would be everyone being able to read the binary if we all store
them the same way, and I'm sure there's many other advantages.  But the
headache is everyone agreeing to store them whichever way.
From
G.S.
On Mon, Aug 15, 2011 at 6:02 AM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Stephen and Geoffrey,
>
> I would prefer we stick with the longer IO output.  The reason is not
> as much that we believe that a halo truly does exist with that
> specified precision, but to do our very best to ensure that we
> communicate between sessions the precise location.  This may also come
> into play with very high precision runs.
>
> My personal preference would be to utilize an all-binary storage
> format as our *primary* storage format and then allow ASCII for
> secondary, caveat emptor purposes.  I believe that both the IRATE
> group and the Galacticus group are pushing forward with halo
> cataloging methods that will be binary.
>
> -Matt
>
> On Sun, Aug 14, 2011 at 9:24 AM, Stephen Skory <s at skory.us> wrote:
> > Hi all,
> >
> >> With the current setting, the halo attributes
> >> are outputted with 9 decimal points, but the ellipsoid parameters
> determined
> >> using the particle's position (when the data is 64 bit) has 16 decimals.
> >
> > just to clarify, what I've done is to add the option to the
> > halos.write_out() function (that outputs the HopAnalysis.out file) to
> > add 5 or so extra columns for the ellipsoid information. So what
> > Geoffrey is thinking about is increasing the precision of all the
> > floats in that text file.
> >
> > --
> > Stephen Skory
> > s at skory.us
> > http://stephenskory.com/
> > 510.621.3687 (google voice)
> > _______________________________________________
> > Yt-dev mailing list
> > Yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20110815/72fb9454/attachment.htm>