[Yt-dev] quick question on particle IO

[Stephen Skory]

Hi Matt,

I've gone through what you started and I fixed the bugs. This diff
below produces identical results to the old way. Could you run this on
your dataset you used and see if I haven't ballooned the memory? I was
using a 40^3 dataset (it's fast!) but it's hard to see big changes in
memory using that. If we like this, I'll remove all the prints, and
apply it to my yt fork, and ask Geoffrey to try it out. If he sees no
big problems, I'll do a pull request.

http://paste.enzotools.org/show/1876/


>  * Initializing ParallelHOPHaloFinder with copies (by dividing) of
> position and mass fields

I replaced those with na.multiply and na.divide, which should be doing
it in-place on the arrays.

>  * Copying position, mass into the fKD object without removing them
> from the halo finder (is there any reason they can't simply be moved
> in and removed from the halo finder, then copied back out?)

No, it should be possible, and I did that in the new diff. I don't
know if it's helped.

>  * Rearrange = True is the default, which I believe copies the entire
> kD-tree inside fKD?

That's correct, it rearranges the position data so that the nearest
neighbor searches are faster, and makes a copy to do this.

> It also looks like some of this is because the fKD tree requires
> position to be (N,3) for memory access speed.

Yes, why I want a C/C++ kdtree someday that is either as fast at this
fortran one, or nearly as fast. I have yet to find it.

> Unfortunately, it's not entirely clear to me how we then copy back out
> of the Forthon kD-tree at the right time; I don't quite now the inner
> workings of pHOP.  But I think this is valid, as I don't believe at
> any point information is lost.

I think I've found the right place, just before the fKD object is
deleted and the tree cleared.

-- 
Stephen Skory
s at skory.us
http://stephenskory.com/
510.621.3687 (google voice)