[Yt-dev] Implementation of a parallel cKDTree

Matt matthewturk at gmail.com
Fri Feb 27 09:20:12 PST 2009


Just FYI, if you need fast kd-trees in C with a Python interface,
scipy.spatial has them; additionally, scipy.spatial is largely
separable from scipy as a whole, so any compilation problems with scipy
will likely not apply to spatial. (
http://docs.scipy.org/doc/scipy/reference/spatial.html ) This could be
extremely useful for a number of problems with generating subsets of
data in yt, clustering algorithms, and ray tracing algorithms. And now
it's multi-threaded.

Sent to you by Matt via Google Reader: Implementation of a parallel
cKDTree via gmane.comp.python.scientific.user by Sturla Molden on
2/27/09 I have fiddled a bit with scipy.spatial.cKDTree for better
performance on multicore CPUs. I have used threading.Thread instead of
OpenMP, so no special compilation or compiler is required. The number
of threads defaults to the number of processors if it can be
determined. The performance is not much different from what I get with
OpenMP. It is faster than using cKDTree with multiprocessing and shared
memory. Memory handling is also improved. There are checks for NULL
pointers returned by malloc or realloc. setjmp/longjmp is used for
error handling if malloc or realloc fail. A memory pool is used to make
sure all complex data structures are cleaned up properly. I have
assumed that crt functions malloc, realloc and free are thread safe.
This is usually the case. If they are not, they must be wrapped with
calls to PyGILState_Ensure and PyGILState_Release. I have not done this
as it could impair scalability. Regards, Sturla Molden
_______________________________________________ SciPy-user mailing list
SciPy-user< at >scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-user
Things you can do from here:
- Subscribe to gmane.comp.python.scientific.user using Google Reader
- Get started using Google Reader to easily keep up with all your
favorite sites
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20090227/f64e7390/attachment.htm>


More information about the yt-dev mailing list