[yt-svn] commit/yt: MatthewTurk: Removing preloading if the field is a field that requires ghost zones. This

[Bitbucket]

1 new commit in yt:


https://bitbucket.org/yt_analysis/yt/changeset/9e3a3e0cfde3/
changeset:   9e3a3e0cfde3
branch:      yt
user:        MatthewTurk
date:        2011-11-11 22:16:31
summary:     Removing preloading if the field is a field that requires ghost zones.  This
will improve substantially parallelism for ghost-zone fields, as it stops the
preload operation from loading for all of the dependency grids.
affected #:  2 files

diff -r e45e9fb182d9e51b4ab2f96600c738d1e59810d5 -r 9e3a3e0cfde31356a4053378214dda23a4ae2236 yt/data_objects/data_containers.py
--- a/yt/data_objects/data_containers.py
+++ b/yt/data_objects/data_containers.py
@@ -1622,11 +1622,12 @@
         # It is probably faster, as it consolidates IO, but if we did it in
         # _project_level, then it would be more memory conservative
         if self.preload_style == 'all':
+            dependencies = self.get_dependencies(fields, ghost_zones = False)
             print "Preloading %s grids and getting %s" % (
                     len(self.source._get_grid_objs()),
-                    self.get_dependencies(fields))
+                    dependencies)
             self.comm.preload([g for g in self._get_grid_objs()],
-                          self.get_dependencies(fields), self.hierarchy.io)
+                          dependencies, self.hierarchy.io)
         # By changing the remove-from-tree method to accumulate, we can avoid
         # having to do this by level, and instead do it by CPU file
         for level in range(0, self._max_level+1):


diff -r e45e9fb182d9e51b4ab2f96600c738d1e59810d5 -r 9e3a3e0cfde31356a4053378214dda23a4ae2236 yt/utilities/parallel_tools/parallel_analysis_interface.py
--- a/yt/utilities/parallel_tools/parallel_analysis_interface.py
+++ b/yt/utilities/parallel_tools/parallel_analysis_interface.py
@@ -790,6 +790,8 @@
         deps = []
         fi = self.pf.field_info
         for field in fields:
+            if any(getattr(v,"ghost_zones", 0) > 0 for v in
+                   fi[field].validators): continue
             deps += ensure_list(fi[field].get_dependencies(pf=self.pf).requested)
         return list(set(deps))

Repository URL: https://bitbucket.org/yt_analysis/yt/

--

This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.