[yt-users] faster method for getting frbs?

[Nathan Goldbaum]

I don't think there's a way to speed it up too much, at least on a single core.

It's likely that most of the cost is in building the index (this
implicitly happens when you do "ds.slice"). This requires parsing the
AMR hierarchy, detecting which fields are available on disk and
inferring which derived fields yt can construct out of those, parsing
parameters, etc.

One way to verify that this is where the time is going is to run your
analysis script under a profiler:

$ python -m cProfile -o prof.out

You can then postprocess the prof.out file to get things like hotspot
statistics and call graphs. I like to use pyprof2html for this:

https://pypi.python.org/pypi/pyprof2html/

(despite the docs on pypi, you should probably install it with "pip
install pyprof2html")

If you have access to a cluster, another option would be to take
advantage of the fact that your workflow is embarrassingly parallel
(i.e. the act of producing a single image is completely independent of
the other images in the time series). yt provides a function called
parallel_objects which makes it easy to construct embarrassingly
parallel analysis pipelines that can be managed using an MPI-parallel
cluster. More detail on yt's parallelism and parallel_objects is
available here:

http://yt-project.org/doc/analyzing/parallel_computation.html
http://yt-project.org/doc/reference/api/generated/yt.utilities.parallel_tools.parallel_analysis_interface.parallel_objects.html

I've seen substantial speedups for generating movies in the past using
this machinery, so I think this may really help.

Hope that helps,

Nathan

On Wed, Feb 3, 2016 at 9:04 AM, Slavin, Jonathan
<jslavin at cfa.harvard.edu> wrote:
> Hi all,
>
> I've been using yt to handle FLASH AMR data stored in HDF5 files.  I want to
> cycle through a series of images generated, plotting each as I go using
> matplotlib.  To do this I have (somewhat simplified):
> for f in files:
>     ds = yt.load(f)
>     slc = ds.slice(2,0)  # its 2D r-z data
>     frb = slc.to_frb(width=(Rmax,'pc'), height=(Zmax, 'pc'),
> resolution=(NX,NY))
>     image = np.array(frb['temp'])
>     imshow(img)
>     etc.
>
> This works the way I want it to, but is very slow.  Is there a way to speed
> it up?
>
> Thanks,
> Jon
>
> --
> ________________________________________________________
> Jonathan D. Slavin                 Harvard-Smithsonian CfA
> jslavin at cfa.harvard.edu       60 Garden Street, MS 83
> phone: (617) 496-7981       Cambridge, MA 02138-1516
> cell: (781) 363-0035             USA
> ________________________________________________________
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>