[yt-users] Projection Performance

[Britton Smith]

To add to what Sam said, I think you can force yt to save the projections
of the individual reginos by adding the node_name=<string> keyword arg to
the projection call.  As long as you give unique names, there shouldn't be
a problem and the same script should be able to reload them from the .yt
file.

Britton

On Tue, May 1, 2012 at 9:05 PM, Sam Skillman <samskillman at gmail.com> wrote:

> Hi Rick,
>
> For me, running on a single 2.93 GHz Xeon core, it
> takes 9.818254e+02 seconds to project the LCL7 for unweighted density. With
> the change I mention below, on 8 cores, this took the time down
> to 1.796410e+02 seconds.  If you have a parallel file system this will
> probably be a better speedup.
>
> For the lightcone, you may benefit quite a lot by swapping out the quad
> tree projection for the (for now) much more parallel overlap projection
> method and running in parallel.  This works best for naturally domain
> decomped simulations like the light cone.  Note this is only faster in
> parallel.  The quad tree method is much faster in serial. You would do this
> with:
>
> pf.h.proj = pf.h.overlap_proj
>
> after you've loaded the parameter file, and run your script (as is) with
> mpirun -np N your_script.py --parallel
>
> You can probably get decent speedup values for N = 32 or 64. There are
> some scaling performance plots from yt about a year ago in the method paper.
>
> I also might suggest forgetting about saving all of these through the plot
> collection and instead just make all the projections first.  They will be
> saved in the .yt files and can later be recalled in serial where you can
> play around with the various figures.  It does this automatically so when
> you try to load up a plot collection with a density projection, for
> example, it will just read it in from the .yt file.  That way you can just
> get all the adaptive projections and pan/zoom/export however you want
> later.  Note that this method of saving the projections will only work for
> the full box projections.  The region based projections are not currently
> saved automatically.
>
> Anyways, those are my two cents.
> Sam
>
>
>
>
>
> On Tue, May 1, 2012 at 6:15 PM, Richard P Wagner <rpwagner at sdsc.edu>wrote:
>
>> Hi,
>>
>> I wanted to build a sequence of projections using various color maps
>> along each axis. The data set I'm using the z = 0 one from the L7
>> simulation some of you are familiar with. Here's the paste of my current
>> script:
>>  http://paste.yt-project.org/show/2335/
>>
>> (The early sys.exit is deliberate.)
>>
>> Does anyone have an estimate of the time doing this projection serially
>> should take? After two hours it was still going without having produced the
>> first image. The same plots done on the z = 2.75 data took about 15 minutes
>> (this data has about 1/5 the number of grids, though).
>>
>> I will also gladly take advice on better methods for the projections, or
>> if the benefit to doing this in parallel is worth it.
>>
>> Thanks,
>> Rick
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20120501/e769388c/attachment.html>