[yt-users] Performance when using derived fields

Nathan Goldbaum nathan12343 at gmail.com
Wed Nov 26 21:08:54 PST 2014


On Wed, Nov 26, 2014 at 8:09 PM, Daniel Fenn <dsfenn at gmail.com> wrote:

> Hi everyone. Is it normal for slices to take substantially longer to be
> created if they're made from a derived field rather than an intrinsic one?
>
> Specifically, I'm having an issue creating an axis-aligned slice using the
> divergence of the velocity field. It's taking around 6.5 minutes just to
> make the slice, whereas if I use temperature or density, it takes around 10
> seconds or so for the same dataset.
>

Velocity divergence is particularly expensive because it needs ghost zones,
which must be filled in using cascading interpolation. Depending on the AMR
structure in your data, this might be quite expensive.

That said, 6.5 minutes for a slice is pretty bad.

Is there any chance you can profile the execution of your script? If you
run your script like so:

$ python -m cProfile myscript.py

you should get a summary printed to stdout of where the interpreter is
spending most of its time. You can also visualize this in more depth if you
dump the profile data to a file, and then use pyprof2html or snakeviz to
load and display the data, e.g:

$ python -m cProfile -o out.cprof myscript.py
$ pyprof2html out.cprof

cProfile comes with pyhon, but pyp4orf2html and snakeviz must be installed
separately using pip.

It would be great if you could file an issue about this.  Even better if
you can reproduce it using one of the test datasets on yt-project.org/data
and attach a script that we can run ourselves.


>
> I also notice that the amount of time it takes is not dependent on the
> number of processors I'm using. I've used 1, 12, and 24 processors, with
> identical results, even though I'm calling enable_parallelism(), and I can
> see that all the processes are running.
>

> I read in the docs that slice operations aren't generally done in
> parallel, but in this case it seems that maybe it would be beneficial. A
> similar operation in VisIt completes much faster, so I'm wondering if I've
> misconfigured something, or if there is something I can do to speed things
> up.
>
> I'd appreciate any thoughts anyone has on the subject.
>
> Thanks,
>
> Dan
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20141126/901cc44d/attachment.htm>


More information about the yt-users mailing list