[yt-users] python timing

[Matthew Turk]

Hi Eric,

On Thu, May 3, 2012 at 1:41 PM, Eric Hallman <hallman at txcorp.com> wrote:
> Is there a standard or best methodology for tracking python code
> performance?  I see in the intertubes that one can use the time module, or
> timeit or other things.  How about memory usage?  Or is this typically
> handled by debugging tools?
>
> What I really want to do is isolate logjams in python code by function call
> etc. in a fairly non-intrusive way.
>
> Thanks for any advice.

If you're using yt, there are three methods you can call upon.  If
not, I'd recommend using the cProfile module as a start, moving on to
what was kernprof and is now lineprof, and ending at Bryan
O'Sullivan's recent statprof.  For visualizing, "pip install jinja2
pyprof2html" will get you pyprof2html, which turns a cProfile file to
HTML which is nicely viewable.

In yt, performance timers (written by Stephen) can help, but for very
simple stuff:

with parallel_profile("some_prefix"):
    some_expensive_operation()

will output .cprof files with processor IDs in them, which can be run
through pyprof2html.  For memory, get_memory_usage() is a yt function
that gives you the current resident size of your PID.

There are a few more memory debuggers, but thye get a bit tricky and I
can't comment on their success-rates.

-Matt

>
> Eric
> --
> Eric Hallman
> Tech-X Corporation               hallman at txcorp.com
> 5621 Arapahoe Ave, Suite A       Phone: (720) 254-5833
> Boulder, CO 80303                Fax:   (303) 448-7756
> --
>
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>