[yt-users] python timing

[Eric Hallman]

Matt,
  OK.  Thanks a lot for your advice on this.  This looks like a perfect place to start!

Eric
On May 3, 2012, at 1:44 PM, Matthew Turk wrote:

> Hi Eric,
> 
> On Thu, May 3, 2012 at 1:41 PM, Eric Hallman <hallman at txcorp.com> wrote:
>> Is there a standard or best methodology for tracking python code
>> performance?  I see in the intertubes that one can use the time module, or
>> timeit or other things.  How about memory usage?  Or is this typically
>> handled by debugging tools?
>> 
>> What I really want to do is isolate logjams in python code by function call
>> etc. in a fairly non-intrusive way.
>> 
>> Thanks for any advice.
> 
> If you're using yt, there are three methods you can call upon.  If
> not, I'd recommend using the cProfile module as a start, moving on to
> what was kernprof and is now lineprof, and ending at Bryan
> O'Sullivan's recent statprof.  For visualizing, "pip install jinja2
> pyprof2html" will get you pyprof2html, which turns a cProfile file to
> HTML which is nicely viewable.
> 
> In yt, performance timers (written by Stephen) can help, but for very
> simple stuff:
> 
> with parallel_profile("some_prefix"):
>    some_expensive_operation()
> 
> will output .cprof files with processor IDs in them, which can be run
> through pyprof2html.  For memory, get_memory_usage() is a yt function
> that gives you the current resident size of your PID.
> 
> There are a few more memory debuggers, but thye get a bit tricky and I
> can't comment on their success-rates.
> 
> -Matt
> 
>> 
>> Eric
>> --
>> Eric Hallman
>> Tech-X Corporation               hallman at txcorp.com
>> 5621 Arapahoe Ave, Suite A       Phone: (720) 254-5833
>> Boulder, CO 80303                Fax:   (303) 448-7756
>> --
>> 
>> 
>> 
>> 
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- 
Eric Hallman
Tech-X Corporation               hallman at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 254-5833
Boulder, CO 80303                Fax:   (303) 448-7756
--



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20120503/2706c608/attachment.html>