How to Measure Python Speed with QCachegrind

Python on IBM iWant to know which parts of your Python program take how much time? Which parts are slow, and how often they are called? Python includes tools to measure speed, but it’s not easy to visualize their meaning.

To make it easier, the QCachegrind tool graphically displays profiler output from a variety of languages. We first introduced our build of QCachegrind as a PHP tool, but you can put your Python programs under the QCachegrind microscope as well.

Generating Profiles with cProfile

The cProfile module is built into Python. To profile pieces of your code, you can include cProfile and invoke it with a fragment of code to run:

import cProfile
# [...]
cProfile.run("my_function()")

This will get you output like this:

         8294 function calls (8087 primitive calls) in 0.596 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       31    0.000    0.000    0.000    0.000 :103(release)
       31    0.000    0.000    0.000    0.000 :143(__init__)
       31    0.000    0.000    0.000    0.000 :147(__enter__)
       31    0.000    0.000    0.000    0.000 :151(__exit__)
       31    0.000    0.000    0.000    0.000 :157(_get_module_lock)
       31    0.000    0.000    0.000    0.000 :176(cb)
     45/4    0.000    0.000    0.058    0.015 :211(_call_with_frames_removed)

[...]

This is already useful, but we want to save it to a file instead, for a better visualization. To write the data to a file, you can provide a filename as the second argument:

cProfile.run("my_function(2)", "with_two")

Viewing Profiles in QCachegrind

First, you’ll need to convert your profiling data from Python’s native profile format to the one QCachegrind uses. The pyprof2calltree utility can do this for us. You’ll need to run this conversion on the same system you captured the data from, because of how Python stores the data.

Install it from pip:

pip3 install pyprof2calltree

Then use it on the profiler output that cProfile generated:

pyprof2calltree -i cprofile -o cachegrind.output

Once you copy over the output file to your computer, you can then open the generated file in QCachegrind.

Viewing a Python profile output

Viewing Python profile output

Try QCachegrind with Your Python Applications

QCachegrind is free, open source, and can be downloaded for multiple platforms in a version we built for the community. It works with output from PHP, Python, Node.js, and more.

If you try QCachegrind with your applications, let us know what you discover.

1 reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.