How to Measure Python Speed with QCachegrind
Want to know which parts of your Python program take how much time? Which parts are slow, and how often they are called? Python includes tools to measure speed, but it’s not easy to visualize their meaning.
To make it easier, the QCachegrind tool graphically displays profiler output from a variety of languages. We first introduced our build of QCachegrind as a PHP tool, but you can put your Python programs under the QCachegrind microscope as well.
Generating Profiles with cProfile
The cProfile module is built into Python. To profile pieces of your code, you can include cProfile and invoke it with a fragment of code to run:
1 2 3 |
import cProfile # [...] cProfile.run("my_function()") |
This will get you output like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
8294 function calls (8087 primitive calls) in 0.596 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 31 0.000 0.000 0.000 0.000 :103(release) 31 0.000 0.000 0.000 0.000 :143(__init__) 31 0.000 0.000 0.000 0.000 :147(__enter__) 31 0.000 0.000 0.000 0.000 :151(__exit__) 31 0.000 0.000 0.000 0.000 :157(_get_module_lock) 31 0.000 0.000 0.000 0.000 :176(cb) 45/4 0.000 0.000 0.058 0.015 :211(_call_with_frames_removed) [...] |
This is already useful, but we want to save it to a file instead, for a better visualization. To write the data to a file, you can provide a filename as the second argument:
1 |
cProfile.run("my_function(2)", "with_two") |
Viewing Profiles in QCachegrind
First, you’ll need to convert your profiling data from Python’s native profile format to the one QCachegrind uses. The pyprof2calltree utility can do this for us. You’ll need to run this conversion on the same system you captured the data from, because of how Python stores the data.
Install it from pip:
1 |
pip3 install pyprof2calltree |
Then use it on the profiler output that cProfile generated:
1 |
pyprof2calltree -i cprofile -o cachegrind.output |
Once you copy over the output file to your computer, you can then open the generated file in QCachegrind.
Try QCachegrind with Your Python Applications
QCachegrind is free, open source, and can be downloaded for multiple platforms in a version we built for the community. It works with output from PHP, Python, Node.js, and more.
If you try QCachegrind with your applications, let us know what you discover.
Hello Calvin,
It is a great sharing…I am very much pleased with the contents you have mentioned. I wanted to thank you for this great article.