Main»pprof.py: A performance profiler

pprof.py: A performance profiler

Performance profiles are a great way to visualize the performance of optimization software on a set of test problems. (Probably they can also be used sensibly for comparing other types of algorithms.) This is a short Python script to create performance profiles.

Requirements

To use pprof.py, you will need to make sure you have working versions of

  • Python (version 1.5 or later)
  • Gnuplot (probably your Linux system already has this installed)
  • Gnuplot.py: This is a Python interface to Gnuplot. It does not come with the usual Python distribution, but it is pure Python so there's no compiling involved. If you install from the tar files (available here), just remember to set the environment variable PYTHONPATH accordingly.

Download

Note: you may have to right-click (or whatever works for your browser) and choose "Download" to make sure that these file arrive safely.

Usage

Here is the output from pprof.py --help: 

 Usage: pprof.py [OPTION]... [FILE 1] [FILE 2]...[FILE N]
 Create a performance profile chart from metrics in FILEs.
 Output is an eps file sent to stdout.

 Example 1:
   Profile the metrics in column 3 of the files solver1, solver2, and solver3.
   Use a log2 scale for the x-axis.  Redirect the stdout to profile.eps.

     pprof.py -l 2 -c 3 solver1 solver2 solver3 > profile.eps

 Example 2:
   Specify a title, linestyle and failure threshold.  Pop up an X window.

     pprof.py -c 3 -t "Plot title" --linestyle "linespoints"               --term "x11" solver1 solver2 solver3

 See Dolan and More',
    "Benchmarking optimization software with performance profiles",
    available at http://www-unix.mcs.anl.gov/~more/cops/

 Options
   -c, --column=COLUMN      get metrics from column COLUMN (default 1)
   -h, --help               get some help
   -l, --log=BASE           logBASE scale for x-axis (default linear)
       --legend             insert a legend
       --linestyle=STYLE    use STYLE as Gnuplot line style (default steps)
       --sep RE             use regexp RE to indicate new column (default space)
       --term=TERM          use TERM  Gnuplot terminal (default postscript)
   -t, --title=LABEL        use LABEL as  title  (default none)
   -x, --xlabel=LABEL       use LABEL for x-axis (default none)
   -y, --ylabel=LABEL       use LABEL for y-axis (default none)

 - Use non-positive values to indicate that the algorithm failed.
 - Use --sep 'S' to indicate columns are separated by character S.
 - Use --sep "r'RE'" to separate instead by a regular expression.
 - Any line starting with a % or a # is ignored.

Example

Applying the above example to the metrics found in solver1, solver2, solver3 gives (after converting the eps file to png):

Acknowledgments

Thanks to

  • Liz Dolan for testing and finding bugs.
  • Dominique Orban for adding many needed options.