A note on research methodology and benchmarking optimization algorithms

Search Swinburne Research Bank

Home

Please use this identifier to cite or link to this item: http://hdl.handle.net/1959.3/25528

Download PDF (6 pages) (Adobe Acrobat PDF, -1 bytes)

Title: A note on research methodology and benchmarking optimization algorithms
Author(s): Brownlee, Jason
Abstract: A pervasive problem in the field of optimization algorithms is the lack of meaningful and consistent algorithm benchmarking methodology. This includes but is not limited to issues of the selection of problem instances, the selection of algorithm specifications the algorithm configuration parameters and interpretation of results. The intention of this paper is to summarize the literature related to benchmarking optimization algorithms with a focus on benchmarking in the face of the 'no free lunch' theorem and useful statistical tools for interpreting results. This context for this review is biologically inspired optimization algorithms applied to continuous function optimization although the principles extend beyond these themes.
Publication type: Technical report
Research centre: Swinburne University of Technology. Faculty of Information and Communication Technologies. Centre for Information Technology Research
Source: Complex Intelligent Systems : technical reports
Publication year: 2007
Keyword(s): Algorithm benchmarking statistics; Experimental design problem classes; No free lunch theorem; Optimization; Parameter tuning; Sensitivity analysis
Publisher: Swinburne University of Technology
Publisher URL: http://www.ict.swin.edu.au/personal/jbrownlee/#2007
Full text