Bartz-Beielstein, Thomas; Doerr, Carola; Bossek, Jakob; Chandrasekaran, Sowmya; Eftimov, Tome; Fischbach, Andreas; Kerschke, Pascal; Lopez-Ibanez, Manuel; Malan, Katherine M.; Moore, Jason H.; Naujoks, Boris; Orzechowski, Patryk; Volz, Vanessa; Wagner, Markus; Weise, Thomas:
Benchmarking in Optimization : Best Practice and Open Issues
In: CIplus, Vol. 2/2020
2020Report in SeriesOpen Access
Faculty of Computer Science and Engineering Science » Institut für Data Science, Engineering, and Analytics
Related: 1 publication(s)
Title:
Benchmarking in Optimization : Best Practice and Open Issues
Author:
Bartz-Beielstein, ThomasTH Köln
DHSB-ID
THK0001582
GND
124999476
ORCID
0000-0002-5938-5158ORCID iD
SCOPUS
57190702501
Other
person connected with TH Köln
;Doerr, Carola;Bossek, Jakob;Chandrasekaran, SowmyaTH Köln
DHSB-ID
THK0002508
SCOPUS
57219767441
Other
person connected with TH Köln
;Eftimov, Tome;Fischbach, AndreasTH Köln
DHSB-ID
THK0002214
SCOPUS
56315875200
Other
person connected with TH Köln
;Kerschke, Pascal;Lopez-Ibanez, Manuel;Malan, Katherine M.;Moore, Jason H.;Naujoks, BorisTH Köln
DHSB-ID
THK0001910
SCOPUS
8955334400
Other
person connected with TH Köln
;Orzechowski, Patryk;Volz, Vanessa;Wagner, Markus;Weise, Thomas
Place of publication:
Köln
Publisher:
Technische Hochschule Köln
Year of publication:
2020
„Publication Channel“:
Open Access
Extent:
58 Seiten
URN
urn:nbn:de:hbz:832-cos4-9022
Language of text:
English
Appears in
CIplus
in:
Vol. 2/2020
ISSN
2194-2870 Show availability online & print
ZDB ID
2804635-3
Type of resource:
Text
Type of resource:
Text
Access Rights:
open access
Peer Reviewed:
Peer Reviewed
Practice Partner:
No
Category:
Research
Part of statistic:
Part of statistic

Abstract in English:

This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well- specified problems, suitable algorithms, adequate performance measures, thoughtful analysis, effective and efficient designs, comprehensible presentations, and guaranteed reproducibility. The final goal is to provide well-accepted guidelines (rules) that might be useful for authors and reviewers. As benchmarking in optimization is an active and evolving field of research this manuscript is meant to co-evolve over time by means of periodic updates.