A Benchmark Study on Steepest Descent and Conjugate Gradient Methods-Line Search Conditions Combinations in Unconstrained Optimization

Creative Commons License


CROATIAN OPERATIONAL RESEARCH REVIEW, vol.13, no.1, pp.77-97, 2022 (ESCI) identifier identifier

  • Publication Type: Article / Article
  • Volume: 13 Issue: 1
  • Publication Date: 2022
  • Doi Number: 10.17535/crorr.2022.0006
  • Journal Indexes: Emerging Sources Citation Index (ESCI), Scopus, ABI/INFORM, Aerospace Database, Communication Abstracts, EconLit, INSPEC, Metadex, zbMATH, Directory of Open Access Journals, Civil Engineering Abstracts
  • Page Numbers: pp.77-97
  • Keywords: conjugate gradient, line search, step length, steepest descent, optimization, ALGORITHMS, MODEL
  • Süleyman Demirel University Affiliated: Yes


In this paper, it is aimed to computationally conduct a performance benchmarking for the steepest descent and the three well-known conjugate gradient methods (i.e., Fletcher-Reeves, Polak-Ribiere and Hestenes-Stiefel) along with six different step length calculation techniques/conditions, namely Backtracking, Armijo-Backtracking, Goldstein, weak Wolfe, strong Wolfe, Exact local minimizer in the unconstrained optimization. To this end, a series of computational experiments on a test function set is completed using the combinations of those optimization methods and line search conditions. During these experiments, the number of function evaluations for every iteration are monitored and recorded for all the optimization method-line search condition combinations. The total number of function evaluations are then set a performance measure when the combination in question converges to the functions minimums within the given convergence tolerance. Through those data, the performance and data profiles are created for all the optimization method-line search condition combinations with the purpose of a reliable and an efficient benchmarking. It has been determined that, for this test function set, the steepest descent-Goldstein combination is the fastest one whereas the steepest descent-exact local minimizer is the most robust one with a high convergence accuracy. By making a trade-off between convergence speed and robustness, it has been identified that the steepest descent-weak Wolfe combination is the optimal choice for this test function set.