# Design of Experiments for the Tuning of Optimisation Algorithms

@inproceedings{Ridge2007DesignOE, title={Design of Experiments for the Tuning of Optimisation Algorithms}, author={Enda Ridge}, year={2007} }

This thesis presents a set of rigorous methodologies for tuning the performance of algorithms that solve optimisation problems. Many optimisation problems are difficult and time-consuming to solve exactly. An alternative is to use an approximate algorithm that solves the problem to an acceptable level of quality and provides such a solution in a reasonable time. Using optimisation algorithms typically requires choosing the settings of tuning parameters that adjust algorithm performance subject… Expand

#### Figures, Tables, and Topics from this paper

table 10.1 figure 10.1 figure 10.2 figure 10.3 figure 10.4 figure 10.5 figure 10.6 table 11.1 figure 11.1 figure 11.10 figure 11.11 figure 11.12 figure 11.13 figure 11.14 figure 11.15 figure 11.16 figure 11.17 figure 11.18 figure 11.19 figure 11.2 table 11.2 figure 11.20 figure 11.3 figure 11.4 figure 11.5 figure 11.6 figure 11.7 figure 11.8 figure 11.9 figure 2.1 table 2.1 figure 2.2 figure 2.3 figure 2.4 figure 2.5 figure 2.6 figure 2.7 figure 2.8 table 3.1 table 4.1 figure 5.1 figure 5.2 figure 5.3 figure 6.1 table 6.1 figure 6.2 figure 6.3 table 7.1 figure 7.1 figure 7.2 figure 7.3 figure 7.3 figure 7.4 figure 7.5 table 8.1 figure 8.1 figure 8.2 figure 8.3 figure 8.4 figure 8.5 figure 8.6 table 9.1 figure 9.1 figure 9.10 figure 9.11 figure 9.12 figure 9.13 figure 9.2 figure 9.3 figure 9.4 figure 9.5 figure 9.6 figure 9.7 figure 9.8 figure 9.9 figure A.1 table A.1 figure A.2 table A.2 figure A.3 figure A.4 figure A.5 figure A.6 figure B.1 figure B.2 figure B.3 figure B.4 figure D.1 figure D.2 figure D.3 figure D.4 figure D.5 figure D.6 figure D.7 figure D.8 figure D.9

#### 27 Citations

Tuning genetic algorithm parameters using design of experiments

- Computer Science
- GECCO Companion
- 2020

The effect of tuning the operators and parameters of a genetic algorithm for solving the Traveling Salesman Problem is explored using Design of Experiments theory and it is shown that the parameters selected result in improved performance both in the quality of results obtained and the convergence rate when compared with untuned parameter settings. Expand

Sample size calculations for the experimental comparison of multiple algorithms on multiple problem instances

- Computer Science, Mathematics
- J. Heuristics
- 2020

This work presents a statistically principled method for estimating the required number of instances in the experimental comparison of multiple algorithms on a given problem class of interest, based on optimal sample size ratios that minimise the totalrequired number of runs subject to a desired accuracy in the estimation of paired differences. Expand

Parameter tuning versus adaptation : proof of principle study on differential evolution

- 2008

The efficacy of an optimization method often depends on the choosing of a number of control parameters. Practitioners have traditionally chosen these control parameters manually, often according to… Expand

Determining Whether a Problem Characteristic Affects Heuristic Performance

- Mathematics, Computer Science
- Recent Advances in Evolutionary Computation for Combinatorial Optimization
- 2008

This chapter presents a rigorous Design of Experiments (DOE) approach for determining whether a problem characteristic affects the performance of a heuristic, and results demonstrate that for a given instance size, an increase in the standard deviation of the cost matrix of instances results in an increases in the difficulty of the instances. Expand

A semi-automated design of instance-based fuzzy parameter tuning for metaheuristics based on decision tree induction

- Computer Science
- J. Oper. Res. Soc.
- 2015

This paper presents an approach that semi-automatically designs the fuzzy logic rule base to obtain instance-specific parameter values by means of decision trees that enables the user to automate the process of converting insights about instance- specific information and its impact on heuristic performance into a fuzzy rule base IPTS system. Expand

Archive of SID A Simulated Annealing Algorithm for Unsplittable Capacitated Network Design

- Mathematics, Computer Science
- 2012

A simulated annealing approach with innovative representation and neighborhood structure for unsplittable MCND problem and the results show that the proposed SA can find near optimal solution in much less time than exact algorithm. Expand

A hybrid simulated annealing and column generation approach for capacitated multicommodity network design

- Computer Science
- J. Oper. Res. Soc.
- 2013

A hybrid simulated annealing (SA) and column generation (CG) algorithm for the path-based formulation of the capacitated multicommodity network design (PCMND) problem and is compared with the solutions of CPLEX solver and the best-known method in the literature under different time limits. Expand

Best practices for comparing optimization algorithms

- Computer Science, Mathematics
- 2017

This paper systematically review the benchmarking process of optimization algorithms, and provides suggestions for each step of the comparison process and highlights the pitfalls to avoid when evaluating the performance of optimized algorithms. Expand

DIMMA: A Design and Implementation Methodology for Metaheuristic Algorithms - A Perspective from Software Development

- Computer Science
- Int. J. Appl. Metaheuristic Comput.
- 2010

The proposed methodology consists of three main phases and each phase has several steps in which activities that must be carried out are clearly defined in this paper. Expand

A combined greedy-walk heuristic and simulated annealing approach for the closest string problem

- Mathematics, Computer Science
- Optim. Methods Softw.
- 2014

Experimental results show that the proposed simulated annealing approach allows one to locate good solutions for the problem, that are close to the optimum found by the IP formulation and outperforms other heuristic methods for the CSP. Expand

#### References

SHOWING 1-10 OF 126 REFERENCES

Analyzing heuristic performance with response surface models: prediction, optimization and robustness

- Computer Science
- GECCO '07
- 2007

This paper is the first use of desirability functions, a well-established technique in DOE, to simultaneously optimise these conflicting goals. Expand

A systematic procedure for setting parameters in simulated annealing algorithms

- Mathematics, Computer Science
- Comput. Oper. Res.
- 1998

A systematic procedure to find appropriate values for parameters quickly without much human intervention by using a nonlinear optimization method, the simplex method for nonlinear programming is suggested. Expand

Fine-Tuning of Algorithms Using Fractional Experimental Designs and Local Search

- Computer Science, Mathematics
- Oper. Res.
- 2006

The development of CALIBRA is described, a procedure that attempts to find the best values for up to five search parameters associated with a procedure under study and is able to find parameter values that either match or improve the performance of the procedures resulting from using the parameter values suggested by their developers. Expand

Tuning the Performance of the MMAS Heuristic

- Engineering, Computer Science
- SLS
- 2007

An in-depth Design of Experiments (DOE) methodology for the performance analysis of a stochastic heuristic for the Travelling Salesperson Problem (TSP) using the Response Surface Methodology. Expand

Experimental Evaluation of Heuristic Optimization Algorithms: A Tutorial

- Computer Science
- J. Heuristics
- 2001

The methodological issues that must be confronted by researchers undertaking experimental evaluations of heuristics, including experimental design, sources of test instances, measures of algorithmic performance, analysis of results, and presentation in papers and talks are highlighted. Expand

Screening the parameters affecting heuristic performance

- Computer Science
- GECCO '07
- 2007

This research presents a Design of Experiments (DOE) approach that uses a Fractional Factorial Design to screen the tuning parameters of Ant Colony System (ACS) for the Travelling Sales person problem. Expand

Experimental research in evolutionary computation

- Computer Science
- GECCO '07
- 2007

It is demonstrated how SPO improves the performance of many search heuristics significantly, however, this performance gain is not available for free and costs of this tuning process are discussed. Expand

Sequential Experiment Designs for Screening and Tuning Parameters of Stochastic Heuristics

- 2006

This paper describes a sequential experimentation approach for efficiently screening and tuning the parameters of a stochastic heuristic. Stochastic heuristics such as ant colony algorithms often use… Expand

On Optimal Parameters for Ant Colony Optimization Algorithms

- Computer Science
- IC-AI
- 2005

A hybrid ACO algorithm, similar to the one independently developed in [16], which uses a genetic algorithm in the early stages to ‘breed’ a population of ants possessing near optimal behavioural parameter settings for a given problem. Expand

Near Parameter Free Ant Colony Optimisation

- Computer Science
- ANTS Workshop
- 2004

This paper uses ant colony optimisation to evolve suitable parameter values (using its own optimisation processes) while it is solving combinatorial problems and reveals that the use of the augmented solver generally performs well against one that uses a standard set of parameter values. Expand