Direct searches do not use gradient or second-derivative information. They do not use models of the surface. Direct searches only use function evaluation, and the trial solution sequence is directed either by human logical or stochastic rules. Typically they creep up to optima, as opposed to understanding the surface and jumping to, or near to, the perfect answer. One might think, then, that direct searches are inferior. Well, they are inferior to second-order methods but only for the limited class of applications that meet the ideal conditions of deterministic functions with continuum variables and derivatives, no constraints, no flat spots, quadratic-ish surfaces, etc. However, most of the applications that I’ve had to consider are not from that ideal, trivial category. For example, time and space discretization in the models that are used to calculate the OF will generate surface discontinuities, which misdirect second-order optimizers. In general, I find that direct search algorithms beat the best of gradient and second-order optimizers when considering application versatility, speed to find the answer, simplicity of code, robustness, etc.
