A new exploratory self-adaptive derivative free training algorithm is developed. It only evaluates error function that is reduced to a set of sub-problems in a constrained search space and the search directions follow rectilinear moves. To accelerate the training algorithm, an interpolation search is developed that determines the best learning rates. The constrained interpolation search decides the best learning rates such that the direction of search is not deceived in locating the minimum trajectory of the error function.

The proposed algorithm is practical when the error function is ill conditioned implying that the Hessian matrix property is unstable, or the...