Towards a Theory of Experimentation for Expected Improvement

This paper concerns the role of experimentation in engineering design, especially the process of making improvements through parameter design. A simple mathematical model is proposed for studying experimentation including a model of adaptive one-factor-at-a-time experimentation. Theorems are proven concerning the expected value of the improvement provided by adaptive experimentation. Theorems are also proven regarding the probability that factor effects will be exploited by the process. The results suggest that adaptive one-factor-at-a-time plans tend to exploit two-factor interactions when they are large or otherwise exploit main effects if interactions are small. As a result, the adaptive process provides around 80% of the improvements achievable via parameter design while exploring a small fraction of the design alternatives (less than 20% if the system has more than five variables).