100 Automatic Inconsistency Detection for Online Responses

Online responses in multi-criteria decision making (MCDM) often contain inconsistencies caused by variations in the mood, attention, or mental state of the respondents. Such inconsistent responses in online questionnaires often lead to undesired outcomes. The initial research motivation was to resolve the shortcomings of conventional approaches in their diminishing efficiency when the number of criteria increases. In order to detect possible inconsistencies more efficiently, a qualitative triad comparison technique based on the rules derived from the properties of transitivity was proposed, in contrast to the consistency ratio computation technique used by the conventional quantitative approaches. The initial research attempt was on the experimentation of a more efficient comparison technique by comparing random responses three at a time, instead of computing the consistency ratio upon receiving all the responses. It was then found that the respondents tend to lose concentration when they are required to answer a large number of questions. In order to lighten the mental burden of the respondents, further research was then conducted to reorder the questions based on the optimal triads formation rules. Although the research outcomes appeared to have added a significant value to the conventional approaches, both the techniques were used only upon completion of the questionnaire. The research work was then directed to a possible detection at an earlier stage. It was then found that the detection could start as early as after receiving (n-1) responses, among a total of n(n-1)/2 responses. Subsequent to the (n-1)^th response, detection would then be automatic, for each response. It is then argued that the proposed qualitative triad comparison approach for specially sequenced questions is necessary to automatically detect the logically inconsistent responses after receiving (n-1) responses, especially when more than half a dozen criteria are involved.