Questionnaire Design Problems Limit Usefulness of Results

FPCD-82-9: Published: Mar 11, 1982. Publicly Released: Mar 11, 1982.

Additional Materials:


Office of Public Affairs
(202) 512-4800

GAO reviewed the design of two merit systems review and studies (MSRS) questionnaires, one dealing with the merit system and the other with reprisal.

GAO found that the questionnaires' objectives are very broad and address complex issues. GAO questions the appropriateness of using abstract data to meet descriptive and evaluative objectives. Neither questionnaire adequately provides a baseline or the necessary comparison that would permit an assessment of the relevancy of any problem identified. Questions in both address very abstract concepts, though this is less of an issue in the reprisal questionnaire because several key phrases are defined on the cover. In the merit system questionnaire, respondents are asked for views on statements that are vague and use ambiguous words. One way to minimize the effect of abstract terms and phrases is to define them. While respondents may disagree with the definition, at least they should understand them within the questionnaire context. Both questionnaires contain design flaws which limit the usefulness of the results or make the data very difficult to interpret. Some of the flaws are: (1) biased titles; (2) biased questions that could lead some respondents to give different answers than they would give to a neutral wording; (3) the omission of neutral and escape response alternatives; (4) inadequately defined answer choices where the range of choices is not spread out in equal appearing intervals; (5) tendency of respondent to use response sets; (6) mixed timeframes that are confusing; and (7) lack of adequate space to answer questions. GAO believes that these design problems adversely affect the questionnaires' usefulness in achieving major objectives.

Sep 23, 2015

Aug 31, 2015

Aug 3, 2015

Jul 15, 2015

Jul 7, 2015

Jun 16, 2015

May 20, 2015

Mar 6, 2015

Jan 22, 2015

Looking for more? Browse all our products here