Employment:

Questionnaire Design Problems Limit Usefulness of Results

FPCD-82-9: Published: Mar 11, 1982. Publicly Released: Mar 11, 1982.

Additional Materials:

Contact:

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

GAO reviewed the design of two merit systems review and studies (MSRS) questionnaires, one dealing with the merit system and the other with reprisal.

GAO found that the questionnaires' objectives are very broad and address complex issues. GAO questions the appropriateness of using abstract data to meet descriptive and evaluative objectives. Neither questionnaire adequately provides a baseline or the necessary comparison that would permit an assessment of the relevancy of any problem identified. Questions in both address very abstract concepts, though this is less of an issue in the reprisal questionnaire because several key phrases are defined on the cover. In the merit system questionnaire, respondents are asked for views on statements that are vague and use ambiguous words. One way to minimize the effect of abstract terms and phrases is to define them. While respondents may disagree with the definition, at least they should understand them within the questionnaire context. Both questionnaires contain design flaws which limit the usefulness of the results or make the data very difficult to interpret. Some of the flaws are: (1) biased titles; (2) biased questions that could lead some respondents to give different answers than they would give to a neutral wording; (3) the omission of neutral and escape response alternatives; (4) inadequately defined answer choices where the range of choices is not spread out in equal appearing intervals; (5) tendency of respondent to use response sets; (6) mixed timeframes that are confusing; and (7) lack of adequate space to answer questions. GAO believes that these design problems adversely affect the questionnaires' usefulness in achieving major objectives.

Jul 23, 2014

May 15, 2014

Apr 9, 2014

Mar 5, 2014

Jan 23, 2014

Dec 20, 2013

Dec 6, 2013

Dec 2, 2013

Nov 4, 2013

Jul 19, 2013

Looking for more? Browse all our products here