Questionnaire Design Problems Limit Usefulness of Results

FPCD-82-9: Published: Mar 11, 1982. Publicly Released: Mar 11, 1982.

Additional Materials:

Contact:

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

GAO reviewed the design of two merit systems review and studies (MSRS) questionnaires, one dealing with the merit system and the other with reprisal.

GAO found that the questionnaires' objectives are very broad and address complex issues. GAO questions the appropriateness of using abstract data to meet descriptive and evaluative objectives. Neither questionnaire adequately provides a baseline or the necessary comparison that would permit an assessment of the relevancy of any problem identified. Questions in both address very abstract concepts, though this is less of an issue in the reprisal questionnaire because several key phrases are defined on the cover. In the merit system questionnaire, respondents are asked for views on statements that are vague and use ambiguous words. One way to minimize the effect of abstract terms and phrases is to define them. While respondents may disagree with the definition, at least they should understand them within the questionnaire context. Both questionnaires contain design flaws which limit the usefulness of the results or make the data very difficult to interpret. Some of the flaws are: (1) biased titles; (2) biased questions that could lead some respondents to give different answers than they would give to a neutral wording; (3) the omission of neutral and escape response alternatives; (4) inadequately defined answer choices where the range of choices is not spread out in equal appearing intervals; (5) tendency of respondent to use response sets; (6) mixed timeframes that are confusing; and (7) lack of adequate space to answer questions. GAO believes that these design problems adversely affect the questionnaires' usefulness in achieving major objectives.

Oct 19, 2020

Jun 22, 2020

Apr 14, 2020

Feb 14, 2020

Feb 10, 2020

Sep 9, 2019

Apr 10, 2019

  • employment icon, source: GAO

    Priority Open Recommendations:

    Department of Labor
    GAO-19-395SP: Published: Apr 3, 2019. Publicly Released: Apr 10, 2019.

Apr 9, 2019

Apr 8, 2019

Mar 28, 2019

Looking for more? Browse all our products here