Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research

@article{King2004EnhancingTV,
  title={Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research},
  author={Gary King and Christopher J. L. Murray and Joshua A. Salomon and Ajay Tandon},
  journal={American Political Science Review},
  year={2004},
  volume={98},
  pages={191 - 207}
}
We address two long-standing survey research problems: measuring complicated concepts, such as political freedom and efficacy, that researchers define best with reference to examples; and what to do when respondents interpret identical questions in different ways. Scholars have long addressed these problems with approaches to reduce incomparability, such as writing more concrete questions—with uneven success. Our alternative is to measure directly response category incomparability and to… 
Improving Anchoring Vignettes Designing Surveys to Correct Interpersonal Incomparability
We report the results of several randomized survey experiments designed to evaluate two intended improvements to anchoring vignettes, an increasingly common technique used to improve interpersonal
Comparability of Survey Measurements
TLDR
This chapter discusses some methods to prevent, detect, and correct for incomparability, and the methods associated with them, for different aspects of total survey error (TSE).
Introduction to the Virtual Issue: Addressing Measurement Comparability in Survey Research
Measurement Comparability in Survey Research What should be done when respondents interpret identical questions in vastly different ways? This is a long-standing problem in survey research. Indeed,
A meeting report: cross-cultural comparability of questionnaire measures in large-scale international surveys
The value of cross-country comparisons is at the heart of large-scale international surveys. Yet the validity of such comparisons is often challenged, particularly in the case of latent traits whose
Comparative Research is Harder Than We Thought: Regional Differences in Experts’ Understanding of Electoral Integrity Questions
Expert evaluations about countries form the backbone of comparative political research. It is reasonable to assume that such respondents, no matter the region they specialize in, will have a
Comparing Incomparable Survey Responses: Evaluating and Selecting Anchoring Vignettes
TLDR
With surveys on a diverse range of topics in a range of countries, it is illustrated how the proposed methods can improve the ability of anchoring vignettes to extract information from survey data, as well as saving in survey administration costs.
The heterogeneous thresholds ordered response model: identification and inference
TLDR
A joint test of the validity of the assumptions of response consistency and vignette equivalence based on the fact that the underlying statistical model is overidentified if the two assumptions hold is proposed.
We Need to Compare, But How? Measurement Equivalence in Comparative Public Administration
There is an increasing interest to study public administrations, public managers or citizens interactions with, and views towards government from a comparative perspective in order to put theories to
The Effects of Vignette Scoring on Reliability and Validity of Self-Reports
TLDR
Results suggest that the underlying assumptions and effects of the anchoring vignette approach should be carefully examined as the increased correlations and reliability estimates can be observed even for response variables that are independent random draws and uncorrelated with any other variable.
...
...

References

SHOWING 1-10 OF 75 REFERENCES
Approaches to equivalence in cross-cultural and cross-national survey research
Three notions of item equivalency are distinguished. They correspond to the back-translation approach, the psychometric IRT approach, and the facet-theoretical approach. The latter defines equivalent
An application of Rasch analysis to questionnaire design : using vignettes to study the meaning of work in the Current Population Survey
As part of the research to develop a new questionnaire for the Current Population Survey (CPS), a special respondent debriefing study was designed to learn more about how respondents understand and
Factor and ideal point analysis for interpersonally incomparable data
Interpersonally incomparable responses pose a significant problem for survey researchers. If the manifest responses of individuals differ from their underlying true responses by monotonic
Measuring social judgments : the factorial survey approach
'Assign this book in a graduate social psychology methods course. It deserves to be read by social psychologists, and the factorial survey technique should be fully integrated into the tool kit of
Individual differences scaling and group attitude structures: British party imagery in 1974
Our purpose in this paper is to investigate the applicability of multidimknsional scaling techniques to problems of summarisation and representation of data generated in large surveys. What we are
Interactional Troubles in Face-to-Face Survey Interviews
For statistically based social science, survey research is the principal means of obtaining data about the social world. The interview from this point of view is a standardized data-collection
Innovations in Experimental Design in Attitude Surveys
In the last decade a revolution has occurred in the design of public opinion surveys. The principal breakthrough has been to combine the distinctive external validity advantages of the representative
Comparing Interest Group Scores across Time and Chambers: Adjusted ADA Scores for the U.S. Congress
Interest group ratings are widely used in studies of legislative behavior. Since the set of votes used is not constant over time and across chambers, the scales underlying the scores can shift and
Analyzing Incomplete Political Science Data: An Alternative Algorithm for Multiple Imputation
TLDR
This work adapts an algorithm and uses it to implement a general-purpose, multiple imputation model for missing data that is considerably faster and easier to use than the leading method recommended in the statistics literature.
Health-Related Quality-of-Life Assessments in Diverse Population Groups in the United States
TLDR
The nature of diversity, importance of ensuring the adequacy of HRZOL measures in diverse groups, methods for assessing comparability of HRQOL measures across groups, and methodological and analytical challenges are discussed.
...
...