|
Researchers faced with choices about the measures and methods to employ in studying parenting and child outcomes require a framework for making decisions about study design. As discussed in Chapter 14 of the monograph, such a framework can encompass both methodological and substantive considerations. Methodological considerations include feasibility and data quality. Based on the New Chance Observational Study, as well as other work, we conclude that
developmentalists can draw on a methodological repertoire within a survey framework that includes maternal self-reports on parenting, interviewer ratings, cognitive assessments of children, interviews with children, and administration of observational protocols. The availability of such a broad range of measurement techniques means that researchers can assemble a "balanced portfolio" of measures that spread the measurement "risk" across methods having different strengths and vulnerabilities.
The overarching substantive consideration in asking whether to include observational work within this portfolio is the "value added" of the observational data. Such work could be undertaken as part of a freestanding study, in which all participants provide both self-reported data, perhaps enhanced by interviewer observations, and observational data. Or observational work could be embedded within a larger survey effort, as was the case in this study. In considering the embedded model, the "analytical leverage" provided by the observational data is a consideration. If the subsample participating in the observational component of the study is selected randomly, it is possible to generalize to the larger sample, and/or to the population from which it was drawn, through statistical weighting procedures. Other potential analytical strategies include using the richness of the observational measures to help in interpreting data from other sources, including assessing how much confidence to place in other measures that are potentially subject to different method effects. Finally, observational measures from a subsample may be used to augment quantitative survey findings.
Researchers, however, should not expect to find a single decision rule for determining when observational research within a survey model is warranted. Every study is different. Survey-based observational work makes sense when it serves well-articulated analytical objectives and is conceptualized as an integral part of an overall research design. If resources are available and the potential added value of observational measures is clear for a particular study, the observational method is a valuable adjunct to more conventional survey approaches.
|