References to Papers and Data

Methods and data provided here:

• 2013-14 Common Core of Data for schools and school districts
• American Community Survey and District Spatial Tabulations 2009-13
• Tipton, E. (2014) Stratified sampling using cluster analysis: A sample selection strategy for improved generalizations from evaluations. Evaluation Review, 37(2): 109-139.
• Tipton, E. (2014) How generalizable is your evaluation? Comparing a sample and population through a generalizability index. Journal of Educational and Behavioral Statistics, 39(6): 478 – 501.

Planning future evaluations:

• Tipton, E., Hedges, L.V., Vaden-Kiernan, M., Borman, G.D., Sullivan, K. & Caverly, S. (2014) Sample selection in randomized evaluations: A new method using propensity score stratified sampling. Journal of Research on Educational Effectiveness, 7(1): 114-135.
• Roschelle, J., Feng, M., Gallagher, H.A., Murphy, R., Harris, C., Kamdar, D., & Trinidad, G. (2014) Recruiting participants for large-scale random assignment evaluations in school settings. Menlo Park, CA: SRI International.
• Tipton, E., Fellers, L., Caverly, S., Vaden-Kiernan, M., Borman, G., Sullivan, K., & Ruiz de Castilla, V. Site selection in evaluations: A follow-up evaluation of site recruitment in two scale-up studies. Forthcoming in Journal of Research on Educational Effectiveness.
• Tipton, E. & Peck, L. A Design-based approach to improve external validity in welfare policy evaluations. Working paper.

Post-hoc assessment and adjustments:

• Stuart, E. A., Cole, S. R., Bradshaw, C. P., & Leaf, P. J. (2011) The use of propensity scores to assess the generalizability of results from randomized trials. Journal of the Royal Statistical Society, Series A, Part 2, 369-386.
• Olsen, R.B., Orr, L.L., Bell, S.H., and Stuart, E.A. (2013) External validity in policy evaluations that choose sites purposively. Journal of Policy Analysis and Management, 32 107-121.
• Tipton, E. (2013) Improving generalizations from evaluations using propensity score subclassification: Assumptions, properties, and contexts. Journal of Educational and Behavioral Statistics, 38: 239-266.
• O'Muircheartaigh, C., & Hedges, L. V. (2014). Generalizing from unrepresentative evaluations: a stratified propensity score approach. Journal of the Royal Statistical Society: Series C (Applied Statistics), 63(2), 195-210.
• Stuart, E. A., Bradshaw, C. P., & Leaf, P. J. (2015). Assessing the generalizability of randomized trial results to target populations. Prevention Science, 16(3), 475-485.
• Tipton, E., Hallberg, K., Hedges, L.V., & Chan, W. Implications of small samples for generalization: Adjustments and rules of thumb. Working paper.

Other papers on generalization:

• Hedges, L. V. (2013). Recommendations for practice: Justifying claims of generalizability. Educational Psychology Review, 25(3), 331-337.
• Schochet, P. Z., Puma, M., & Deke, J. (2014) Understanding variation in treatment effects in education impact evaluations: An overview of quantitative methods (NCEE 2014–4017). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Analytic Technical Assistance and Development. Retrieved from http://ies.ed.gov/ncee/edlabs.

If you have other papers to add to this list, please contact us with the citation.