Search

Begin New Search Political Geography United States of America Author Theodore Svoronos

1. Testing the Validity of the Single Interrupted Time Series Design

Author:
Katherine Baicker and Theodore Svoronos
Publication Date:
07-2019
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
Given the complex relationships between patients’ demographics, underlying health needs, and outcomes, establishing the causal effects of health policy and delivery interventions on health outcomes is often empirically challenging. The single interrupted time series (SITS) design has become a popular evaluation method in contexts where a randomized controlled trial is not feasible. In this paper, we formalize the structure and assumptions underlying the single ITS design and show that it is significantly more vulnerable to confounding than is often acknowledged and, as a result, can produce misleading results. We illustrate this empirically using the Oregon Health Insurance Experiment, showing that an evaluation using a single interrupted time series design instead of the randomized controlled trial would have produced large and statistically significant results of the wrong sign. We discuss the pitfalls of the SITS design, and suggest circumstances in which it is and is not likely to be reliable.
Topic:
Health, Governance, and Health Care Policy
Political Geography:
North America and United States of America

2. Two-Stage Examinations: Can Examinations Be More Formative Experiences?

Author:
Dan Levy, Mae Klinger, and Theodore Svoronos
Publication Date:
09-2018
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
Two-stage examinations consist of a first stage in which students work individually as they typically do in examinations (stage 1), followed by a second stage in which they work in groups to complete another examination (stage 2), which typically consists of a subset of the questions from the first examination. Data from two-stage midterm and final examinations are used to assess the extent to which individuals improve their performance when collaborating with other students. On average, the group (stage 2) score was about one standard deviation above the individual (stage 1) score. While this difference cannot be interpreted as the causal effect of two-stage examinations on learning, it suggests that individuals experienced substantial performance gains when working in groups in an examination. This average performance gain was comparable with the average difference between the top performer of the group in stage 1 and the group’s stage 1 average, and was equivalent to about two-thirds of the difference between the “super student” score (i.e. the sum of the maximum score for each question in stage 1) and the group’s stage 1 average. This last result suggests that group collaboration takes substantial (albeit partial) advantage of the aggregate knowledge and skills of the group’s individual members. Student feedback about their experience with two-stage examinations reveal that that these types of examinations are generally perceived to be more helpful for learning and are less stressful than traditional examinations. Finally, using data on group gender compositions, we investigate the potential role of gender dynamics on group efficiency.
Topic:
Education, Gender Issues, Collaborative Learning, and Collaborative Efficiency
Political Geography:
Global Focus and United States of America