M.I.T. report finds Boston charter school students outperform peers on tests, 4-year college enrollment
May 22, 2013
Research finds positive impact of charters reaches beyond MCAS to SAT and advanced placement tests
Boston – A new report by researchers from the School Effectiveness and Inequality Initiative at M.I.T. has found charter school students in Boston achieve higher scores on the MCAS, SAT and Advance Placement tests. The report, Charter Schools and the Road to College Readiness: The Effects on College Preparation, Attendance and Choice, was released at an Understanding Boston forum at the Boston Foundation this morning. The Boston Foundation and NewSchools Venture Fund provided funding for the data analysis.
“The performance of Massachusetts charter schools – and especially those in Boston – on the MCAS has been widely reported, and this research draws a conclusive link between those scores and charter school attendance in Boston,” noted Paul S. Grogan, President and CEO of the Boston Foundation. “Add to that the research demonstrating charter students’ improved performance on the SAT and Advanced Placement, and the case grows stronger that charters need to be a growing part of school improvement in Boston and other Massachusetts cities.”
“This study breaks new ground by analyzing multiple measures of charter school impact, beyond just MCAS,” said Jim Peyser, a Managing Partner with NewSchools Venture Fund. “Its compelling findings show that Boston charters are doing a much better job of preparing young people for success in college.”
Josh Angrist and Parag Pathak, Directors of the School Effectiveness and Inequality initiative at the Massachusetts Institute of Technology, led a team of researchers who compiled and analyzed a wide range of data on applicants to six Boston charter high schools. Their statistical approach uses randomized charter admissions lotteries to control for differences between students who do and don’t choose to attend charters. In addition, by following everyone who is offered a charter seat, no matter how long they spend attending charter school, they eliminate the bias that might otherwise arise when some students change schools.
“This study of charters finds that some of the short-run gains seen in our earlier studies of Boston charters are remarkably persistent,” said Angrist about the findings. “More research continues to be needed,” added Pathak, “But we can reasonably conclude here that charter attendance raises the probability that students pass the MCAS, increases SAT scores, boosts the likelihood that students take Advanced Placement tests, and shifts them away from two-year colleges toward four-year degree programs, especially in public colleges and universities.”
Among the findings:
MCAS: Charter school attendance increases the likelihood that applicants meet Massachusetts graduation competency standards and qualify for an Adams Scholarship. Charter students were 16 percent more likely to meet the competency standards of the MCAS in their first try, and were 18 percent more likely to qualify for an Adams Scholarship, which uses MCAS scores in its award criteria. Recipients receive free tuition at a Massachusetts public college or university.
SAT: The research team found that charter attendance had little effect on the likelihood that students took the SAT, but charter school students were 11 percentage points more likely to score above the bottom quartile of state SAT scores. Boston charter attendance lifted a student’s total SAT score by a little over 100 points, a large and statistically significant result.
Charters and special education students: The researchers also explored the impact of charter attendance on special education students, finding that while there was little difference in how charters and traditional schools classified special education students, charter attendance increases the rate at which special education students meet state competency standards markedly, and appears to generate increased human capital for special education students, as evidenced by especially large gains in MCAS and SAT scores. On the SAT, the charter attendance impact was 164 points.
The researchers conclude that, “With the exception of Adams Scholarship qualification and a possible delay in high school graduation, special education students seem to get as much or more from charter attendance as does the general applicant population.”
Advanced Placement: Charter school attendance increases the likelihood that a student takes at least one AP test by 28 percentage points. More than half of charter students take at least one AP test, compared with about a quarter of the students in traditional public schools. Charter students were also more likely to score a 3 or higher on the test. Charter attendance increases the probability of earning a score of 3 by a marginally significant 9.5 percentage points, but generates no significant increase in the likelihood of earning a 4 or 5.
Graduation: On measures of graduation, however, the researchers found no statistically significant impact of charter attendance on graduation, finding a slightly lower on-time graduation rate and a slightly higher graduation rate for students within two years of their expected graduation date. In neither case, however, were the results statistically significant.
College enrollment: The researchers looked at where students went after graduation. They found that charter attendance increases college enrollment by about six percentage points in the on time sample, an estimate that grows to 13 points if one gives students an extra two years after graduation to enroll. These gains are noteworthy but not statistically significant, because of the relatively small sample sizes.
A more statistically striking finding came when the researchers examined the types of schools where students enroll. The researchers found that charter attendance reduced the likelihood that a student would attend a two-year college by 10 percent, while increasing the probability of enrolling in a four-year college by 23 percentage points within two years of graduation. Charter attendance showed a particular correlation to attendance at four-year public colleges.
The researchers also explored a theory that charters are driving out less desirable students in order to improve scores. In addition to counting all students who attended a charter in the charter schools group even after they left the charter program, researchers also looked at the “peer effect” of charters to see if changes in the charter population during high school strengthened the overall performance of the classes, and found that while charter students were more likely than their peers to change schools, that increased school switching did not appear to lead to further changes in peer composition that might explain the beneficial effects of charter attendance.
******
The Boston Foundation, Greater Boston’s community foundation, is one of the oldest and largest community foundations in the nation, with net assets of more than $900 million. In 2012, the Foundation and its donors made $88 million in grants to nonprofit organizations and received gifts of close to $60 million. The Foundation is a partner in philanthropy, with some 900 separate charitable funds established by donors either for the general benefit of the community or for special purposes. The Boston Foundation also serves as a major civic leader, provider of information, convener and sponsor of special initiatives that address the region’s most pressing challenges. The Philanthropic Initiative (TPI), an operating unit of the Foundation, offers special consulting services to philanthropists. Through its services and its work to advance the broader field of strategic philanthropy, TPI has influenced billions of dollars of giving worldwide. For more information about the Boston Foundation and TPI, visit www.tbf.org or call 617-338-1700.