In a recent New Yorker article, Dale Russakoff reported on eight restructured schools created as part of Newark’s controversial school reform plan. Despite the best efforts of teachers to help students succeed, she wrote, “children’s traumas outside school posed bigger problems.” One fifth-grader’s father had been murdered; one girl was distraught about conflict between her parents. Ultimately, only one of these schools showed progress in student scores on the state’s standardized literacy and math tests, while scores for some of the others actually fell.

Newark’s struggle demonstrates the obvious: Upsetting events outside the classroom can undermine academic achievement, and students in disadvantaged communities are more likely to be subject to such events. A new study examines this problem from a slightly different angle, isolating the effect of violent crime in a student’s neighborhood on a subsequent high-stakes test. The researchers, led by NYU sociologist Patrick Sharkey, compared the scores of “exposed” students (whose blocks experienced a violent crime in the seven previous days) to those who lived on blocks where a violent crime occurred in the week after the test. They found that overall the “exposed” group performed modestly worse on the test. When they broke the data down, they saw that the impact of the crimes was significant only for African-American students, and intriguingly, it applied only to the verbal portion, not the math.

The study indicates that, at least for some students, mere awareness of recent violence near their homes can impair performance, above and beyond any ongoing effects of disadvantage. Along with other research on the relationship between community violence and cognitive functioning, this study raises questions about what we can and should expect from schools. Should schools assume a greater role in trying to address emotional and psychological issues, through, say, expanded counseling services or training for teachers? At the same time, is it fair to blame schools for stubbornly low test scores if the results are influenced by external factors? The research suggests that we should give schools more responsibility while holding them less accountable; we should both broaden their mandate and recognize their limits.

Julian Vasquez Heilig, a professor of educational policy and planning at the University of Texas at Austin, believes that schools need to be more comprehensive in serving children. Often, after a violent crime in the community, “the kids come back to school and the teacher’s like, OK, let’s talk about Common Core,” he said. “Teachers need to recognize that students that are coming to them have some very basic needs that need to be met socially and emotionally.”

As an inspiration, he cites the work of Jeff Duncan-Andrade in Oakland. According to Duncan-Andrade’s organization’s website, he began acting as “teacher, life coach, big brother, and offered his home as a safe haven when trauma struck in a student’s life.” In his most recent cohort, 24 of 26 students reportedly went on to college. Duncan-Andrade is planning to open a K-12 school in Oakland that serves as a center in the surrounding neighborhood and offers services in health and housing as well as education.

This concept, sometimes called a “full-service community school,” has gained traction in recent years, especially in California. The Oakland district has announced its intention to adopt this model in all schools. Vallejo, another California city, has introduced the strategy in at least 15 of its schools, offering inclusive academic, social and health services to address the needs of the “whole child.”

These ambitious programs remain unusual, and even they are not guaranteed to raise test scores. Projects like the Harlem Children’s Zone attempt to fill the need for a supplementary approach that also involves families and neighborhoods, but criminal justice policy and economic policy may be ultimately as important to kids’ academic outcomes as education policy.

The research complicates the current vogue for holding teachers and schools accountable for student test scores. The federal No Child Left Behind legislation originally left states little leeway, compelling them to use test score data to evaluate schools. Low scores could result in support such as professional development training, but in the most extreme cases could lead to staff firings and school closures. In 2011, waivers were issued that allow states to use criteria other than test scores for evaluation. But according to a new study by Morgan Polikoff, a professor of education at the University of Southern California, only 18 states have taken advantage of these waivers to incorporate factors such as student surveys and attendance into the evaluations. In Newark, several schools were closed as a result of poor test scores.

This is not to say that tests are worthless. They tell us something, but what they tell us might be more complex than a simple measure of student knowledge and teacher quality. They might tell us that a given school needs more resources to attend to students’ mental health, to fortify students to tackle algebra equations and dangling participles. Philanthropists like Mark Zuckerberg, who contributed $100 million to the Newark plan and just announced a $120 million donation to Bay Area public schools, should take note.