Allison Gerber wasn’t always sold on the summer youth jobs programs that many U.S. cities have been running for years.

“I’m somebody who started in workforce development, thinking in the sector strategies space,” Gerber said last Thursday at the Brookings Institution’s Falk Auditorium, referring to the strategy of working closely with employers to pave career pathways, relying heavily on on-the-job training and wraparound services such as childcare provision. The approach is intended to train or retrain workers for job types that, due largely to technological advancement, simply didn’t exist 10 or 15 years ago, or even five years ago. There never has been a real education system designed for these jobs, so it makes sense, and for many it has been essential.

“Summer youth employment with little links to related education opportunities and no real wraparound is almost the opposite of what I had been working and thinking about,” said Gerber.

Under former Mayor Vincent Gray, Gerber served as executive director of the District of Columbia’s Workforce Investment Council. It was there that she started to wonder if there was something important she was missing about summer youth jobs programs.

“What I would see every year was the line that would wrap around the building of children and parents waiting to get their name in the system,” Gerber said. “If there’s one thing I know about young people, you don’t stand in line for things you don’t want. That to me was a sign that something of value was here.”

Two years ago, Gerber started at the Annie E. Casey Foundation, based in Baltimore. She had just finished a visit to Ferguson, Missouri, to discuss and learn about the role of workforce development for youth in the aftermath of Michael Brown’s death at the hands of police. She remembers being in the plane, sitting on the tarmac, waiting to take off, when Baltimore started to rise up in response to the death of Freddie Gray while in police custody.

“Baltimore erupted. It became apparent youth were telling us we were not investing enough in young people, especially in Baltimore,” Gerber said. “That was the moment I really changed how I thought about summer youth employment.”

Gerber told that story last week at the launch event for “Youth summer jobs programs: Aligning ends and means,” a new report from the Metropolitan Policy Program at Brookings. (The Annie E. Casey Foundation funded the report.)

The report offers information and guidance to city leaders, policymakers and funders as they consider supporting larger and better summer jobs programs for youth. The authors interviewed representatives from 31 organizations working on or studying summer youth jobs programs, coming up with a core set of practices that characterized what they considered high-performing summer jobs initiatives, as well as recommendations to develop stronger summer jobs programs.

When it came to performance, “On this, we know less than we like. Until recently there was little research on impact of summer jobs programs,” said Martha Ross, co-author of the report, at the event.

While some programs like Chicago and NYC could show they reduced arrests or youth mortality, Ross pointed out, Chicago did not find an impact on grades and NYC did not find an impact on later earnings. “It’s complicated and expensive to conduct research to show causality,” she added.

Even the simplest measure, how many youth come through summer jobs programs every year, is hard to track down. One big reason why is the disappearance of dedicated federal funding for summer youth jobs programs. The Workforce Investment Act of 1998 ended the long-standing practice of standalone summer jobs programs in favor of more comprehensive, year-round services.

“The Federal Summer Youth Employment Program, which began over three decades ago as a response to urban unrest, evolved into a well-developed infrastructure connecting youth, community service, learning and earning in a symbiotic way that helped develop the workplace and personal skills of our young people and added value in the community,” wrote Linda Harris of the Center for Law and Social Policy, back in 2007.

In 2009, the federal stimulus package directed $1.2 billion to states for employment and training for youth, strongly encouraging states to support summer jobs programs. That resulted in 300,000 youth in summer employment in the summer of 2009, the report notes.

The Workforce Innovation and Opportunity Act (WIOA) went into effect in 2015, reshaping federal funding streams again, eliminating the dedicated stream for summer youth employment programs, instead lumping it in with a range of activities to support disadvantaged youth.

“The number of [youth summer jobs] participants declined by 50 to 90 percent after WIOA eliminated standalone provision for summer jobs,” Ross said.

With all the back-and-forth at the federal level, as the report notes, there is no comprehensive national census of summer jobs programs. NYC has the largest, serving 54,263 youth in 2015. Other large programs include Chicago (24,679), Los Angeles (11,644), Washington, D.C. (13,230), Boston (10,360), Philadelphia (8,813), Baltimore (8,137), San Francisco (7,937) and Detroit (5,594).

Smaller cities also run programs, such as Missoula, Montana (43 youth); Erie County, Pennsylvania (142); Charlottesville, Virginia (153); Madison, Wisconsin (415); Cincinnati (776); and Pittsburgh (1,901).

But, as Ross pointed out, “A headcount does not answer the most important question, which is whether young people leave the program better prepared than when they came in.”

Ross highlighted recommendations like measuring how many youth leave programs with name and contact info of a professional reference, or leaving with a work readiness credential, or some other certification.

Relationship-building with professional supervisors and other colleagues is a key component of a successful summer jobs program that often gets taken for granted. At the Brookings event, Ana Galeas, a high school senior from Washington, D.C., also spoke. She has taken advantage of the district’s summer jobs program to build a year-long relationship with her first summer jobs employer, DC SCORES, a youth program that combines poetry and spoken word, soccer, and service learning.

Galeas knows some who weren’t as lucky to get a placement that was as effective in providing a nurturing relationship off the bat.

“Mentoring is a big need. Some of my friends don’t know until a week before where they are going to work,” Galeas said. “They meet supervisors just before starting. We need somebody to be there for us, for networking to happen. There is a point that you have to go into the real world. This is just a program. You’re not going to stay here forever.”