PDFs and tools

Measuring success

Evaluation strategies help family support programs prove they make a difference


In the 1999-2000 school year, Verde Elementary School in North Richmond had the lowest attendance rates in the West Contra Costa Unified School District. In 2001, Verde Involving Parents, a collaborative project of the North Richmond Family Resource Center and Verde Elementary School, began training community residents to reach out to parents whose children didn't show up at school (see Grassroots Snapshot--Parent-to-parent approach turns school around). In just two years, VIP could show that it contributed to a 38 percent decrease in absences, and reduced tardiness by nearly 75 percent.

While this is clearly a win for the kids and families at Verde, it also was a win for the family resource center (FRC). Since school funding is based on the number of students who attend, the FRC could show that its program brought more money into the school. This year, the district invested some of its own resources in the FRC's effort, which, in turn, leveraged additional grant funding for the VIP program.

Show an impact

When funds are tight, being able to use data to prove that your program makes a difference is "everything," says Sid Gardner, president of Children and Family Futures, an Irvine-based nonprofit that provides technical assistance to FRCs. As state and county budgets shrink and private foundations suffer stock market losses, family support programs "need to show...that they can reach families more effectively or save money," says Gardner.

Since its start in 1994, the Mutual Assistance Network of Del Paso Heights (MAN) in Sacramento has used data to make its case to funders. Now it's grown from a community gardening project to a $2.6 million agency. "It's hard to argue with success," says Executive Director Richard Dana. "I can't tell you how valuable it is to walk into a room and say: 'Five years ago, the infant low-birth-weight rate of Del Paso Heights was five times higher than the whole county. Now it's even with the county's.' We can really show an impact."

Doing the numbers

To survive in tough times, FRCs need to get more sophisticated about what information they gather and how to use the results to improve programs and raise funds, says Gardner. "The first question FRCs should ask themselves is, 'What are we best at improving?' Most will realize that their own information systems are not sufficient to answer the question." For many FRCs, developing evaluation strategies on top of meeting community needs is daunting. Here are some tips from the field:

Tracking community information

Early on, MAN got support from foundations to hire a consultant to gather "baseline" data on the community, "to determine whether our programs were truly meeting the needs of the community," says Dana. For example, high school records showed that most behavior problems were occurring among girls, while MAN's programs were mostly targeting boys. "We adjusted our program to serve more girls," says Dana.

Most of the information is collected by government agencies:

  • School districts keep track of students' behavior trends, attendance, academic achievement, special education needs, and eligibility for free and reduced-price lunch.
  • U.S. Census tracks race and ethnicity, employment, income level, and more.
  • Local police departments usually keep crime rates by zip code.
  • County human services agencies track how many families access social services.
  • Hospitals track birth rates, low-birth-weight babies, and teen pregnancy.

Some data can be accessed via the Internet, but often FRCs have to negotiate to get the information they need.  "We were running our youth programs for three years before the school began to work with us," recalls Dana. "It took a long-term approach to get our program allowed inside the [school data] system." The key was trust: "We told them: 'We're here to support you, not compete with you.'"

Tracking program information

Intake questions are critical, says Gardner. He challenges FRCs to think of three questions they could add "that would help explain how you serve the community." Age of children can be important-"if you know how many families have kids in preschool, you are in a better position to talk to First 5," says Gardner.

Universities can help: "If you have computers, ask the local university for help," says Iris Alfaro, research associate at Children and Family Futures. Graduate students hungry for evaluation experience might volunteer to create a database and input information.

Evaluation specialists are often more efficient than FRC staff, who have to learn as they go, says Dana. Often, the costs can be built into a contract or grant agreement.

Follow up with families three to six months after they leave the program. Even if you can only interview a small sample, it can make a big difference, says Don Cohon of the Institute for the Study of Community-Based Services, which has evaluated the San Francisco-based Edgewood Center's Kinship Support Network. Edgewood's first evaluation was based on 24 caregivers, but it helped leverage two major foundation grants. "We were able to demonstrate to funders that this was a need because we had data to support it."

 


Resources

  • Contra Costa County Service Integration Program, 925-313-1793
  • Family Development Matrix, California State University, Monterey Bay Institute for Community Collaborative Studies, includes many sample family development matrix forms used by FRC networks, including Solano Children's Network. http://iccs.csumb.edu/html/community/matrix

Solano County: Database gets results

The Solano County Children's Network has found that a little data collection can go a long way toward making FRCs more effective.

Reporting to funders: Each of Solano County's FRCs receives funds from many different sources, says Jenny Ocón, community programs coordinator. "All of the funders were asking for something different," and the existing database wasn't up to the task. With the help of a computer specialist, the network developed a new database that keeps track of the information that each funder needs and generates the appropriate reports.

Tracking families' progress: Solano County FRC staff also developed a "Family Development Matrix" (see resources), which measures how the family is progressing in key areas, such as housing, employment, and knowledge of resources. FRC staff members received training in updating families' progress and entering results into the database. This helps staff evaluate how families are doing. They can also print out a simple chart and use it as a tool to talk with families about their progress and their goals, says Ocón.

Generating income: The Solano County FRCs' database also allows them to get reimbursed for services they provide. The database is programmed to give county officials everything they need to bill the state for some Medi-Cal costs-so far this strategy has brought in about $20,000 in income.

It wasn't easy, says Ocón. The network had to raise funds to support a computer specialist. Making sure each FRC is properly equipped and "enters data in a consistent manner" is still a challenge. But it's been worth it: "People have come to the point that they wouldn't want to do without it. They're able to say, 'Yeah, we are making a difference.'"


Use our articles

Use the Children's Advocate in your work! Feel free to reprint these articles, as handouts or in your own publication – just credit us and be sure to send us a copy.