http://www.researchforum.org/project_general_306.html

Rural Welfare to Work Strategies Demonstration Evaluation

General Information

View a brief abstract of this project.

View a complete, printer-friendly profile of this project.
Evaluator(s) Mathematica Policy Research, Inc.
Investigator(s) Stuart Kerachsky (Mathematica Policy Research, Inc.)
Michael Ponza (Mathematica Policy Research, Inc.)
Sponsor(s) US Department of Health and Human Services, Administration for Children and Families
Funder(s) US Department of Health and Human Services, Administration for Children, Youth, and Families
Consultant(s) Not applicable
Not applicable
 
Domain Income Security/TANF
Status Operational with Findings
Duration Sep 2000 - Sep 2005
Type Research and/or Program Evaluation
Goal To increase knowledge through information sharing and through research to provide sound information about effective approaches in working with rural populations.
Program/Policy Description This multi-year national evaluation is designed to learn how best to help TANF and other low-income rural families move from welfare to work. The evaluation will lead to increased information on well-conceived rural welfare-to-work strategies and lessons about the operational challenges and methods to address them that can be used by state and local TANF agencies and others. During the initial phase of this initiative, 10 states received planning grants to develop strategies targeted to serve rural TANF populations. These states received assistance from a Federal technical assistance contractor. In FY 2001, MPR was selected to evaluate the implementation phase of the demonstration and the first participating state, Illinois, began implementation in a five-county site.
Notes No notes reported.
 
Last Updated 01/25/05
Type of Summary Reviewed
External Reviewer(s) James Dolson (US Department of Health and Human Services, Administration for Children and Families)
Contact(s) Michael Ponza (mponza@mathematica-mpr.com)
Mathematica Policy Research, Inc.
600 Alexander Park
(T) (609) 275-2361
(F) (609) 799-0005
Publications Department Jacqueline Allen (jallen@mathematica-mpr.com)
Mathematica Policy Research, Inc.
P.O. Box 2393
(T) (609)-275-2350
(F) (609)-799-0005
Submitter(s) Research Forum Staff (info@researchforum.org)
National Center for Children In Poverty
215 West 125th St, 3rd Fl
(T) (646)284-9600
(F) not reported

Populations Studied

Target Population Recipients/participants/clients
Rural Populations
Subgroups Analyzed None
Sample Size and Unit Phase 1: States used planning grants to develop strategies targeted to serve rural populations.

Phase 2: Visits are conducted in each site 14 months after demonstration start-up. Using random assignment, individuals eligible for scarce program slots are being assigned to either the WtW program group (who are offered WtW services) or a control group (who are not offered WtW services but who may use all other available services). Follow-up interviews will be conducted with participants 15 months after program entry.

Illinois: 300 program and 300 control group participants sampled from July 2001 - December 2002.
Nebraska: 330 program and 270 control group participants sampled from April 2002 - March 2004.
Tennessee: 1,320 program and 880 control group participants sampled from October 2002 - March 2004.

Sites Studied

Phase 1:
Iowa,
Illinois,
Louisiana,
Maryland,
Minnesota,
Mississippi,
Missouri,
New York,
Vermont,
Washington.

Phase 2:
Illinois,
Nebraska,
and
Tennessee

Program Components, Policies, and Activities Evaluated

Employment activities

  • Employment Activities - misc.

Administration/Implementation

  • Administration/Implementation - misc.
Variation in program components across sites? Yes
Notes on program components Administration/Implementation: The evaluation will address what types of services or policy approaches are provided; what issues and challenges are associated with implementation; what are the net costs and benefits; and what elements of the programs policymakers should consider in designing future programs for rural welfare-to-work strategies.

Employment activities: The evaluation will address the net impacts of the welfare-to-work programs on individuals and their families.

Outcomes Assessed

Program implementation

  • Program Implementation - misc.

Financial costs and benefits/cost-effectiveness

  • Financial costs and benefits/cost-effectiveness - misc.

Employment

  • Employment - misc.

Types of Studies

Type Implementation/Process Study
Aim To identify implementation issues, draw lessons about service delivery challenges and innovations in rural areas, and provide details about how programs achieve desired results. Data will come primarily from visits to the demonstration sites. The visits will include discussions with administrators and staff of WtW programs and related agencies, focus groups with participants and control group members, and program observation. Data from surveys, as well as program records, will be used to examine sample members’ participation in program activities and use of services.
 
Type Impact Study (Controlled Experiment)
Aim To determine what difference WTW programs make in employment and family well-being outcomes.
 
Type Cost-Benefit Study
Aim To calculate estimates of net program cost-effectiveness based on data from the impact and implementation studies and published research.
 

Data Sources

Source Focus Group
Title Participant Focus Group
Sample Characteristics/Data Collection Participants will be included in focus group. Sample size not reported.
Sites All sites.
Response Rate/Attrition Notes To be reported.
Additional Execution Notes Possible data collection method. Methods vary across 10 sites.
 
Source Administrative data
Title Human Service Program Data
Sample Characteristics/Data Collection Administrative data from human service programs.
Sites All sites.
Response Rate/Attrition Notes To be reported.
Additional Execution Notes Possible data collection method. Methods vary across 10 sites.
 
Source Interview
Title Comprehensive interview
Sample Characteristics/Data Collection Conducted at baseline and 15 months after program entry.
Sample size not reported.
Sites All sites.
Response Rate/Attrition Notes To be reported.
Additional Execution Notes Possible data collection method. Methods vary across 10 sites.
 
Source Field Research
Title Agency Viisits
Sample Characteristics/Data Collection Visits are conducted 14 months after demonstration start-up. Visits include discussions with staff of WTW programs and related agencies and program observation. Sample size not reported.
Sites All sites.
Response Rate/Attrition Notes Not applicable.
Additional Execution Notes N/A
 

Findings Available

Final Implementation Findings
Interim Descriptive/Analytical Findings

Findings

04/01/01: Rural Welfare to Work Strategies Project: Planning and Promoting Welfare to Work Experiments in Rural Areas: Some Lessons of Experience
Interim Descriptive/Analytical Findings:

  • “The usual methods to actively engaging clients in employment-oriented activities becomes more expensive…services cost more when it is difficult for clients to get to offices and for agency personnel to reach clients, develop jobs, and coordinate services.”
  • “The welfare caseload in each rural office may be low, so achieving adequate sizes for meaningfully testing new initiatives will often involve multiple offices, and that means experimenters will need to recruit multiple sites and coordinate them. Dispersed operations complicate random assignment evaluations, because random assignment requires delivering at least two different service packages in each area.”
  • “While finding jobs accessible to TANF recipients can be a problem in urban areas, the problem can be even more severe for rural areas, and it is complicated when the job are provided by only a few employers. Local outcomes are likely to depend on choices made by a few firms, so incentives exist to specifically tailor innovations to the needs of those specific firms. This in turn complicates general testing across multiple sites. It can also create a vulnerable situation when success depends on the cooperation from a handful of employers. For example, staff in rural Ohio report that a major truck manufacturing plant with starting salaries as high as $14.00/hour (and good benefits) will not hire TANF participants.”
  • “The problem of choice arises in services as well. In rural areas the number of service providers is probably small, with the consequence that it is difficult to respond to underperformance--and overpricing--by switching to a different service provider. Again, this means that when experiments are undertaken, the sponsoring agency may have to take a much larger hand in ensuring that necessary services are available and delivered in accordance with the intent of the experiment than would be the case if multiple providers were available.”
  • “In urban areas employers have many opportunities to observe good results when people are hired from groups--including TANF recipients--traditionally judged to be risky. A strong economy has multiplied such examples, which means there are good chances for TANF applicants and recipients to find and keep jobs. In rural areas, the experiences tend to be further away, so welfare agencies need to make more effort to gain cooperation from employers and find ways to communicate the benefits that businesses achieve when the successfully employ workers from more challenging backgrounds.”
  • “Paradoxically, while the short supply of some kinds of knowledge may be detrimental to rural welfare to work efforts, in some cases the absence of urban anonymity can hurt workers, and the stability of rural populations can make for long memories. Front-line staff tell repeatedly about a problem that “Susan” may have getting a job because employers will remember her “Uncle Bob” as an unproductive employee who causes a lot of resentment among other staff.”
    		•
     
    10/01/02: Rural Welfare to Work Strategies Demonstration Evaluation: A Summary of the Evaluation Design and Demonstration Programs
    Interim Implementation Findings:

  • Illinois Future Steps Program: All five counties in the Future Steps service area have participated. Sample enrollment is expected to exceed the target, equaling approximately 660. MPR has created a customized management information system for Future Steps to help gather information on service delivery and, at the same time, provide records that will be useful to Future Steps staff.
  • Building Nebraska Families Program: As of October 4, 2002, the sample numbered 150, with 81 people in the program group and 69 in the control group. Enrollment will continue through March 2004, or until the target sample of 600 is reached. MPR has developed a database, the BNF Information System (BNFIS), to systematically record information on service delivery and enable BNF staff to maintain records on participants. Designed in consultation with BNF staff, the BNFIS uses printed forms and Access database software to document participants’ contact and demographic information, their initial and follow up responses to statements on the program checklist, the content of teaching sessions and other contacts with participants, and success marker ratings.
  • Tennessee First Wheels Program: Sample enrollment for the First Wheels evaluation is expected to begin in October 2002 and continue through March 2004, or until the target sample size of 2,200 is reached. Mathematica Policy Research, Inc. has developed the First Wheels Applicant Tracking Information System (FWATIS) to help implement the demonstration evaluation. The central eligibility determination worker will use this database to collect information on each applicant and track application approvals and denials. RC&D staff will collect data on loans—including the amount and terms, the type of vehicle purchased, and repayment activity—in a separate First Wheels program participant database.
    		•
     
    04/01/04: Rural Welfare-to-Work Strategies Demonstration Evaluation: Final Program Implementation Report
    Final Implementation Findings:
    • Overall, the RWtW programs’ implementation has conformed to their models. The programs all developed and benefited from earlier pilot projects that were refined by state welfare agencies and their partner organizations.
    • Program staff members, especially in Future Steps and BNF, help clients address a range of specific issues and employment barriers. To varying extents, they also identify and refer clients to various outside service providers, resources, and potential employers.
    • Although clients’ employment cannot necessarily be attributed to their participation in the demonstration programs, program records suggest that many participants in Future Steps and BNF do have some success in securing jobs. (Most First Wheels clients must already be employed to qualify for a loan.)
    • Independent partner organizations bring the advantages of location, reputation, and staff expertise to their programs. Previous collaboration experience between the state welfare agencies and their partners helped smooth program startup and operation in Illinois and Nebraska.
    • Future Steps and BNF served at least as many clients as planned. First Wheels has had difficulty reaching its goals, largely because of competing priorities in local offices and insufficient coordination on outreach between the state welfare agency and its partner organization.
    • Program staff members typically work in locations distant from program administrators and exercise substantial discretion and autonomy in their daily work. The programs have sometimes found it challenging to identify and hire people who have the combination of desired staff qualifications and who are familiar with the rural communities the programs serve.
     
    04/01/04: Rural Welfare to Work Strategies Demonstration Evaluation: Implementing Welfare-to-Work Programs in Rural Places: Lessons from the Rural Welfare-to-Work Strategies Demonstration Evaluation
    Interim Implementation Findings:
    • Overall, the Rural WtW programs' implementation has conformed to their models. The programs all developed and benefited from earlier pilot projects that were refined by state welfare agencies and their partner organizations.
    • Program staff members, esp. in Future Steps and BNF, help clients to address a range of specific issues and employment barriers. To varying extents they also identify and refer clients to various service providers, resources, and potential employers.
    • Although clients' employment cannot necessarily be attributed to their participation in the demonstration programs, program records suggest that many participants in Future Steps and BNF do have some success in securing jobs.
    • Independent partner organizations bring the advantages of location, reputation, and staff expertise to their programs. Previous collaboration experience between the state welfare agencies and their partners helped smooth program start up and operation in IL and NE.
    • Future Steps and BNF served at least as many clients as planned. First Wheels has had difficulty reaching its goals, largely because of competing priorities in local offices and insufficient coordination on outreach between state welfare agency and its partner organization.
    • Program staff members typically work in locations distant from program administrators and exercise substantial discretion and autonomy in their daily work. The programs have sometimes found it challenging to identify and hire people who have the combination of desired staff qualifications and who are familiar with the rural communities the programs serve.
     

    Recommendations

    Recommendations not yet available.
     

    Existing Publications

    04/01/01 Rural Welfare to Work Strategies Project: Planning and Promoting Welfare to Work Experiments in Rural Areas: Some Lessons of Experience Macro
    12/01/00 Rural Welfare to Work Strategies Project: Iowa Rural Welfare to Work Strategies Project: Lee County CAR Project Documentation
    12/01/00 Rural Welfare to Work Strategies Project: Rural Welfare to Work in Mississippi: The Challenge of Urban Influence
    11/01/00 Rural Welfare to Work Strategies Project: Iowa Rural Welfare to Work Strategies Project: Final Report
    08/01/00 Rural Welfare to Work Strategies Project: Final Performance Report of the Washington State Rural Welfare to Work Strategies Project
    07/01/00 Rural Welfare to Work Strategies Project: Illinois Rural Welfare to Work Strategies Project
    05/01/00 Rural Welfare to Work Strategies Project: Identifying Barriers to Finding and Keeping Employment for Rural TANF Recipients in Missouri - Final Report Macro
    04/01/00 Rural Welfare to Work Strategies Project: Transportation in Rural Communities: Strategies for Serving Welfare Participants and Low-Income Individuals Macro
    03/01/00 Rural Welfare to Work Strategies Project: Maryland Rural Welfare to Work Research Planning Grant
    02/01/00 Rural Welfare to Work Strategies Project: Microenterprise in Rural Communities: Opportunities for Welfare Participants and Low-Income Workers Macro
    11/01/99 Rural Welfare to Work Strategies Project: Welfare to Work in Vermont: A Summary by the Rural Welfare to Work Strategies Grant
    06/01/99 Rural Welfare to Work Strategies Project: Conceptual Framework for Rural Welfare to Work Strategies Macro
    06/01/99 Rural Welfare to Work Strategies Project: Research Synthesis Macro
    11/01/00 Rural Welfare to Work Strategies Project: Child Care for Welfare Participants in Rural Areas Macro
    02/01/01 Rural Welfare to Work Strategies Project: Substance Abuse and Welfare Recipients in the Rural Setting Macro
    04/01/04 Rural Welfare-to-Work Strategies Demonstration Evaluation: Final Program Implementation Report Mathematica
    10/01/02 Rural Welfare to Work Strategies Demonstration Evaluation: A Summary of the Evaluation Design and Demonstration Programs Mathematica
    04/01/04 Rural Welfare to Work Strategies Demonstration Evaluation: Implementing Welfare-to-Work Programs in Rural Places: Lessons from the Rural Welfare-to-Work Strategies Demonstration Evaluation Mathematica

    Forthcoming Publications

    00/00/00 Rural Welfare-to-Work Strategies Demonstration Evaluation: Interim Program Impact Report Mathematica
    06/01/07 Rural Welfare-to-Work Strategies Demonstration Evaluation: Final interim program impact report Mathematica
    06/01/07 Rural Welfare-to-Work Strategies Demonstration Evaluation: Final Cost-Benefit Report Mathematica

    215 W. 125th Street, 3rd Floor New York, NY 10027 646.284.9600     info@nccp.org
    This Page Copyright © 2002 The Trustees of Columbia University in the City of New York, for its National Center for Children in Poverty (NCCP). All Rights Reserved.