|
Indiana Welfare Reform Evaluation
General Information
View a brief abstract of this project.
View a complete, printer-friendly profile of this project.
| Evaluator(s) |
Abt Associates, Inc.
|
| Investigator(s) |
Erik Beecroft
(Abt Associates, Inc.)
|
| Sponsor(s) |
Indiana's Family and Social Services Administration (FSSA), Division of Family and Children
|
| Funder(s) |
Indiana's Family and Social Services Administration (FSSA), Division of Family and Children
US Department of Health and Human Services
|
| Subcontractor(s) |
Urban Institute
Indiana University
|
| |
| Domain |
Income Security/TANF
Child/Family
|
| Status |
Completed (final report released)
|
| Duration |
Jun 1995 - Jun 2002
|
| Type |
Research and/or Program Evaluation
|
| Goal |
To evaluate the implementation, impacts, and cost-effectiveness of Indianas changes in welfare policy that began in May, 1995.
|
| Program/Policy Description |
Indianas welfare reform is a comprehensive initiative that made substantial changes to both the overall AFDC program and the AFDC employment and training program, IMPACT (Indiana Manpower Placement and Comprehensive Training Program). Welfare recipients are required to participate in IMPACT if they meet federal qualifications for participation in the JOBS program ( including if the youngest child is older than 1 year, which was changed in 1997-1998 from 3 years).
Indianas welfare reforms main provisions are currently organized under three broad headings: 1) new requirements to foster personal responsibility, including a social contract- the Personal Responsibility Agreement (PRA)-; 2) a shift in employment and training services to a Work First approach; and 3) special work incentives and supports.
|
| Notes |
Includes enhanced child outcomes survey as part of the Project on State-Level Child Outcomes: Enhancing Measurement of Child Outcomes in State Welfare Evaluations and Other State Data Collections.
|
| |
| Last Updated |
09/23/03
|
| Type of Summary |
Reviewed
|
| External Reviewer(s) |
Erik Beecroft
(Abt Associates, Inc.)
|
| Contact(s) |
Erik Beecroft (erik_beecroft@abtassoc.com)
Abt Associates, Inc.
4808 Montgomery Lane
(T) (301) 913-0546
(F) (301) 652-3635
|
| Publications Department |
Abt Publications (not reported)
Abt Associates, Inc.
4808 Montgomery Lane
(T) not reported
(F) not reported
|
Populations Studied
| Target Population |
Recipients/participants/clients
Former recipients ("leavers")
Children
|
| Subgroups Analyzed |
Single parent families
Two-parent families
Children 1-6
Children 7-18
|
| Sample Size and Unit |
Approximately 66,000 welfare applicants and recipients.
Of these, 63,000 are program group (reformed AFDC and IMPACT) and 3,000 are control group (traditional AFDC and IMPACT) members.
|
Sites Studied
Indiana- statewide
Program Components, Policies, and Activities Evaluated
Employment activities
- Job readiness activities
- Job search
Financial incentives
- Elimination of 100 hour rule
- Increased asset limit
Financial disincentives/Sanctions
- Reduced benefits for non-compliance
Program requirements
- Parenting or social contract
- Paternity identification
- School attendance
- Living arrangements for unwed pregnant or parenting minors
- Immunizations for children
Time limits
Family caps
Eligibility
Administration/Implementation
- Administration/Implementation - misc.
| Variation in program components across sites? |
No
|
| Notes on program components |
Changes in eligibility: Clients applying for public assistance must undergo an additional needs assessment at the point of application.
Employment activities: IMPACT participants must spend 8 weeks in job readiness/job search activities. Applicants must register for work at employment office at the time of application.
Family caps: Benefits do not increase for births occurring more than 10 months after first welfare receipts if the child was conceived in a month when the mother was receiving welfare. Exceptions include births due to incest or sexual assault and the first birth to a minor parent.
Financial disincentives/sanctions: See program requirements. IMPACT participants are required to meet education and training requirements. Failure to meet requirements results in elimination of the adult portion of the grant for two months and increases in strictness as noncompliance continues.
Financial incentives: The State disregards child support payments during the first six months after job entry. Resource limit is increased if recipients find employment. The 100 hour rule is eliminated.
Program requirements: Clients are required to sign the Personal Responsibility Agreement. Failure to sign the agreement without good cause can lead to a $90 reduction in monthly AFDC benefits. All welfare recipients are required to submit proof that all children for whom they receive benefits have their standard childhood immunizations up to date. Noncompliance results in $90 monthly sanction until proof is received. Parents of children who do not meet local school attendance standards are required to meet with caseworkers. Continued school absence results in a $90 monthly sanction. All minor parents under the age of 18 must live with a parent, legal guardian or other related adult, or in an adult-supervised supportive living arrangement. Participants in IMPACT must spend at least 20 hours a week in work-related activities. Parents must agree to live in safe and secure homes and not use drugs or abuse alcohol. Paternity identification is required.
Time Limits: A lifetime maximum of 24 months of welfare receipt after entry (for mandatory IMPACT participants only). Children of parents who meet the time limit are still eligible for federal support.
|
Outcomes Assessed
Education
- High school graduation/GED receipt
- School attendance
Employment
- Job attainment
- Job retention
- Number of hours worked for wages
Family and relationship outcomes
- Births/pregnancies
- Family formation and stability/Living arrangements
Income security
Attitudes towards work, welfare, and program
- Attitudes towards work, welfare, and program - misc.
Service utilization
- Service utilization - misc.
Program implementation
- Program Implementation - misc.
Financial costs and benefits/cost-effectiveness
- Financial costs and benefits/cost-effectiveness - misc.
Child Outcomes
- Child overall development
Types of Studies
| Type |
Implementation/Process Study
|
| Aim |
To understand how Indiana developed its broad welfare reform goals and then translate these goals into an operational program.
To examine the programs overall design and management at the State level, and day-to-day operations at the local level, and examine individual reform components and the mechanisms for weaving them into a coherent whole.
To monitor change over time, seeking to identify the points at which key milestones, such as moving past the start-up phase to a period of steady-state operations, are achieved.
|
| |
| Type |
Impact Study (Controlled Experiment)
|
| Aim |
To measure whether the reform is having its intended effects on families receiving assistance. Key concerns are whether the reform: 1) increases total family income while reducing reliance on welfare, 2) enhances childrens well being, and 3) promotes responsible childbearing and family stability.
|
| |
| Type |
Cost-Benefit Study
|
| Aim |
To assess the effects of the reform on social and economic outcomes for families and determine whether the results were financially worthwhile.
|
| |
Data Sources
| Source |
Field Research
|
| Title |
Early site visits, key informant interviews, and observations.
|
| Sample Characteristics/Data Collection |
137 individuals interviewed, including the DFC (Division of Families and Children) county director and other administrative personnel; public assistance supervisors and caseworkers; IMPACT supervisors and family case coordinators; school personnel; outside education and training service providers; and Step Ahead Council members and staff.
41 separate observations of program activities such as prescreening, intake and assessment procedures, orientation sessions, and self-sufficiency planning interviews were conducted.
11 different local welfare offices visited in the eight research sites.
First round of site visits occurred between April and June 1996. A second round of site visits was conducted in the fall of 1998.
|
| Sites |
Clark County, Indiana
Lake County, Indiana
Madison County, Indiana
Marion County, Indiana
Miami County, Indiana
St. Joseph County, Indiana
Vigo County, Indiana
Wabash County, Indiana
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
Field interviewers used semi-structured question guides to assess implementation, and ongoing operations at the local level and document any start-up problems and operational issues affecting implementation.
While they were on site, interviewers also collected policy manuals and other written documentation to supplement material from interviews and observations.
|
| |
| Source |
Interview
|
| Title |
FSSA (Family and Social Services Administration)Central Office staff
involved in welfare reform
|
| Sample Characteristics/Data Collection |
Data collected March, 1996
|
| Sites |
Indianapolis, Indiana
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Administrative data
|
| Title |
Indiana Client Eligibility System (ICES) records
|
| Sample Characteristics/Data Collection |
Records on program participation (number of records not reported.)
Data collected 1995 through end of study.
|
| Sites |
Indiana-statewide
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
Data collected on socioeconomic characteristics and welfare experiences.
|
| |
| Source |
Survey
|
| Title |
Periodic mail survey
|
| Sample Characteristics/Data Collection |
Respondents in each county may include county DFC (Division of Family and Children) directors, IMPACT managers, and a selected number of public assistance caseworkers and IMPACT family case coordinators.
First survey wave collected spring 1997. Second survey conducted Fall 1999.
|
| Sites |
Indiana-statewide
|
| Response Rate/Attrition Notes |
>90% for both surveys
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Administrative data
|
| Title |
Unemployment Insurance (UI) records;
Wage data records
|
| Sample Characteristics/Data Collection |
Data obtained from the Department of Workforce Development.
Collected January 1994 through present.
|
| Sites |
Indiana-statewide
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Survey
|
| Title |
Client Follow-up survey
|
| Sample Characteristics/Data Collection |
1,570 welfare clients.
Sample of program and control group members.
Data collected in Spring, 1997.
|
| Sites |
Indiana-statewide
|
| Response Rate/Attrition Notes |
Reported response rate: 71%
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Administrative data
|
| Title |
Caseload data
|
| Sample Characteristics/Data Collection |
Data at state and county level provided from FSSAs Office of Reports and Statistics (number of records not reported).
Data collected 1990 through present.
|
| Sites |
Indiana-statewide
|
| Response Rate/Attrition Notes |
N/A
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Survey
|
| Title |
Wave 2 Client Follow-Up Survey
|
| Sample Characteristics/Data Collection |
2,350 clients, equally divided between program and control group members. Data collected March through November 2000.
|
| Sites |
Indiana - statewide
|
| Response Rate/Attrition Notes |
Interviewing not yet complete as of October 2000.
|
| Additional Execution Notes |
No notes reported.
|
| |
| Source |
Survey
|
| Title |
Child Outcomes Survey
|
| Sample Characteristics/Data Collection |
1,850 clients with children age 5-12 at the time of the survey equally divided between program and control group members. Data collected March through November 2000.
|
| Sites |
Indiana - statewide
|
| Response Rate/Attrition Notes |
Interviewing not yet complete as of October 2000.
|
| Additional Execution Notes |
No notes reported.
|
| |
Findings Available
Interim Implementation Findings
Interim Impact Findings
Final Impact Findings
Findings
|
08/01/97:
Indiana Welfare Reform Evaluation: Assessing Program Implementation and Early Impacts on Cash Assistance
|
|
Interim Implementation Findings:
The majority of features included in Indianas ambitious and comprehensive welfare reform were successfully implemented and fully operational(vi).
There were four areas in which implementation was not wholly successful. These areas included: (1) enhanced intake, (2) the school attendance requirement, (3) case management for IMPACT clients, and (4) completely limiting communications about welfare reform provisions with the control group(vii).
Interim Impact Findings:
One general finding is that many clients leave welfare fairly quickly with no additional incentives or assistance from the agency(xi).
Analysis of early impacts on AFDC outcomes reveals comparatively large negative impacts on average total welfare payments(xii).
Impacts for mandatory clients could have several possible sources, including intensified E&T participation, and the shift to a Work First model, and time limits and earnings incentives specific to the Placement task. The weight of evidence thus far points to an invigorated IMPACT program. In particular, process study findings suggest the transition to a Work First approach exerted far more pervasive effects on both staff and clients at the local level than technical provisions such as time limits and earnings incentives(xiv).
The reform led to lower fractions of treatment than control families receiving AFDC payments, an effect also concentrated among IMPACT-mandatory clients(xiv).
The reform had much smaller effects on the percent of families remaining technically eligible for AFDC- that is, those the State considered to be active AFDC cases- than on the percent actually receiving payments...The most likely explanation for this discrepancy is that the reform seeks to encourage work and make work pay by allowing clients in the placement track to retain eligibility for AFDC-related child care, Medicaid, and other supportive services even after their earnings exceed the payment standard (and result in a zero grant)(xiv).
Relatively large impacts on payment receipt, especially for recent applicants, point to a substantial role for welfare reform in Indianas impressive recent caseload declines(xv).
|
| |
|
11/01/98:
Indiana Welfare Reform Evaluation: Program Implementation and Economic Impacts After Two Years
|
|
Interim Implementation Findings:
"The States employment and training program shifted to a Work First model" (ii).
"Indiana implemented a two-track service model for more and less job ready clients" (iv).
"Some signs suggest time limits influence on behavior will grow" (v).
"Indiana implemented personal responsibility agreements (PRA), but the programs emphasis on parenting responsibilities was not as forceful as its emphasis on work" (vi).
Interim Impact Findings:
"Many recipients subject to the Indiana reform worked and many left welfare over the two-year follow-up period"(ix).
"Welfare reform contributed to increases in employment and decreases in welfare in Indiana. Impacts were concentrated among job-ready recipients without young children" (x).
"Impact patterns suggests employment impacts derive from more rapid job entry" (xiii).
"The Zero-Grant policy dampened impacts on TANF eligibility and slightly increased the accumulation of time on lifetime assistance clocks" (xiii).
"Maintaining employment remains a substantial challenge for most recipients who go to work" (xiv).
"Welfare reform increased the share of income from earnings, but not average total income" (xv).
|
| |
|
06/01/02:
Indiana Welfare Reform Evaluation: The Impacts of Welfare Reform on Children (Final Report)
|
Final Impact Findings
For elementary school-age children (ages 5 to 13):
- Overall, analyses of the survey data suggest the program did not have large impacts on elementary school-age children (those between 5 and 12).
- Welfare reform had no statistically significant impacts on education or health outcomes
- For outcomes involving social behavior and emotional well-being, the study found only one statistically significant impact: the increase in arrest rates for the subset of elementary school-age children who were 10 to 12 years old at the time. It is not clear whether this is a real effect of welfare reform or only a chance artifact of multiple significance tests.
- The program did not affect average household income.
- The program did increase adult's employment, which was accompanied by a slight increase in child care use.
Findings for adolescents (ages 13 to 17):
- For all but one of the small number of outcomes available for adolescents, the study found no statistically significant impact of welfare reform. The exception was a small adverse effect on adolescents' school performance
|
| |
|
09/01/03:
Indiana Welfare Reform Evaluation: The Indiana Welfare Reform Evaluation: Five-Year Impacts, Implementation, Costs and Benefits
|
Final Implementation Findings
- Indiana implemented a strong work first program.
- The switch to an integrated worker model posed challenges.
- Welfare reform led to increased local control over service delivery and reliance on contracting for services.
- Indiana made ongoing changes to its performance-based contracting process.
Final Descriptive/Analytical Findings
- For both the first-year and later cohorts, Indianas program produced positive and statistically significant impacts on overall IMPACT participation rates and on rates of participation in each of the three IMPACT activity types.
- Only about 7% of Welfare Reform group members reached the 24 month adult time limit within five years of follow up.
- Compared to their likelihood of reaching the time limit, Welfare Reform group members were twice as likely to have had a family cap birth.
- For the first year cohort, the Personal Responsibility Agreement (PRA) sanctions were more likely to be applied than the family cap or the 24 month limit.
Final Impact Findings
First Cohort
- Indiana's program reduced receipt of TANF and food stamps.
- Indiana's welfare reform program increased earnings and employment rates in each of the follow-up years.
- Welfare reform did not affect total income.
- Welfare reform did not affect health insurance rates for adults or children.
- Indiana's program did not affect reported child maltreatment.
Later Cohort
- TANF payments for the later cohort were larger than the former cohort.
- Indianas program increased the average employment rate across the eight follow-up quarters but did not significantly increase earnings.
- Indianas program did not produce impacts on income (measured as the sum of earnings, TANF payments, and food stamp benefits).
- Welfare reform may have decreased substantiated child maltreatment reports.
Final Cost/Benefit Findings
- The economic benefits of welfare reform of families, resulting mainly from increased employment, slightly outweighed the loss of welfare payments and other income.
- Welfare reform benefited tax payers because savings more than offset welfare expenditures.
|
| |
Recommendations
|
Indiana Welfare Reform Evaluation: Assessing Program Implementation and Early Impacts on Cash Assistance (08/01/97)
|
|
Implementation recommendations:
To strengthen the reforms message from the outset, we recommend that local offices adhere to the original design of having a public assistance caseworker hold the enhanced intake separate from the eligibility interview and expand the enhanced intake interview to include a careful review of the PRA with the client(viii).
Improved safeguards against exposing control group members to the reform are needed, especially at intake. Currently planned responses include: moving the ICES random assignment module to an earlier stage in applicant processing, assigning workers to specialize in serving control group members in each office, developing more written procedures, providing specialized training, carefully monitoring office procedures and providing ongoing reinforcement of these efforts(viii).
As the Central Office develops policies related to the upcoming modifications in the school attendance requirement, it should consider incorporating further policy guidance on verification procedures and on developing and monitoring attendance improvement plans(viii).
The State should take steps to strengthen case management. Resources are the key barrier to furthering this goal and increasing staffing is one possible response. If it is not feasible, the State should consider strategies for targeting case management on populations that are most likely to need or benefit from it, such as: former clients returning to welfare, individuals in sanction status, and clients nearing the time limit(viii).
As the State implements modifications to its original waiver, it would be beneficial to provide more detailed staff training on the new procedures both at the outset and on an ongoing basis(viii).
More attention should be given to providing clients a complete and detailed account of how time limits operate. In order to bring home the reality of time limits, additional efforts should be made to remind clients how many months of assistance they have used up (and how many months they have left) at every scheduled appointment with eligibility workers, IMPACT workers, and outside provider staff(viii).
Impact Recommendations:
After having done its best to help the most responsive clients achieve self-sufficiency, the State may face greater challenges as it works with the more disadvantaged clients who remain on welfare. On the other hand, recent policy changes increase greatly the number of clients exposed to Work First, time limits, and make work pay provisions. These changes suggest intriguing questions about the future relationship between welfare reform and caseload change, questions that will be addressed in upcoming reports(xvi).
|
| |
|
Indiana Welfare Reform Evaluation: Program Implementation and Economic Impacts After Two Years (11/01/98)
|
|
Interim recommendations:
"Such strategies may represent an important step, but are not in themselves enough, to move a substantial number of welfare recipients to true self-sufficiency. This recognition has impelled many states to invest substantial time and funds in building on the basic Work First framework"(xvi).
|
| |
Existing Publications
|