http://www.researchforum.org/project_general_75.html

Indiana Welfare Reform Evaluation

General Information

View a brief abstract of this project.

View a complete, printer-friendly profile of this project.

Evaluator(s) Abt Associates, Inc.
Investigator(s) Erik Beecroft (Abt Associates, Inc.)
Sponsor(s) Indiana's Family and Social Services Administration (FSSA), Division of Family and Children
Funder(s) Indiana's Family and Social Services Administration (FSSA), Division of Family and Children
US Department of Health and Human Services
Subcontractor(s) Urban Institute
Indiana University
 
Domain Income Security/TANF
Child/Family
Status Completed (final report released)
Duration Jun 1995 - Jun 2002
Type Research and/or Program Evaluation
Goal To evaluate the implementation, impacts, and cost-effectiveness of Indiana’s changes in welfare policy that began in May, 1995.
Program/Policy Description Indiana’s welfare reform is a comprehensive initiative that made substantial changes to both the overall AFDC program and the AFDC employment and training program, IMPACT (Indiana Manpower Placement and Comprehensive Training Program). Welfare recipients are required to participate in IMPACT if they meet federal qualifications for participation in the JOBS program ( including if the youngest child is older than 1 year, which was changed in 1997-1998 from 3 years).

Indiana’s welfare reform’s main provisions are currently organized under three broad headings: 1) new requirements to foster personal responsibility, including a social contract- the Personal Responsibility Agreement (PRA)-; 2) a shift in employment and training services to a “Work First” approach; and 3) special work incentives and supports.

Notes Includes enhanced child outcomes survey as part of the Project on State-Level Child Outcomes: Enhancing Measurement of Child Outcomes in State Welfare Evaluations and Other State Data Collections.
 
Last Updated 09/23/03
Type of Summary Reviewed
External Reviewer(s) Erik Beecroft (Abt Associates, Inc.)
Contact(s) Erik Beecroft (erik_beecroft@abtassoc.com)
Abt Associates, Inc.
4808 Montgomery Lane
(T) (301) 913-0546
(F) (301) 652-3635
Publications Department Abt Publications (not reported)
Abt Associates, Inc.
4808 Montgomery Lane
(T) not reported
(F) not reported

Populations Studied

Target Population Recipients/participants/clients
Former recipients ("leavers")
Children
Subgroups Analyzed Single parent families
Two-parent families
Children 1-6
Children 7-18
Sample Size and Unit Approximately 66,000 welfare applicants and recipients. Of these, 63,000 are program group (reformed AFDC and IMPACT) and 3,000 are control group (traditional AFDC and IMPACT) members.

Sites Studied

Indiana- statewide

Program Components, Policies, and Activities Evaluated

Employment activities

  • Job readiness activities
  • Job search

Financial incentives

  • Elimination of 100 hour rule
  • Increased asset limit

Financial disincentives/Sanctions

  • Reduced benefits for non-compliance

Program requirements

  • Parenting or social contract
  • Paternity identification
  • School attendance
  • Living arrangements for unwed pregnant or parenting minors
  • Immunizations for children

Time limits

  • Time Limits - misc.

Family caps

  • Family Caps - misc.

Eligibility

  • Eligibility - misc.

Administration/Implementation

  • Administration/Implementation - misc.
Variation in program components across sites? No
Notes on program components Changes in eligibility: Clients applying for public assistance must undergo an additional needs assessment at the point of application.

Employment activities: IMPACT participants must spend 8 weeks in job readiness/job search activities. Applicants must register for work at employment office at the time of application.

Family caps: Benefits do not increase for births occurring more than 10 months after first welfare receipts if the child was conceived in a month when the mother was receiving welfare. Exceptions include births due to incest or sexual assault and the first birth to a minor parent.

Financial disincentives/sanctions: See program requirements. IMPACT participants are required to meet education and training requirements. Failure to meet requirements results in elimination of the adult portion of the grant for two months and increases in strictness as noncompliance continues.

Financial incentives: The State disregards child support payments during the first six months after job entry. Resource limit is increased if recipients find employment. The 100 hour rule is eliminated.

Program requirements: Clients are required to sign the Personal Responsibility Agreement. Failure to sign the agreement without good cause can lead to a $90 reduction in monthly AFDC benefits. All welfare recipients are required to submit proof that all children for whom they receive benefits have their standard childhood immunizations up to date. Noncompliance results in $90 monthly sanction until proof is received. Parents of children who do not meet local school attendance standards are required to meet with caseworkers. Continued school absence results in a $90 monthly sanction. All minor parents under the age of 18 must live with a parent, legal guardian or other related adult, or in an adult-supervised supportive living arrangement. Participants in IMPACT must spend at least 20 hours a week in work-related activities. Parents must agree to live in “safe and secure” homes and not use drugs or abuse alcohol. Paternity identification is required.

Time Limits: A lifetime maximum of 24 months of welfare receipt after entry (for mandatory IMPACT participants only). Children of parents who meet the time limit are still eligible for federal support.

Outcomes Assessed

Education

  • High school graduation/GED receipt
  • School attendance

Employment

  • Job attainment
  • Job retention
  • Number of hours worked for wages

Family and relationship outcomes

  • Births/pregnancies
  • Family formation and stability/Living arrangements

Income security

  • Welfare receipt

Attitudes towards work, welfare, and program

  • Attitudes towards work, welfare, and program - misc.

Service utilization

  • Service utilization - misc.

Program implementation

  • Program Implementation - misc.

Financial costs and benefits/cost-effectiveness

  • Financial costs and benefits/cost-effectiveness - misc.

Child Outcomes

  • Child overall development

Types of Studies

Type Implementation/Process Study
Aim To understand how Indiana developed its broad welfare reform goals and then translate these goals into an operational program.

To examine the program’s overall design and management at the State level, and day-to-day operations at the local level, and examine individual reform components and the mechanisms for weaving them into a coherent whole.

To monitor change over time, seeking to identify the points at which key milestones, such as moving past the “start-up” phase to a period of “steady-state” operations, are achieved.

 
Type Impact Study (Controlled Experiment)
Aim To measure whether the reform is having its intended effects on families receiving assistance. Key concerns are whether the reform: 1) increases total family income while reducing reliance on welfare, 2) enhances children’s well being, and 3) promotes responsible childbearing and family stability.
 
Type Cost-Benefit Study
Aim To assess the effects of the reform on social and economic outcomes for families and determine whether the results were financially worthwhile.
 

Data Sources

Source Field Research
Title Early site visits, key informant interviews, and observations.
Sample Characteristics/Data Collection 137 individuals interviewed, including the DFC (Division of Families and Children) county director and other administrative personnel; public assistance supervisors and caseworkers; IMPACT supervisors and family case coordinators; school personnel; outside education and training service providers; and Step Ahead Council members and staff.

41 separate observations of program activities such as prescreening, intake and assessment procedures, orientation sessions, and self-sufficiency planning interviews were conducted.

11 different local welfare offices visited in the eight research sites. First round of site visits occurred between April and June 1996.
A second round of site visits was conducted in the fall of 1998.

Sites Clark County, Indiana
Lake County, Indiana
Madison County, Indiana
Marion County, Indiana
Miami County, Indiana
St. Joseph County, Indiana
Vigo County, Indiana
Wabash County, Indiana
Response Rate/Attrition Notes N/A
Additional Execution Notes Field interviewers used semi-structured question guides to assess implementation, and ongoing operations at the local level and document any start-up problems and operational issues affecting implementation.

While they were on site, interviewers also collected policy manuals and other written documentation to supplement material from interviews and observations.

 
Source Interview
Title FSSA (Family and Social Services Administration)Central Office staff involved in welfare reform
Sample Characteristics/Data Collection Data collected March, 1996
Sites Indianapolis, Indiana
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Administrative data
Title Indiana Client Eligibility System (ICES) records
Sample Characteristics/Data Collection Records on program participation (number of records not reported.)
Data collected 1995 through end of study.
Sites Indiana-statewide
Response Rate/Attrition Notes N/A
Additional Execution Notes Data collected on socioeconomic characteristics and welfare experiences.
 
Source Survey
Title Periodic mail survey
Sample Characteristics/Data Collection Respondents in each county may include county DFC (Division of Family and Children) directors, IMPACT managers, and a selected number of public assistance caseworkers and IMPACT family case coordinators.
First survey wave collected spring 1997. Second survey conducted Fall 1999.
Sites Indiana-statewide
Response Rate/Attrition Notes >90% for both surveys
Additional Execution Notes No notes reported.
 
Source Administrative data
Title Unemployment Insurance (UI) records; Wage data records
Sample Characteristics/Data Collection Data obtained from the Department of Workforce Development.
Collected January 1994 through present.
Sites Indiana-statewide
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Survey
Title Client Follow-up survey
Sample Characteristics/Data Collection 1,570 welfare clients.
Sample of program and control group members.
Data collected in Spring, 1997.
Sites Indiana-statewide
Response Rate/Attrition Notes Reported response rate: 71%
Additional Execution Notes No notes reported.
 
Source Administrative data
Title Caseload data
Sample Characteristics/Data Collection Data at state and county level provided from FSSA’s Office of Reports and Statistics (number of records not reported).
Data collected 1990 through present.
Sites Indiana-statewide
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Survey
Title Wave 2 Client Follow-Up Survey
Sample Characteristics/Data Collection 2,350 clients, equally divided between program and control group members.
Data collected March through November 2000.
Sites Indiana - statewide
Response Rate/Attrition Notes Interviewing not yet complete as of October 2000.
Additional Execution Notes No notes reported.
 
Source Survey
Title Child Outcomes Survey
Sample Characteristics/Data Collection 1,850 clients with children age 5-12 at the time of the survey equally divided between program and control group members.
Data collected March through November 2000.
Sites Indiana - statewide
Response Rate/Attrition Notes Interviewing not yet complete as of October 2000.
Additional Execution Notes No notes reported.
 

Findings Available

Interim Implementation Findings
Interim Impact Findings
Final Impact Findings

Findings

08/01/97: Indiana Welfare Reform Evaluation: Assessing Program Implementation and Early Impacts on Cash Assistance
Interim Implementation Findings:

“The majority of features included in Indiana’s ambitious and comprehensive welfare reform were successfully implemented and fully operational”(vi).

“There were four areas in which implementation was not wholly successful. These areas included: (1) enhanced intake, (2) the school attendance requirement, (3) case management for IMPACT clients, and (4) completely limiting communications about welfare reform provisions with the control group”(vii).

Interim Impact Findings:

“One general finding is that many clients leave welfare fairly quickly with no additional incentives or assistance from the agency”(xi).

“Analysis of early impacts on AFDC outcomes reveals comparatively large negative impacts on average total welfare payments”(xii).

“Impacts for mandatory clients could have several possible sources, including intensified E&T participation, and the shift to a Work First model, and time limits and earnings incentives specific to the Placement task. The weight of evidence thus far points to an invigorated IMPACT program. In particular, process study findings suggest the transition to a Work First approach exerted far more pervasive effects on both staff and clients at the local level than technical provisions such as time limits and earnings incentives”(xiv).

“The reform led to lower fractions of treatment than control families receiving AFDC payments, an effect also concentrated among IMPACT-mandatory clients”(xiv).

“The reform had much smaller effects on the percent of families remaining technically eligible for AFDC- that is, those the State considered to be “active” AFDC cases- than on the percent actually receiving payments...The most likely explanation for this discrepancy is that the reform seeks to encourage work and “make work pay” by allowing clients in the placement track to retain eligibility for AFDC-related child care, Medicaid, and other supportive services even after their earnings exceed the payment standard (and result in a “zero grant”)”(xiv).

“Relatively large impacts on payment receipt, especially for recent applicants, point to a substantial role for welfare reform in Indiana’s impressive recent caseload declines”(xv).

 
11/01/98: Indiana Welfare Reform Evaluation: Program Implementation and Economic Impacts After Two Years
Interim Implementation Findings:

"The State’s employment and training program shifted to a Work First model" (ii).

"Indiana implemented a two-track service model for more and less job ready clients" (iv).

"Some signs suggest time limits’ influence on behavior will grow" (v).

"Indiana implemented personal responsibility agreements (PRA), but the program’s emphasis on parenting responsibilities was not as forceful as its emphasis on work" (vi).

Interim Impact Findings:

"Many recipients subject to the Indiana reform worked and many left welfare over the two-year follow-up period"(ix).

"Welfare reform contributed to increases in employment and decreases in welfare in Indiana. Impacts were concentrated among job-ready recipients without young children" (x).

"Impact patterns suggests employment impacts derive from more rapid job entry" (xiii).

"The Zero-Grant policy dampened impacts on TANF eligibility and slightly increased the accumulation of time on lifetime assistance clocks" (xiii).

"Maintaining employment remains a substantial challenge for most recipients who go to work" (xiv).

"Welfare reform increased the share of income from earnings, but not average total income" (xv).

 
06/01/02: Indiana Welfare Reform Evaluation: The Impacts of Welfare Reform on Children (Final Report)
Final Impact Findings For elementary school-age children (ages 5 to 13):
  • Overall, analyses of the survey data suggest the program did not have large impacts on elementary school-age children (those between 5 and 12).
  • Welfare reform had no statistically significant impacts on education or health outcomes
  • For outcomes involving social behavior and emotional well-being, the study found only one statistically significant impact: the increase in arrest rates for the subset of elementary school-age children who were 10 to 12 years old at the time. It is not clear whether this is a real effect of welfare reform or only a chance artifact of multiple significance tests.
  • The program did not affect average household income.
  • The program did increase adult's employment, which was accompanied by a slight increase in child care use.
Findings for adolescents (ages 13 to 17):
  • For all but one of the small number of outcomes available for adolescents, the study found no statistically significant impact of welfare reform. The exception was a small adverse effect on adolescents' school performance
 
09/01/03: Indiana Welfare Reform Evaluation: The Indiana Welfare Reform Evaluation: Five-Year Impacts, Implementation, Costs and Benefits
Final Implementation Findings
  • Indiana implemented a strong work first program.
  • The switch to an integrated worker model posed challenges.
  • Welfare reform led to increased local control over service delivery and reliance on contracting for services.
  • Indiana made ongoing changes to its performance-based contracting process.
Final Descriptive/Analytical Findings
  • For both the first-year and later cohorts, Indiana’s program produced positive and statistically significant impacts on overall IMPACT participation rates and on rates of participation in each of the three IMPACT activity types.
  • Only about 7% of Welfare Reform group members reached the 24 month adult time limit within five years of follow up.
  • Compared to their likelihood of reaching the time limit, Welfare Reform group members were twice as likely to have had a family cap birth.
  • For the first year cohort, the Personal Responsibility Agreement (PRA) sanctions were more likely to be applied than the family cap or the 24 month limit.
Final Impact Findings
    First Cohort
  • Indiana's program reduced receipt of TANF and food stamps.
  • Indiana's welfare reform program increased earnings and employment rates in each of the follow-up years.
  • Welfare reform did not affect total income.
  • Welfare reform did not affect health insurance rates for adults or children.
  • Indiana's program did not affect reported child maltreatment.
    Later Cohort
  • TANF payments for the later cohort were larger than the former cohort.
  • Indiana’s program increased the average employment rate across the eight follow-up quarters but did not significantly increase earnings.
  • Indiana’s program did not produce impacts on income (measured as the sum of earnings, TANF payments, and food stamp benefits).
  • Welfare reform may have decreased substantiated child maltreatment reports.
Final Cost/Benefit Findings
  • The economic benefits of welfare reform of families, resulting mainly from increased employment, slightly outweighed the loss of welfare payments and other income.
  • Welfare reform benefited tax payers because savings more than offset welfare expenditures.
 

Recommendations

Indiana Welfare Reform Evaluation: Assessing Program Implementation and Early Impacts on Cash Assistance (08/01/97)
Implementation recommendations:

“To strengthen the reform’s message from the outset, we recommend that local offices adhere to the original design of having a public assistance caseworker hold the enhanced intake separate from the eligibility interview and expand the enhanced intake interview to include a careful review of the PRA with the client”(viii).

“Improved safeguards against exposing control group members to the reform are needed, especially at intake. Currently planned responses include: moving the ICES random assignment module to an earlier stage in applicant processing, assigning workers to specialize in serving control group members in each office, developing more written procedures, providing specialized training, carefully monitoring office procedures and providing ongoing reinforcement of these efforts”(viii).

“As the Central Office develops policies related to the upcoming modifications in the school attendance requirement, it should consider incorporating further policy guidance on verification procedures and on developing and monitoring attendance improvement plans”(viii).

“The State should take steps to strengthen case management. Resources are the key barrier to furthering this goal and increasing staffing is one possible response. If it is not feasible, the State should consider strategies for targeting case management on populations that are most likely to need or benefit from it, such as: former clients returning to welfare, individuals in sanction status, and clients nearing the time limit”(viii).

“As the State implements modifications to its original waiver, it would be beneficial to provide more detailed staff training on the new procedures both at the outset and on an ongoing basis”(viii).

“More attention should be given to providing clients a complete and detailed account of how time limits operate. In order to “bring home the reality” of time limits, additional efforts should be made to remind clients how many months of assistance they have used up (and how many months they have left) at every scheduled appointment with eligibility workers, IMPACT workers, and outside provider staff”(viii).

Impact Recommendations:

“After having done its best to help the most responsive clients achieve self-sufficiency, the State may face greater challenges as it works with the more disadvantaged clients who remain on welfare. On the other hand, recent policy changes increase greatly the number of clients exposed to Work First, time limits, and make work pay provisions. These changes suggest intriguing questions about the future relationship between welfare reform and caseload change, questions that will be addressed in upcoming reports”(xvi).

 
Indiana Welfare Reform Evaluation: Program Implementation and Economic Impacts After Two Years (11/01/98)
Interim recommendations:

"Such strategies may represent an important step, but are not in themselves enough, to move a substantial number of welfare recipients to true self-sufficiency. This recognition has impelled many states to invest substantial time and funds in building on the basic Work First framework"(xvi).

 

Existing Publications

08/01/97 Indiana Welfare Reform Evaluation: Assessing Program Implementation and Early Impacts on Cash Assistance Abt
11/01/98 Indiana Welfare Reform Evaluation: Program Implementation and Economic Impacts After Two Years Abt
09/01/97 Indiana Welfare Reform Evaluation: Who Is On and Who Is Off: Comparing Characteristics and Outcomes for Current and Former TANF Recipients Abt
06/01/02 Indiana Welfare Reform Evaluation: The Impacts of Welfare Reform on Children (Final Report) Abt
01/01/98 Indiana Welfare Reform Evaluation: Project on State-Level Child Outcomes Child Trends
09/01/03 Indiana Welfare Reform Evaluation: The Indiana Welfare Reform Evaluation: Five-Year Impacts, Implementation, Costs and Benefits Abt