http://www.researchforum.org/project_general_37.html

GAIN Evaluation

General Information

View a brief abstract of this project.

View a complete, printer-friendly profile of this project.

Evaluator(s) MDRC
Investigator(s) James Riccio (MDRC)
Daniel Freedlander (MDRC)
Stephen Freedman (MDRC)
Sponsor(s) California Department of Social Services
Funder(s) California Department of Social Services
US Department of Health and Human Services
Subcontractor(s) Not applicable
 
Domain Income Security/TANF
Status Completed (final report released)
Duration Jul 1989 - Dec 1997
Type Research and/or Program Evaluation
Goal To evaluate the components of California Greater Avenues for Independence (GAIN).
Program/Policy Description GAIN aims to increase employment and foster self-sufficiency among people receiving Aid to Families with Dependent Children (AFDC). GAIN was originally mandatory for all welfare recipients except single parents with children under the age of six (and was later reduced to three under JOBS). Welfare recipients are exempt from remaining registered in the program if they obtain a full-time job or are chronically ill. GAIN’s regulations permit temporary deferral from the participation requirement for those who have a part-time job, temporary illness, family emergency, or another situation that precludes attending an activity.
Notes No notes reported.
 
Last Updated 08/07/00
Type of Summary Reviewed
External Reviewer(s) James Riccio (MDRC)
Contact(s) James Riccio (not reported)
MDRC
16 East 34th Street
19th Floor
(T) (212)-340-8822
(F) (212)-684-0832
Publications Department MDRC Publications (publications@mdrc.org)
MDRC
16 East 34th Street
19th Floor
(T) (212) 532-3200
(F) (212) 684-0832

Populations Studied

Target Population Recipients/participants/clients
Applicants
Subgroups Analyzed Single parent families
Sample Size and Unit 33,000 welfare applicants and recipients.

Random sample of program group (assigned to participate in GAIN) and control group (not allowed to participate in GAIN but able to use other community services) members.

Sites Studied

Alameda county (Oakland), California
Butte county, California
Los Angeles county (Los Angeles), California
Riverside county (Riverside), California
San Diego county (San Diego), California
Tulare county, California

Program Components, Policies, and Activities Evaluated

Employment activities

  • Job skills training
  • Job readiness activities
  • Job search
  • Job development

Educational activities

  • Adult Basic Education (ABE) courses
  • English as a Second Language (ESL)
  • GED courses

Financial incentives

  • Coverage for work-related expenses
  • Financial Incentives - misc.

Financial disincentives/Sanctions

  • Reduced benefits for non-compliance

Program requirements

  • Work requirement
  • Community or alternative work

Social/Support services

  • Child care
  • Transitional child care
  • Transitional health benefits
  • Transportation
  • Case management
Variation in program components across sites? Yes
Notes on program components Educational activities: Registrants who do not have a high school diploma or a GED certificate, score low on the reading or math part of the CASAS basic skills test, or are not proficient in English, are determined to be "in need of basic education" and must enter an appropriate program (GED, ABE, or ESL).

Employment activities: Registrants participate in a job search activity which includes job-seeking and interviewing skills for three weeks. Registrants who are participating in an appropriate education or employment activity prior to registration in the GAIN program are referred to as participating in "self-initiated" programs and may continue to receive GAIN support for two years. Registrants who complete their training activities who have not found jobs may participate in vocational or on-the-job training, PREP (non-paid employment experience in a public or non-profit agency), supported work, or other forms of "post-assessment" training.

Financial disincentives/sanctions: Failure to comply with program rules results in a sanction (a reduction of the monthly welfare grant).

Program operations: Program implementation is studied.

Program requirements: Participants must complete work or community work requirement.

Support Services: GAIN helps registrants find, and pays for, child care services for children under 13. Child care assistance continues for a one-year transitional period if the registrant becomes employed. GAIN also reimburses participants for transportation costs. Participants may also receive assistance for program expenses such as books and tools. Case managers monitor the participants activities and authorize support services and sanctions. In Riverside county, a special study was conducted on the effects of assigning GAIN registrants to case managers with different-size caseloads. Case managers in one group (the "enhanced" group) were assigned half as many registrants as case managers in the other group (the "regular" group).

Outcomes Assessed

Education

  • High school graduation/GED receipt
  • Education - misc.

Employment

  • Job readiness/training
  • Job attainment
  • Job retention

Family and relationship outcomes

  • Births/pregnancies
  • Family formation and stability/Living arrangements

Income security

  • Earnings
  • Food stamps receipt
  • Welfare receipt

Attitudes towards work, welfare, and program

  • Attitudes towards work, welfare, and program - misc.

Standard of living

  • Standard of living - misc.

Service utilization

  • Service utilization - misc.

Program implementation

  • Program Implementation - misc.

Health/ physical well-being (including prenatal health)

  • Health/ physical well-being - misc.

Financial costs and benefits/cost-effectiveness

  • Financial costs and benefits/cost-effectiveness - misc.

Types of Studies

Type Implementation/Process Study
Aim To study the operational issues related to the GAIN program and their effects on program participants and staff.
 
Type Impact Study (Controlled Experiment)
Aim To determine the rates of employment, AFDC receipt, and Food Stamp receipt and the average earnings, AFDC payments, and Food Stamp payments for individuals registered in GAIN.

To measure the differences in the outcomes for program enrollees compared to what their outcomes would have been in the absence of GAIN.

 
Type Cost-Benefit Study
Aim To measure the benefits and costs of the GAIN program from the welfare sample perspective, the government budget perspective, the taxpayer perspective, and the perspective of society as a whole.
 

Data Sources

Source Administrative data
Title Comprehensive Adult Student Assessment Program (CASAS)
Sample Characteristics/Data Collection 33,000 welfare recipients.
Sample of all program and control group members.
Collected prior to random assignment.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes Basic reading and math test included in assessment.
 
Source Administrative data
Title Automated Welfare Payment Records measuring welfare receipt/payment levels and Food Stamp benefits
Sample Characteristics/Data Collection Records for 33,000 welfare recipients.
Sample of all program and control group members.
Collected up to 2 years prior to random assignment and through 6/93.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Administrative data
Title California State Unemployment Insurance Earnings and Benefits Records measuring employment/earnings data and Unemployment Insurance (UI) benefit payments
Sample Characteristics/Data Collection Records for 33,000 welfare recipients.
Sample for all program and control group members.
Collected up to two years prior to random assignment through 6/93.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Survey
Title GAIN in-person registrant survey
Sample Characteristics/Data Collection 3,544 completed interviews with welfare recipients (single-parent families).
1,925 program and 1,619 control group members.
Collected 26 - 37 months after random assignment.
Sites All sites.
Response Rate/Attrition Notes Reported response rate: 80%
Additional Execution Notes No notes reported.
 
Source Administrative data
Title GAIN-26 Form (measuring background characteristics such as race/ethnicity, family composition, education and training, and welfare and employment history);
Case file participation data
Sample Characteristics/Data Collection Computerized records for 9,421 welfare recipients in Alameda and Los Angeles counties.
Sample of all program group members for participation data. Background data (GAIN-26) collected on all experimentals and controls.
Manual records for welfare clients in San Diego, Tulare, Butte, and Riverside counties.
Random subsample of 1,439 program group members.
Collected for 5 years after random assignment.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Survey
Title MDRC self-administered Staff Activities and Attitudes Survey
Sample Characteristics/Data Collection 960 completed questionnaires for GAIN staff.
Collected 1 and 2 years after program initiation.
Sites All sites.
Response Rate/Attrition Notes Reported response rate: 94%-100% across sites.
Additional Execution Notes Survey administered to all GAIN staff in each site.
 
Source Interview
Title In depth, in-person, non-structured interviews
Sample Characteristics/Data Collection Interviews with program case managers and administrators (number in sample not reported).
Data collection schedule not reported.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Administrative data
Title Fiscal and program participation data to estimate costs and benefits of GAIN program
Sample Characteristics/Data Collection Data obtained from tables in U.S. department of Health and Human Services; U.S. Congress, House Committee on Ways and Means; and communications with administrators in the U.S. Department of Labor, California Regional Office, and the California Department of Social Services.
Collected 7/88 until 6/93.
Sites All sites.
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 

Findings Available

Interim Implementation Findings
Final Implementation Findings
Interim Impact Findings
Final Impact Findings
Interim Cost-benefit Findings
Final Cost-benefit Findings

Findings

05/01/93: GAIN Evaluation: Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work Program
Interim Implementation Findings:

"A sizable number of controls used non-GAIN employment-related activities, usually vocational training and post-secondary education"(xxvi).

"The GAIN program substantially increased experimentals' participation in job search and basic education"(xxvi).

Interim Impact Findings:

"GAIN increased the proportion of experimentals who were ever employed in year 3 by 6 percentage points above the control group rate. At the same time, a majority of experimentals as well as controls did not work at all during that year"(xxx).

"Riverside and San Diego produced earnings gains mostly by increasing the rate and duration of employment, while Alameda and Butte produced about half their earnings gains by increasing the amount of money earned per quarter of employment"(xxxi).

"GAIN increased the proportion of experimentals who had more substantial earnings"(xxxiii).

"Several counties increased the proportion of registrants who made a permanent exit from AFDC during the available follow-up period, although this effect was not large"(xxxv).

"For the two basic education subgroups, GAIN produced earnings gains and welfare savings, but not always for both groups in each county"(xxv).

"GAIN produced earnings gains for the heads to two-parent families (AFDC-Us) that were about the same in years 3 as in year 2, and welfare savings that were somewhat lower. Butte had the most impressive earnings impacts, which were large and sustained over time"(x1viii).

"GAIN has a positive impact on AFDC-UP experimentals’ rate of employment in years 3 in three counties (Butte, Los Angeles, and Riverside). However, it did not reduce the proportion on welfare"(x1viii).

Interim Cost-Benefit Findings:

"For all six counties combined, county welfare departments spent an average of $2,899 per experimental within the five years after orientation"(x1).

"GAIN expenditures were heaviest for job search, basic education, and vocational training and post-secondary education"(x1ii).

"The total cost of GAIN varied widely by county, ranging from under $4,000 per experimental in four counties (Butte, Riverside, San Diego and Tulare) to almost $6,000 or more in two counties (Alameda and Los Angeles)"(x1iii).

"The average net cost of all GAIN and non-GAIN services per experimental was $3,422 for all six counties combined, but varied widely across the counties"(x1iii).

"The findings across the six counties point to GAIN’s potential to produce net financial gains for both education subgroups. However, different strategies may involve important trade-offs between the welfare sample and government budget"(x1vii).

"GAIN’s benefit-cost results for AFDC-Ups show a large positive effect from the welfare sample perspective in Butte only, and a modest positive return on the government’s investment in Butte and Riverside only"(1).

 
07/01/96: GAIN Evaluation: Five Year Impacts on Employment, Earnings, and AFDC Receipt
Final Implementation Findings:

"The six counties made different decisions about how much to emphasize quick entry into the labor market versus the longer and more expensive process of building registrants’ human capital through education and training"(xxv).

"All six counties successfully communicated to registrants that the participation requirement was real and would be enforced, although the counties varied in the extent to which they relied on GAIN’s formal penalty process"(xxv).

Final Impact Findings:

"GAIN increased the average earnings of experimentals by 23% during the 5 years after orientation. In all but one county these effects were statistically significant and were large or grew larger over time. GAIN reduced experimentals’ average AFDC payments by 7 percent, a result with a leveling off of GAIN’s impacts on this measure by the third year"(xxvii).

"GAIN’s impacts varied by county. One county (Riverside) has large earnings gains and welfare savings during the 5 years of follow-up, and these effects were achieved across a wide variety of subgroups. Three counties (Alameda, Butte, and San Diego) had more moderate earnings gains and welfare savings. Of the two remaining counties, one (Los Angeles) achieved welfare savings but with little effect on earnings gains, while the other (Tulare) produced earnings gains but with little effect on welfare payments until the last two years of follow-up"(xxvii).

"GAIN produced a small increase in the proportion of experimentals whose combined income from earnings, AFDC, and Food Stamps exceeded the poverty line. GAIN reduced by a small amount (3 percentage points) the proportion of experimentals who were on AFDC during the last quarter of year 5. Even in the best performing county (Riverside), almost one-third of experimentals received AFDC during that last quarter"(xxxiii).

"GAIN produced earnings and welfare savings for a variety of subgroups, including (in some counties) registrants who had received AFDC for more than two years prior to entering the program, showing GAIN’s potential to reach a difficult-to-serve population"(xxxix).

"GAIN’s impacts on single parents with children under the age of six largely paralleled its impacts on single parents whose children were age 6 or older in the three counties that served a substantial number of single parents with younger children"(xxxix).

"In Riverside, GAIN’s already large impacts on earnings and AFDC payments were not improved for registrants who were assigned to case managers with client-staff ratios of 53-to-1 rather than 97-to-1"(x1).

Final Cost-Benefit Findings:

"In five of the six counties, experimentals, on average, were somewhat better off financially as a result of the GAIN program"(x1iv).

"From the standpoint of the government budget, the program more than paid for itself in two of the six counties (Riverside and San Diego). A third county (Butte) led to the government budget ‘breaking even’"(x1iv).

 

Recommendations

GAIN Evaluation: Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work Program (05/01/93)
"The absence of a more consistent, predictable pattern [of impacts] suggests that giving priority for enrollment into GAIN to particular segments of the welfare caseload may not yield effective results across all counties… At the same time, the challenge remains to improve the consistency of GAIN’s effectiveness across a wide variety of subgroups"(1ii).

"In the absence of more convincing evidence of a payoff from maximizing the use of basic education, a more equal emphasis on upfront job search as well as basic education activities in combination with other factors, could be a better way of serving those lacking basic skills"(1iii).

"Also important is whether other combinations of practices can produce results as good as or (by helping more recipients get higher-quality jobs) better than those found in Riverside- e.g., by instituting a strong job development component in a program emphasizing vocational education and training, or delivering a strong employment message in a program that (unlike Riverside) actually produces a greater net increase in (i.e., impact only) the use of vocationally oriented activities"(1v).

"It is therefore important to ask whether GAIN’s effectiveness can be enhanced by other reforms now under debate or already instituted that aim to improve the financial payoff from working"(1v).

 
"To implement a work first program successfully, a large-scale urban program must ensure that all the major partners embrace the work first philosophy and share the same goals and expectations for the program"(5).

"The shift to a work first program can be made independently of major welfare reform or other changes"(5).

"Serving the entire welfare caseload may create significant challenges for a welfare-to-work program"(6).

"Significant investments of staff and resources are required to implement a work first program"(6).

 

Existing Publications

05/01/93 GAIN Evaluation: Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work Program MDRC
07/01/96 GAIN Evaluation: Five Year Impacts on Employment, Earnings, and AFDC Receipt MDRC
01/01/94 GAIN Evaluation: GAIN: Basic Education in a Welfare to Work Program MDRC
04/01/94 GAIN Evaluation: The Impacts of California's GAIN program on Different Ethnic Groups MDRC