|
Ohio Works First Evaluation
General Information
View a brief abstract of this project.
View a complete, printer-friendly profile of this project.
| Evaluator(s) |
Macro International, Inc.
|
| Investigator(s) |
Richard Mantovani
(Macro International, Inc.)
|
| Sponsor(s) |
Ohio Department of Job and Family Services
|
| Funder(s) |
US Department of Health and Human Services, Administration for Children and Families
Ohio Department of Job and Family Services
|
| |
| Domain |
Income Security/TANF
|
| Status |
Completed (final report released)
|
| Duration |
Oct 1998 - Jun 2004
|
| Type |
Research and/or Program Evaluation
|
| Goal |
To provide information on the success of Ohio Works First in helping recipients achieve personal responsibility and self-sufficiency.
To document the implementation of Ohio Works First and to determine the similarities and differences across counties as these may explain variations in recipient outcomes.
|
| Program/Policy Description |
The Ohio Works First program is Ohios implementation of the TANF program that provides temporary assistance to individuals in need. Key features of the Ohio Works First program include a 36-month time limit on eligibility, an earned income disregard that ignores a portion of employment income in the calculation of benefits, a self-sufficiency contract and three tiered sanction process. In addition, child care, Medicaid, and Food Stamp benefits are extended to individuals leaving Ohio Works First to help them transition off cash assistance. Able-bodied individuals are required to work for 30 hours per week. In addition to Ohio Works First, Ohio implemented the Prevention, Retention and Contingency (PRC) program, which provides short term cash assistance and services to OWF clients and other low income individuals to help them obtain and maintain employment or to meet short term needs that were formerly covered by the Emergency Assistance Program in order to help individuals stay off OWF cash assistance.
|
| Notes |
Reports also available on The Ohio Department of Job and Family Services website: http/jfs.ohio.gov
|
| |
| Last Updated |
10/06/04
|
| Type of Summary |
Reviewed
|
| External Reviewer(s) |
Jacqualyn Martin
(Ohio Department of Job and Family Services)
|
| Contact(s) |
Jon Allen (allenj@odjfs.state.oh.us)
Ohio Department of Job and Family Services
30 E Broad Street
(T) (614) 466 - 6650
(F) (614) 466 - 0292
|
| Submitter(s) |
Research Forum Staff (info@researchforum.org)
National Center for Children In Poverty
215 West 125th St, 3rd Fl
(T) (646)284-9600
(F) not reported
|
Populations Studied
| Target Population |
Recipients/participants/clients
Former recipients ("leavers")
|
| Subgroups Analyzed |
Caseworkers/managers/administrators
|
| Sample Size and Unit |
N=1,052 random sample of former OWF participants who left between October 1997 and March 1998 and had been off OWF for at least 12 consecutive months (for the Legislative Outcome Study.)
N=1,951 randomly selected from a stratified statewide sample of Ohio Works First recipients who received assistance in both January and June 2000(for the longitudinal survey of participants.)
N=13,989 randomly selected Ohio Works First recipients statewide who received assistance in both January and June 2000 (for the longitudinal administrative study).
|
Sites Studied
Ohio
Program Components, Policies, and Activities Evaluated
Employment activities
- Job skills training
- Job readiness activities
- Job search
Educational activities
- High school completion
- Post-secondary education
Financial incentives
Financial disincentives/Sanctions
- Strengthened JOBS sanctions
Program requirements
- Work requirement
- Broadened JOBS participation requirement
Social/Support services
- Child care
- Transitional child care
- Health benefits
- Transitional health benefits
- Transportation
Administration/Implementation
- Changes in welfare office environment/culture
- Program enforcement of sanctions
Time limits
Food stamps
Diversionary activities
- Diversionary activities - misc.
| Variation in program components across sites? |
No
|
| Notes on program components |
Administration/Implementation: Assessment of the correlation between administration and implementation of Ohio Works First and socioeconomic outcomes of recipients. Administration of time limits.
Diversion Activities: Assessment of diversion strategies used by OWF offices.
Educational Activities: Assessment of the correlation between high school education and post secondary education and outcomes of recipients.
Employment Activities: Assessment of the correlation of job training, readiness programs, unsubsidized employment, and work experience with outcomes of recipients. Researchers did not study in-depth the activity itself but changes in participation in the various work and education activities.
Food Stamps: Use of food stamps after leaving OWF.
Social/Support Services: Childcare and medical care benefits for those leaving cash assistance.
Time Limits: Strategies for OWF participants who are approaching their 36-month time limit on eligibility.
|
Outcomes Assessed
Employment
- Job attainment
- Job retention
Income security
- Earnings
- Food stamps receipt
- Medicaid receipt
- Welfare receipt
- Overall income
Sanctions
Family and relationship outcomes
- Family and relationship outcomes - misc.
Caseload Dynamics
Types of Studies
| Type |
Descriptive/Analytical Study
|
| Aim |
Legislative Outcome Study
To examine the demographic characteristics, use of various services, and employment outcomes of individuals who left OWF during the first six months after it was implemented and remained off for at least 12 months.
|
| |
| Type |
Implementation/Process Study
|
| Aim |
To document the implementation of Ohio Works First and to determine the similarities and differences across counties as these may explain variations in caseload dynamics and recipient outcomes.
|
| |
| Type |
Longitudinal/Prospective Study
|
| Aim |
Administrative and Survey Studies
To evaluate the correlations of individual, assistant group, and county characteristics, and various Ohio Works First program components with client self-sufficiency and employment outcomes over a period of five years.
|
| |
Data Sources
| Source |
Survey
|
| Title |
Legislative Outcome Study Telephone Survey
(Telephone survey with in-person follow-up of non-respondents)
|
| Sample Characteristics/Data Collection |
N= 1,052 individuals who left OWF between October 1997 and March 1998 and remained off for at least 12 consecutive months.
|
| Sites |
Ohio
|
| Response Rate/Attrition Notes |
60%
|
| Additional Execution Notes |
The survey was conducted between October 1999 and April 2000.
|
| |
| Source |
Administrative data
|
| Title |
House Resolution 3734 Data file (HR3734) Wage Records Database
|
| Sample Characteristics/Data Collection |
N= 1,951 OWF recipients who received assistance in both January and June 2000.
N=13,989 Ohio Works First recipients statewide who received assistance in both January and June 2000 (for the longitudinal administrative study).
|
| Sites |
Ohio
|
| Additional Execution Notes |
For the longitudinal administrative study, 36,233 work-required adults who received assistance in both January and June 2000 were identified statewide. From this population, 13,989 were selected for the five-year longitudinal adminstrative study with oversampling in the 10 counties included in the implementation study. Administrative data were used to supply demographics, work activity participation, eligibility, benefit amounts, and other program participation measures. Wage Record Data were merged with administrative data to provide employment and earnings information.
|
| |
| Source |
Survey
|
| Title |
Longitudinal Survey of OWF Participants
|
| Sample Characteristics/Data Collection |
For the longitudinal survey of participants, a stratified sample of 1,951 individuals was randomly selected from the 13,989 individuals selected for the longitudinal administrative data sample.
|
| Sites |
Ohio
|
| Response Rate/Attrition Notes |
First wave: 1,138 completed Wave 1 = 83.5% Second wave: 953 of the 1,138 completed Wave 2 = 83.7% Third wave: 685 of the 953 who completed Waves 1 and 2 completed Wave 3 = 71.9%
|
| Additional Execution Notes |
To qualify for the interviews, respondents had to be either on OWF or else off of OWF for no more than three months at the time of the first interview. The survey was conducted in three waves over the course of approximately two years.
- First wave conducted between August 2000 and April 2001.
- Second wave conducted between May 2001 and February 2002.
- Third wave conducted between May 2002 and February 2003.
The results were weighted to be representative of the statewide caseload in Ohio.
|
| |
| Source |
Interview
|
| Title |
On-site interviews with County Departments of Job and Family Services (CDJFS) staffs.
|
| Sample Characteristics/Data Collection |
Using a semi-structured guide, in-person interviews were conducted with CDJFS staffs in ten selected Ohio counties.
|
| Sites |
Four metropolitan counties: (Franklin, Hamilton, Lucas, and Stark), Ohio;Six non-metropolitan counties: (Belmont, Clark, Gallia, Greene, Licking and Scioto), Ohio.
|
| Response Rate/Attrition Notes |
Not applicable.
|
| Additional Execution Notes |
The staff interviewed varied from county to county but typically included the agency director, a fiscal officer, OWF caseworkers, OWF supervisors, and other key staff as designated by the agency director. The sites were visited between Fall 2000 and Spring 2001 and again between Fall 2002 and Winter 2003.
|
| |
| Source |
Survey
|
| Title |
Self-administered survey of CDJFS OWF caseworkers and supervisors
|
| Sample Characteristics/Data Collection |
OWF caseworkers and supervisors in ten selected counties in Ohio.
|
| Sites |
Four metropolitan counties: (Franklin, Hamilton, Lucas, and Stark), Ohio;Six non-metropolitan counties: (Belmont, Clark, Gallia, Greene, Licking and Scioto), Ohio.
|
| Response Rate/Attrition Notes |
Four metropolitan counties: (Franklin, Hamilton, Lucas, and Stark), Ohio;Six non-metropolitan counties: (Belmont, Clark, Gallia, Greene, Licking and Scioto), Ohio.
|
| Additional Execution Notes |
Surveys were distributed to staff with instructions to complete them and mail them to the designated Macro project manager. Surveys were distributed at the time of the on-site interviews.
|
| |
Findings Available
Interim Implementation Findings
Final Implementation Findings
Interim Descriptive/Analytical Findings
Final Descriptive/Analytical Findings
Findings
|
08/01/00:
The Ohio Works First Evaluation: Legislative Outcome Study
|
Outcomes Findings of OWF Leavers
- 90.5% of respondents held at least one job after leaving OWF.
- 66.7% of respondents were employed at the time they were surveyed.
- 61.2% of respondents left OWF because they got a job or increased their earnings in a job they had while on OWF.
- 71.7% felt that their children were better off since leaving OWF.
- 56.5% of the adults in the surveyed households had medical insurance.
- Nearly half of these (45.6%) had employer coverage.
- About one-third of these (34.4%) had Medicaid coverage.
- 82.8% of the youngest children in the surveyed households had medical insurance.
- 59.6% of these children were covered by Medicaid.
- 40.4% of respondents with children under the age of 12 had used child care.
- 75.5% of respondents with children under the age of 12 reported no child care problems.
- Most respondents (91.9%) had no contact with Childrens Protective Services after leaving OWF.
- 22.5 % of respondents knew about the Prevention, Retention, and Contingency (PRC) program.
- 26.1% of these applied for PRC assistance and all received it.
- All PRC users said the program helped them stay off OWF.
- All but one PRC user said they were satisfied with the program.
- Of those that were employed at the time of the survey (adults in 66.7% of all surveyed households):
- 69.1% had full time jobs.
- 57.5% had been working in the same job for more than 12 months.
- 58.7% traveled less than ten miles to work.
- On average, they worked 38.4 hours per week and earned $8.65 per hour and $1,410 per month.
- 27.3% received child support.
- 24.9% received food stamps.
- Of those that were not employed at the time of the survey (adults in 33.3% of all surveyed households):
- 65.2% held at least one job after leaving OWF.
- Only 33.3% were currently looking for work.
- The three primary reasons for not looking for work were health, childcare, and don't want or need to work.
|
| |
|
12/31/02:
The Ohio Works First Longitudinal Survey: Baseline Description of the Surveyed Population
|
Baseline Descriptive and Outcomes Findings- More than 90% of the population is female.
- The population consisted of mostly younger adults. 73% were less than 36.
- 55% were African American and 40% were Caucasian.
- 65% of the population was single.
- Most respondents had one or two children in the household.
- 62% of the population had at least one child under the age of 6.
- In most cases, (62%) the youngest child was under age 6.
- A third of the population was employed, averaging 31 hours per week and $6.23/hour.
- A third of those employed had paid vacation and holidays; 21% had paid sick leave.
- Unearned income regardless of OWF status averaged approximately $551/month, not including OWF cash assistance benefits for those still on OWF.
- 40% were in public or subsidized housing.
- 61% were satisfied with their transportation arrangements.
- 90% used Medicaid to fund their health care and 93% used Medicaid to fund their childrens health care.
|
| |
|
12/31/02:
The Ohio Works First Longitudinal Survey: Interim Site Visit Reports from Ten Counties
|
Interim Implementation Findings- Most county agencies gave special treatment to clients approaching time limits on their OWF eligibility with some agencies intensifying client contact as early as 18 months in advance of client time limits. Special services were targeted at helping clients obtain job skills, conduct job searches, and retain jobs.
- Though OWF caseload levels have declined in most of the study counties, caseworkers felt that their work had actually increased because of the frequency and intensity of work they do with the remaining clients. Caseworkers reported that hard-to-serve clients requiring extra time and attention were on the rise as a proportion of all cases.
- In most of the study counties, eligibility determination and work activities functions were handled by different employees. This arrangement allows staff to acquire specialized knowledge but presents communication and case coordination challenges for staff especially in offices that house these functions in separate facilities or in separate parts of the same facility. All counties in the study employed contractors to a varying degree to provide OWF and PRC services to clients. In some instances, contractors provided highly specialized testing and assessment services at the request of agency caseworkers; in others, they performed all of the functions of a work activities caseworker. Contractors often saw clients off-site, away from the CDJFS office, but some worked in agency quarters and were nearly indistinguishable from agency staff.
|
| |
|
06/30/03:
The Ohio Works First Evaluation: Findings from a Longitudinal Survey of Participants
|
Interim Outcomes Findings- 61% of those off OWF were employed at Wave 2.
- The percentage of individuals earning more than $1,500 a month increased from 7 percent at wave 1 to 19 percent at Wave 2.
- Seventy one percent of those receiving OWF cash assistance in June 2000 were not on OWF when the second interview occurred.
- Sixty five percent of individuals identified as having low self-sufficiency at Wave 1 increased their level of self-sufficiency by the time of the second interview. The percentage of individuals with a high level of self-sufficiency increased from 5 percent at Wave 1 to 15 percent at Wave 2.
- Outcomes did not vary with differences in county size and unemployment rates.
- The presence of a spouse or partner in the household reduced the odds that an individual would be employed. Otherwise, outcomes did not vary with the size and composition of OWF assistance groups.
- The longer an individual had been on OWF, the smaller his or her wage gains, and the less likely he or she was to leave OWF.
- Individuals who had completed high school were more likely to be employed and to be more self-sufficient than those who had not.
- The only OWF work activities associated with positive outcomes were vocational and post secondary education and unsubsidized employment. More specifically:
- Individuals enrolled in vocational education programs at the time of their wave 1 interview tended to show greater gains in self-sufficiency at wave 2 than others.
- Those enrolled in postsecondary education programs at wave 1 were more likely to be employed and to have increased their level of self-sufficiency at wave 2 than others.
- Individuals who were engaged in unsubsidized employment at the time of their first interview were more likely to be employed, off OWF, and more self sufficient at the time of their second interview than those who were not working when first interviewed.
|
| |
|
12/17/03:
The Ohio Works First Evaluation: Final Report
|
|
Final Implementation and Outcomes Findings
- OWF had an effect on the administration of public assistance in three ways. First, it promoted some unique responses to processing caseloads. Of particular importance was how work development activities and diversion approaches were integrated into agency processes. In general, there were three models that existed in 2002-2003:
- An approach in which a single caseworker is responsible for all aspects of an individuals case. Two counties adopted this approach.
- An approach that functionally separates eligibility and workforce development and assigns these activities to different workers. Seven counties adopted this approach.
- An eligibility-focused approach, in which the agency outsources all workforce development work. One county, Hamilton, adopted this approach.
- At various times during the period of this study, counties have adopted different approaches and variations of these approaches, indicating that agencies were constantly rethinking their approach in order to provide better services.
- Second, practices varied from counties that took a strict approach to transitioning individuals off welfare to counties that seemed more lenient. The former have experienced dramatic declines in caseloads, while the latter have experienced a higher rate of recidivism. Additionally, county success in placing individuals into unsubsidized employment (and failing that, their placement of individuals in various work activities) is important. The relationship between unsubsidized employment and work experience is critical, since the latter substitutes for unsubsidized employment when it is difficult to achieve.
- Third, information on OWF outcomes indicates that OWF participation and cash assistance amounts have declined drastically. However, individuals receiving cash assistance have a 16-percent chance of being on OWF after a two-year period, and they are more likely to be work experience participants. Income rises, but not uniformly across the population. Individuals with small earnings when they are receiving welfare see larger earnings during the following two years, while growth among those with larger incomes is small. Family and child well-being, however, remains constant.
- Although changes in earnings and assistance levels are generally similar to those in the rest of the state, among the 10 counties, we find that Franklin and Licking have large increases in earnings as well as smaller decreases in cash assistance. We find the opposite pattern in Scioto. It should be pointed out that Scioto is one of the counties we have identified as aggressively transitioning individuals off OWF, while Franklin is one of the counties that actually saw increases in caseloads. Overall, however, employment income outcomes were determined by individual characteristics more than by county-level characteristics.
|
| |
Recommendations
Existing Publications
|