http://www.researchforum.org/project_general_305.html

Colorado Works Evaluation

General Information

View a brief abstract of this project.

View a complete, printer-friendly profile of this project.

Evaluator(s) Berkeley Policy Associates
Investigator(s) Vincent Valvano (Berkeley Policy Associates)
Sponsor(s) Office of Colorado State Auditor
Funder(s) Office of Colorado State Auditor
Consultant(s) Not applicable
Subcontractor(s) Institute for Survey Research at Temple University
University of Denver Graduate School of Social Work
 
Domain Income Security/TANF
Status Completed (final report released)
Duration Oct 1998 - Nov 2003
Type Research and/or Program Evaluation
Goal To evaluate Colorado Works program.
Program/Policy Description The evaluation examines the extent to which Colorado Works participants are obtaining employment and achieving self-sufficiency; the impact of Colorado Works on the well-being of children and families of participants; the effectiveness of program services in preparing participants for employment; the availability of adequate child care for program participants; the impact of Colorado Works on other state and local assistance programs; and the economic costs and benefits of the program.

This evaluation will focus on the following six areas: Employment and Self-Sufficiency; Impact on Children and Families; Employment Preparation Services; Child Care; Impact on State and Local Programs, and Costs and Benefits.

Notes No notes reported.
 
Last Updated 01/14/04
Type of Summary Reviewed
External Reviewer(s) Vincent Valvano (Berkeley Policy Associates)
Contact(s) Vincent Valvano (vincent@bpacal.com)
Berkeley Policy Associates
44 Grand Ave.
Suite 500
(T) (510) 465-7884
(F) (510) 465-7885
Publications Department Not applicable (N/A)
(T) N/A
(F) N/A

Populations Studied

Target Population Recipients/participants/clients
Former recipients ("leavers")
Low-income households
Subgroups Analyzed Former recipients ("leavers")
Single parent families
Persons diverted from welfare
Two-parent families
Low-wage workers
Persons with substance abuse problems
Domestic violence victims
Persons with mental/physical health problems
Child-only cases
Children in Foster Care
Sample Size and Unit All participants and leavers of the Colorado Works program, July 1997 — June 2003.

Sites Studied

Colorado

Program Components, Policies, and Activities Evaluated

Employment activities

  • Job skills training
  • Job readiness activities
  • Job search
  • On the job training
  • Employment Activities - misc.

Educational activities

  • English as a Second Language (ESL)
  • GED courses
  • Educational Activities - misc.

Program requirements

  • Work requirement
  • Program Requirements - misc.

Social/Support services

  • Child care
  • Transitional child care
  • Transitional health benefits
  • Transportation
  • Multiple services in single location
  • Life Skills and Opportunities Classes (LSO)
  • Substance abuse/dependence treatment
  • Social/Support Services - misc.
  • Treatment for problems such as domestic violence or mental health
  • Referrals to other service organizations

Administration/Implementation

  • Simplification of program rules and procedures
  • Development of new welfare policies
  • Administration/Implementation - misc.

Time limits

  • Time Limits - misc.

Diversionary activities

  • Diversionary activities - misc.

Post-Program activities

  • Post-Program Activities- misc.
Variation in program components across sites? Yes
Notes on program components Not reported.

Outcomes Assessed

Education

  • High school graduation/GED receipt
  • Education - misc.

Employment

  • Job attainment
  • Job retention
  • Employment - misc.

Income security

  • Earnings
  • Food stamps receipt
  • Income security - misc.
  • Overall income

Attitudes towards work, welfare, and program

  • Attitudes towards work, welfare, and program - misc.

Standard of living

  • Standard of living - misc.

Service utilization

  • Service utilization - misc.

Policy changes

  • Policy changes-misc.

Benefit termination

  • Benefit Termination-misc.

Caseload Dynamics

  • Exit effects
  • Caseload dynamics - misc.
  • Recidivism

Types of Studies

Type Descriptive/Analytical Study
Aim To examine the composition of the Colorado Works caseload over time; utilization of supportive services and programs by Colorado Works participants and former participants.
 
Type Impact Study (Quasi-experiment with non-equivalent control groups)
Aim To examine the extent to which Colorado Works participants are obtaining employment and achieving self-sufficiency and the effectiveness of program services in preparing participants for employment.
 
Type Implementation/Process Study
Aim To examine how counties have designed their Colorado Works programs, their assessment and case management processes, utilization of work activities and service providers, and procedures for providing post-program supportive services.
 
Type Longitudinal/Prospective Study
Aim To examine the employment and earnings patterns of Colorado Works participants and former participants for up to six years.
 

Data Sources

Source Administrative data
Title Monthly data from 12 state administrative program databases
Sample Characteristics/Data Collection Work with entire population of program participants.
Sites Statewide administrative data.
Response Rate/Attrition Notes N/A
Additional Execution Notes New data added on a semi-annual basis.
 
Source Field Research
Title Site visits to 15 Colorado counties
Sample Characteristics/Data Collection Conducted with a variety of stakeholders, including state agency staff, county program staff, service providers, employers and advocacy groups
Sites 15 Colorado counties
Response Rate/Attrition Notes N/A
Additional Execution Notes Semi-structured interviews.
 
Source Focus Group
Title Focus groups of Colorado Works participants
Sample Characteristics/Data Collection Conducted with program participants in years 1 and 2 of evaluation.
Sites 15 Colorado counties
Response Rate/Attrition Notes N/A
Additional Execution Notes Focus groups ranged in size from 5 to 10 participants.
 
Source Survey
Title 1. Colorado Works Early Leavers Survey
2. Colorado Works Participant Survey
Sample Characteristics/Data Collection Year 1 Survey: Random sample of participants who exited Colorado Works between July and September 1997.

Year 2 Survey: Random sample of participants who received Colorado Works cash assistance for at least one month between October and December 1999.

Sites Samples for both surveys were drawn from the statewide population of recipients.
Response Rate/Attrition Notes Year 1 Survey: 306 respondents; response rate of 78%.

Year 2 Survey: 554 respondents; response rate of 79%.

Additional Execution Notes Conducted by telephone and in-person.
 

Findings Available

Interim Implementation Findings
Final Implementation Findings
Interim Impact Findings
Interim Descriptive/Analytical Findings
Final Descriptive/Analytical Findings

Findings

11/01/01: Colorado Works Evaluation: Third Annual Report, Part II
Interim Descriptive/Analytical Findings:

This report examines recent trends in Colorado Works caseload size and expenditures. Among the significant findings resulting from analyses of caseload trends, employment outcomes and post exit services participation are:

  • Colorado Works expenditures increased by 12 percent in State Fiscal Year 2001 despite a slight decline in the overall caseload.

  • A relatively small number of recipients will reach the 60-month time limit on lifetime assistance in the next year.

  • Most former Colorado Works leavers do not work continuously in the first year after exit from the program.

  • About one-half of employed leavers will meet the monetary eligibility requirements for Unemployment Insurance in the event of a qualifying job loss.

  • Slightly more than one-half of eligible current and former Colorado Works recipients received federal and state earned income credits for tax year 2000.
  •  
    08/01/01: Colorado Works Evaluation: Third Annual Report, Part I
    Interim Descriptive/Analytical Findings:

    This report examines two components of the Colorado Works program—diversion and work activity participation—which illustrate the diverse types of assistance Colorado counties can offer low-income families under the Temporary Assistance for Needy Families (TANF) program:

  • Diversion — Most counties in Colorado have implemented diversion assistance programs (State Diversion and/or County Diversion) which offer lump-sum cash grants or vendor payments to families to help them remain self-sufficient and avoid enrolling in time-limited monthly cash assistance.

  • Work Activity Participation — Recipients of ongoing Basic Cash Assistance under Colorado Works are required to participate in a work activity for 30 hours per week within 24 months after the start of assistance or when determined to be job ready. The most utilized work activity is paid employment. TANF rules also allow recipients to participate in limited educational work activities. Almost all Colorado counties offer some basic education activities—typically, GED preparation classes—to their recipients.

  • Among the significant findings resulting from analyses of diversion and work activity participation are:1. State Diversion is an effective alternative to Basic Cash Assistance for many Colorado Works participants. 2. Counties have succeeded in targeting State Diversion to recipients who are most likely to benefit from short-term assistance. 3. County Diversion assistance has been effective in helping families maintain self-sufficiency. 4. Few recipients without a high school diploma or equivalent participate in basic education activities while on Colorado Works.5. Occupational skills training programs are helping recipients improve their earnings.
  •  
    11/01/00: Colorado Works Evaluation: Second Annual Report
    Interim Descriptive/Analytical Findings:
    Interim Implementation Findings:

  • Colorado has been about as successful as other states in moving adult TANF recipients into employment

  • Among those who were employed steadily after Colorado Works exit, earnings increased by 15 percent after a year of steady employment, 23 percent after 18 months, and 37 percent after two years.

  • Former recipients tend to find employment in the relatively low-paying services and retail trade industries.

  • Among adults who exited Colorado Works in the last calendar quarter of 1998, 32 percent were employed continuously over the subsequent year, 38 percent were employed sporadically, and 30 percent were not employed after leaving Colorado Works.

  • Twelve-month re-entry rates for the program are about 18 percent for all Colorado Works leavers. Among those who had 15 months of continuous employment, the re-entry rate was just 1 percent. Those with employment lasting between three and six months after exit had a re-entry rate of up to 40 percent. Those with no employment returned to Colorado Works at a rate of about 19 percent.

  • Services provided to Colorado Works recipients and leavers are not sufficient to ensure that employment barriers are addressed.

  • Local economic conditions, as proxied by the county unemployment rate, indeed affect the Colorado Works caseload. Unemployment rates have been at record lows during much of the three years of Colorado Works operation, averaging between 3 and 4 percent statewide. However, two regions of the State--Pueblo County and the San Luis Valley—have seen unemployment rates as high as 11 percent over the first three years of Colorado Works. These areas have also seen the highest concentration of Colorado Works cases per 1,000 women ages 15 to 44.

  • Thirty-nine percent of Colorado Works recipients who reported job skills barriers were employed, compared to 55 percent of those who did not report such barriers. Among Colorado Works leavers who we surveyed, those with job skills barriers were employed at a rate of 53 percent, compared to an employment rate of 70 percent for those without job skills barriers.

  • Participation in federally approved work activities has declined, and county-defined work activities have become more prevalent.

  • Low educational attainment is associated with lower employment rates.

  • Lack of experience in the labor market is prevalent among the Colorado Works population. Among 1999 Colorado Works Participant Survey respondents, 31 percent reported that a lack of education or training had been a problem in getting or keeping a job.

  • Personal and structural barriers impede participants’ abilities to secure or retain employment. Nearly all (85 percent) Colorado Works recipients report one or more of the following barriers to obtaining or maintaining employment: lack of education or job skills, mental health, physical disability, transportation, housing, domestic violence, substance abuse, and lack of child care. About two-thirds of recipients face two or more barriers, and one-third face four or more barriers. The presence of these barriers is highly correlated with employment.

  • Although many counties have in place appropriate assessment and service provision measures, overall provision of services offered by Colorado Works is not sufficient to meet participants’ needs in most of these barrier categories.

  • Counties have developed services targeted to child-only cases.
  •  
    11/01/02: Colorado Works Evaluation: Fourth Annual Report: Part I: TANF and Colorado Works Expenditures, Caseload Trends, and TANF-funded Child Welfare Activities
    Interim Implementation Findings
    • In SFY 2002, TANF-countable expenditures for Child Welfare program services remained stable at $52.1 million, or 23 percent, of total TANF expenditures.
    • Most counties we visited rely on clients to self-report their involvement in both the Child Welfare and Colorado Works systems.
    • Confidentiality issues were often cited by program staff as a barrier to collaboration between the Colorado Works and Child Welfare programs.
    Interim Descriptive/Analytical Findings
    • The Colorado Works program experienced a 12 percent increase in its caseload in SFY 2002.
    • About one in five recipients exiting Colorado Works returns within a year.
    • After three years of employment, the median earnings of former Colorado Works recipients remain low.
    • There has not been a pronounced increase in the entry rates into the Child Welfare program by children in families that have exited Colorado Works cash assistance.
    • The size of the dual-system caseload in Colorado, defined as the proportion of children receiving cash assistance who also have an open Child Welfare case, is significant.
     
    11/01/02: Colorado Works Evaluation: Fourth Annual Report: Part II: Child Care Assistance to Colorado Works and Other Low Income Families
    Interim Implementation Findings
    • CCCAP funding is expected to decline significantly in the coming year.
    • Counties have begun to implement a number of strategies to address CCCAP funding cutbacks.
    Interim Descriptive/Analytical Findings
    • Only one-quarter of all employed Colorado Works leavers transition to subsidized child care after exiting.
     
    11/01/03: Colorado Works Evaluation: Fifth Annual Report
    Final Implementation Findings:
    • Between July 1997 and June 2003 the Colorado Works Basic Cash Assistance (BCA) caseload decreased by about 46 percent, from 27,898 to 15,159.
    • Total Colorado Works Program spending declined during State Fiscal Year (SFY) 2003 for the first time since the start of the program.
    • Overall, less than 1 percent (on average, 74 cases) of the average monthly adult headed caseload of 9,051 in SFY 2003 received cash assistance due to an extension to the 60-month time limit.
    • Re-entry rates among Colorado Works leavers have been increasing for several years.
    • The recent economic slowdown appears to have had a significant impact on the employment rate of recent Colorado Works leavers.
    Final Descriptive/Analytical Findings
    • Colorado Works participants often have difficulty achieving self-sufficiency due to substance abuse problems or mental illness.
    • The counties have undertaken a variety of endeavors to assist Colorado Works participants with mental illness or substance abuse problems.
    • The Colorado Department of Human Services met the legislative mandate to designate a nationally recognized screening instrument to identify substance abuse and mental illness among Colorado Works participants and to provide training to county staff on the use of this screening.
    • Cost data on providing mental health and substance abuse screening, referral, and assessment are very limited, precluding a thorough analysis of the cost effectiveness of these services.
    • Rates of both tax filing and Earned Income Tax Credit (EITC) participation have improved compared with the 1999 rates reported in BPA’s Second Annual Report.
    • The issue of transportation as a barrier to self-sufficiency has not changed since BPA first discussed the problem in the 2000 Second Annual Report.
     

    Recommendations

    Colorado Works Evaluation: Third Annual Report, Part II (11/01/01)
  • The Department of Human Services should continue to work with counties to conduct in-depth assessments of individuals nearing the time limit and ensure that the assessments are conducted in a timely and consistent fashion. These assessments should identify factors that constrain the ability of recipients to become self-sufficient and indicate specific services beyond cash assistance that would benefit recipients. The assessments should also provide the Department with adequate information to allow it to determine if a benefit extension is warranted.

  • DHHS should work with counties to identify and implement practices that can facilitate increased participation in the Food Stamp program among low-income households, including those of former Colorado Works recipients.

  • The Department of Health Care Policy and Financing, in consultation with the Department of Human Services, should work with counties to assess whether additional outreach about Transitional Medicaid assistance should be targeted to Colorado Works leavers. If determined to be necessary, the agencies should develop additional outreach programs to increase awareness of post-program Medicaid assistance among under-enrolled Colorado Works leavers.
  •  
    Colorado Works Evaluation: Third Annual Report, Part I (08/01/01)
  • The Department of Human Services should work with counties to develop policies and procedures to refer State Diversion recipients to appropriate job development and retention programs, including those offered by county Departments of Human Services, Workforce Development Boards, and local community-based organizations.

  • DHHS and Health Care Policy and Financing should work with counties to review their policies and practices for enrolling State Diversion recipients into Medicaid and revise them, as needed, to ensure that all new State Diversion applicants are also considered for Medicaid eligibility.

  • DHHS should work with counties to develop policies and procedures for referring up-front and post-program County Diversion recipients to appropriate job development and retention programs, including those offered by county Departments of Human Services, Workforce Development Boards, and local community-based organizations.

  • DHHS should work with counties to: a) improve their assessment processes to identify Colorado Works recipients who could benefit from placement in GED preparation classes or other basic education activities; and b) to improve case managers’ utilization of assessment information to assist recipients with work activity choices.

  • DHHS should work with counties to determine whether additional Colorado Works recipients can be appropriately referred to and placed in occupational skills training programs (vocational educational training work activities).
  •  
    Colorado Works Evaluation: Fifth Annual Report (11/01/03)
    • The Department should work with the county departments of human and social services to identify training opportunities for appropriate Colorado Works practitioners and outside service providers who have contact with Colorado Works participants in recognizing and identifying mental health and substance abuse problems.
    • The Department should continue to work with the county departments of human and social services to encourage regional and statewide collaboration to leverage resources and coordinate services.
    • The Department should provide support to the county departments of human and social services in their efforts to assist Colorado Works participants with the most severe disabilities to apply for SSI, by helping to establish working relationships with SSA and facilitating these relationships as needed.
    • The Department, in collaboration with the county departments of human and social services, should document the effectiveness of serving Colorado Works participants with co-occurring disorders using an integrated service approach. The Department should provide information about the effectiveness of these programs to the counties.
    • The Department should work with the county departments of social and human services to modify data collection and reporting requirements to ensure data (including cost data) related to screening, assessment, and referral for mental health, substance abuse, or dual diagnosis services to Colorado Works participants are routinely compiled and reported.
    • The Department should enhance its efforts to maximize Earned Income Tax Credit participation among Colorado Works participants and other low-income individuals by evaluating the effectiveness of various outreach methods and by continuing to partner with the counties and other entities to encourage and assist Colorado Works participants and others in filing income tax returns and claiming the EITC.
    • The Department should provide training to all Colorado Works staff, including case managers, on the importance of tax filing for low income populations, including information on claiming the EITC. Sessions on tax filing and the EITC should be included in the Department's annual Professional Development Academy.
     
    Colorado Works Evaluation: Fourth Annual Report: Part I: TANF and Colorado Works Expenditures, Caseload Trends, and TANF-funded Child Welfare Activities (11/01/02)
    • Ensure procedures are in place to allow county program staff to readily identify dual-system cases. In particular, the Department should:
      1. generate monthly status reports identifying children with open Child Welfare and Colorado Works cases (including new and ongoing cases);
      2. distribute status reports on a monthly basis to county program managers in both the Child Welfare and Colorado Works programs.
    • Ensure that the Child Welfare and Colorado Works programs share client information appropriately for the purposes of identifying client needs, developing coordinated service plans, coordinating case management activities, and facilitating joint services discharge or transfer processes. To eliminate confidentiality obstacles, the Department should issue rules that mandate the use of its confidentiality policy and forms as outlined in the CDHS reference manual entitled: Sharing Customer Information through a Common Consent Procedure.
    • Improve coordination between the Child Welfare and Colorado Works programs and eliminate conflicting client requirements by:
      1. Requiring counties to specify in their County Plans formal procedures to ensure that Colorado Works case managers and Child Welfare case workers effectively collaborate in case management and service planning for dual-system clients, including development and modification of Colorado Works Individual Responsibility Contracts for dual-system clients;
      2. Requiring counties to specify in their County Plans how they will train new and existing Child Welfare and Colorado Works staff regarding the basic eligibility criteria, services available, and referral procedures for both programs;
      3. Disseminating to County program managers in both written form and in staff trainings current best practices for Child Welfare and Colorado Works program collaboration, including those identified in this report.
     
    Colorado Works Evaluation: Fourth Annual Report: Part II: Child Care Assistance to Colorado Works and Other Low Income Families (11/01/02)
    • Ensure counties provide child care assistance consistent with legislative intent and departmental policy by addressing prioritization afforded to transitioning Colorado Works recipients, families with income below 130 percent of the federal poverty level, participants in education and training programs, and other eligible low-income families. This should be achieved by:
      1. promulgating rules that clarify which participant groups should receive priority for child care assistance
      2. proposing statutory changes as needed
    • Consider adjusting the parental fee schedule for child care assistance to require all families with more than one child in care to pay an additional fee that would range from $5 to $40, depending on family income.
    • Ensure that counties do not require an LICC application from transitioning Colorado Works recipients by clarifying eligibility rules for LICC (CCR 3.904.1, A3). This rule should be amended to state that counties shall not require applications for transitioning Colorado Works recipients.
    • Streamline the transition from Colorado Works to LICC by changing current rules regarding CCCAP and Colorado Works immunization record requirements. The rule change should reduce transitioning parents’ administrative responsibilities, while complying with federal immunization regulations. This should be achieved by:
      1. allowing school enrollment to serve as sufficient proof of current immunization for school-age children receiving LICC assistance;
      2. requiring counties to collect immunization records for all children under school age at entry to Colorado Works;
      3. requiring that immunization records are current, as defined by Department of Public Health and Environment guidelines, and eliminating the requirement that records be current within 60 days for those transitioning to LICC from Colorado Works.
     

    Existing Publications

    03/01/99 Colorado Works Evaluation: Interim Report BPA
    11/01/99 Colorado Works Evaluation: First Annual Report BPA
    11/01/00 Colorado Works Evaluation: Second Annual Report BPA
    08/01/01 Colorado Works Evaluation: Third Annual Report, Part I BPA
    11/01/01 Colorado Works Evaluation: Third Annual Report, Part II BPA
    11/01/03 Colorado Works Evaluation: Fifth Annual Report BPA
    11/01/02 Colorado Works Evaluation: Fourth Annual Report: Part I: TANF and Colorado Works Expenditures, Caseload Trends, and TANF-funded Child Welfare Activities BPA
    11/01/02 Colorado Works Evaluation: Fourth Annual Report: Part II: Child Care Assistance to Colorado Works and Other Low Income Families BPA