CEEP was one of seven RSA-sponsored projects funded to demonstrate different approaches to implementing informed choice. The other six Choice grantees were Arkansas Rehabilitation Services (AR); Vermont Division of Vocational Rehabilitation (VT); Washington Division of Vocational Rehabilitation (WA); Southwest Business, Industry and Rehabilitation Association (SWBIRA); The Development Team, Incorporated (TDTI); and United Cerebral Palsy Associations, Inc. (UCPA).
InfoUse conducted the federally-funded evaluation of the seven projects. In this section, we present and discuss data from InfoUse’s final report (Stoddard, Hanson, and Temkin, 1999a) in order to illustrate CEEP’s model in the context of the other projects. We show data on participants’ reported choice and satisfaction, patterns of purchased services, and service costs.
To assess participants’ perceptions of the degree of choice and satisfaction experienced in each project, InfoUse conducted a telephone survey of a random sample of Choice project consumers. Table 9 indicates that 71.8% of all demonstration participants surveyed in the InfoUse study said they had had “total choice” (40.4%) or “a lot of choice” (31.4%) in their projects. In comparison, 86.8% CEEP participants reported “total choice" (40.9%) or “a lot of choice” (45.5%), indicating a level of satisfaction somewhat higher than the level across all the Choice projects.
Table 9: Participants' Report of Degree of Overall Choice
|
|
BCIL |
AR |
VT |
WA |
SWBIR |
TDTI |
UCPA |
Totals |
|
N
= |
261 |
132 |
305 |
226 |
302 |
109 |
109 |
1,444 |
|
No choice |
0.0 |
10.0 |
0.0 |
8.3 |
22.2 |
0.0 |
4.5 |
7.2 |
|
Not much choice |
4.5 |
16.7 |
0.0 |
8.3 |
5.6 |
11.1 |
18.2 |
7.0 |
|
Some choice |
9.1 |
23.3 |
10.0 |
0.0 |
27.8 |
16.7 |
13.6 |
14.0 |
|
A lot of choice |
45.5 |
33.3 |
20.0 |
50.0 |
11.1 |
66.7 |
9.1 |
31.4 |
|
Total choice |
40.9 |
16.7 |
70.0 |
33.3 |
33.3 |
5.6 |
54.5 |
40.4 |
|
Total |
100.0 |
100.0 |
100.0 |
99.9 |
100.0 |
100.1 |
99.9 |
100.0 |
Note: All response values are expressed in percentages. Percentages may not sum to 100% due to rounding.
Table 10 shows data on the extent to which the Choice projects met participants’ expectations. Across all of the Choice participants surveyed, 73.9% reported that the projects met or exceeded their expectations, compared to 81.8% at BCIL CEEP. Table 10 also indicates that about four of five (81.4%) of all project participants reported that they would recommend the project to a friend, and 51.5% reported doing so. For CEEP participants, 100% would recommend the project to a friend, with 72.2% reporting that they actually did so.
Table 10: Participant Overall Satisfaction
|
|
BCIL |
AR |
VT |
WA |
SWBIR |
TDTI |
UCPA |
Total |
|
N
= |
261 |
132 |
305 |
226 |
302 |
109 |
109 |
1,444 |
|
Project
exceeded expectations |
40.9 |
16.7 |
36.4 |
66.7 |
22.2 |
27.8 |
21.7 |
35.4 |
|
Project
lived up to expectations |
40.9 |
40.0 |
54.5 |
16.7 |
38.9 |
33.3 |
34.8 |
38.5 |
|
Project
fell below expectations |
18.2 |
43.3 |
9.1 |
16.7 |
38.9 |
39.8 |
43.5 |
26.2 |
|
Question
not answered |
0.0 |
0.0 |
0.0 |
0.0 |
0.0 |
0.0 |
0.0 |
0.0 |
|
Total |
100 |
100 |
100 |
100 |
100 |
100 |
100 |
100.1 |
|
Participant
would recommend project to a friend |
100 |
90.0 |
90.9 |
83.3 |
56.3 |
77.8 |
69.6 |
81.4 |
|
Participant
has recommended project to a friend |
72.2 |
56.7 |
54.5 |
58.3 |
27.8 |
55.6 |
34.8 |
51.5 |
Note: All response values are expressed in percentages. Percentages may not sum to 100% due to rounding.
Tables 11 and 12 are derived from the purchased services dataset developed in InfoUse’s case review process. For each separate service purchase during a specific 18 month study period, InfoUse identified the item purchased, the type of source of purchase, and the dollar cost of the purchase. InfoUse used data from actual financial transaction records derived from case file data (Stoddard, Hanson & Temkin, 1999a).
Each of the seven Choice projects implemented unique models for helping participants choose goods and services. Thus, the services available to a Choice participant, and consequently the participant’s selection of services in each project, were largely determined by the particular service model used. In some projects, certain services were mandatory or integral to the service model used. In the CEEP model, participants were not required to purchase specific types of services. Instead, they were encouraged to shop in the open market where purchases are concentrated in retail stores and computer stores, as well as educational programs.
Table 11 shows how purchase service dollars were used by participants enrolled in the Choice projects during our sample period. Percentages over 5% are highlighted in bold for ease of reading. For the demonstration program as a whole, 14% of purchase service dollars was spent on business or vocational training, another 8.1% on college or university training, almost 11% on assessment, 9.4% on job placement services, 9.3% on computers and peripherals, and 8.5% on job-finding services. In CEEP, patterns of participant purchase of services varied from this overall pattern. CEEP’s participants emphasized business/vocation (28%) and college/university training (18.9%) and obtaining computers for work and self-employment (25.3%).
Table 12 shows where participants made purchases as a percentage of total purchases. Each column identifies the types of providers where program-purchased service dollars were spent. Overall, Table 12 shows about 30% of purchases going to employment service organizations, and another 11% going to Community Rehabilitation Facilities. Educational institution expenditures accounted for nearly 16%. These are more traditional purchasing sources for vocational rehabilitation. The second highest source is commercial merchandise retailers (17%). In contrast, the majority of CEEP participant purchases were from either educational institutions (44.7%) or from retailers (21.5%).
Table 11: What the Purchased Service Dollars Buy/Percent of Service Funds by Program by Item
|
SERVICES |
BCIL |
AR |
VT |
WA |
SWBIR |
TDTI |
UCPA |
Totals |
|
weights |
7.91 |
3.88 |
10.17 |
7.53 |
9.74 |
3.52 |
3.41 |
|
|
N
= |
261 |
132 |
305 |
226 |
302 |
109 |
109 |
1,444 |
|
n
= |
33 |
34 |
30 |
30 |
31 |
31 |
32 |
221 |
|
Business/Vocational
Training |
28.0 |
30.3 |
9.0 |
4.0 |
*
|
*
|
0.5 |
14.0 |
|
Assessment |
0.2 |
7.5 |
1.6 |
2.8 |
38.6 |
0.1 |
13.5 |
10.8 |
|
Job
Placement |
*
|
*
|
2.6 |
*
|
19.3 |
*
|
50.2 |
9.4 |
|
Computers
and Peripherals |
25.3 |
6.0 |
2.7 |
5.5 |
*
|
38.9 |
*
|
9.3 |
|
Job-Finding
Services |
0.4 |
*
|
0.3 |
14.8 |
18.7 |
0.5 |
25.2 |
8.5 |
|
College/University
Training |
18.9 |
13.4 |
8.3 |
3.3 |
*
|
1.8 |
*
|
8.1 |
|
Self-Employment
Inventory/Set-up |
13.1 |
3.7 |
11.0 |
7.3 |
*
|
3.7 |
*
|
5.6 |
|
Other
Services (excluding any of above) |
0.4 |
6.6 |
0.5 |
22.4 |
2.9 |
2.9 |
0.0 |
5.5 |
|
Maintenance |
0.8 |
17.5 |
8.9 |
0.1 |
*
|
*
|
*
|
4.9 |
|
Assistive
Technology Devices |
*
|
7.3 |
21.3 |
0.6 |
5.2 |
7.8 |
*
|
4.9 |
|
Transportation |
4.9 |
2.8 |
2.0 |
2.9 |
1.3 |
1.0 |
1.5 |
2.7 |
|
Grooming/Clothes |
2.1 |
0.6 |
2.5 |
6.8 |
2.3 |
4.9 |
*
|
2.4 |
|
Miscellaneous
Training |
4.6 |
0.4 |
6.4 |
2.2 |
0.7 |
0.9 |
*
|
2.0 |
|
Motor
Vehicle Purchase |
*
|
*
|
3.7 |
2.2 |
*
|
20.0 |
3.2 |
1.9 |
|
Restoration |
0.7 |
0.0 |
4.3 |
3.2 |
0.2 |
4.2 |
1.0 |
1.3 |
|
Counseling
and Guidance |
*
|
*
|
3.5 |
0.7 |
*
|
4.0 |
4.1 |
1.1 |
|
Motor
Vehicle Repair |
*
|
*
|
8.6 |
0.6 |
*
|
4.0 |
*
|
1.0 |
|
On-the-job
Training |
*
|
1.3 |
*
|
3.5 |
*
|
*
|
0.3 |
0.8 |
|
Assistive
Technology Services |
*
|
1.9 |
*
|
0.5 |
*
|
0.6 |
*
|
0.5 |
|
Adjustment
Training |
*
|
0.6 |
2.9 |
0.6 |
*
|
*
|
*
|
0.5 |
|
Office-Related
Equipment and Supplies |
0.1 |
0.1 |
*
|
1.1 |
*
|
4.6 |
*
|
0.4 |
|
Personal
Assistance Services |
*
|
*
|
*
|
*
|
*
|
*
|
*
|
*
|
|
Case
Management |
*
|
*
|
*
|
*
|
*
|
*
|
*
|
*
|
|
Cannot
Determine |
0.4 |
*
|
*
|
14.9 |
10.7 |
0.2 |
0.3 |
4.2 |
|
TOTAL |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
Note: Unless otherwise indicated, numbers are percentages. * = zero value 0.0 = less than .05 value
Table 12: Where Purchases are Made/Percent of Total Purchases by Project by Source of Purchase
|
SOURCES |
BCIL |
AR |
VT |
WA |
SWBIRA |
TDTI |
UCPA |
Totals |
|
weights |
7.91 |
3.88 |
10.17 |
7.53 |
9.74 |
3.52 |
3.41 |
|
|
N
= |
261 |
132 |
305 |
226 |
302 |
109 |
109 |
1,444 |
|
n
= |
33 |
34 |
30 |
30 |
31 |
31 |
32 |
221 |
|
Employment
Services Organization |
*
|
3.7 |
*
|
30.0 |
90.1 |
1.8 |
76.0 |
30.3 |
|
Commercial
Merchandise Retailer |
21.5 |
12.0 |
37.2 |
28.8 |
2.2 |
49.8 |
*
|
17.0 |
|
Educational
Institution |
44.7 |
17.8 |
14.9 |
7.4 |
*
|
7.0 |
*
|
15.7 |
|
Community
Rehabilitation Facility |
*
|
36.4 |
13.4 |
*
|
*
|
*
|
14.8 |
11.2 |
|
Consumer |
0.2 |
8.2 |
7.0 |
3.3 |
0.3 |
1.5 |
*
|
3.1 |
|
Individuals
(Private) |
0.4 |
2.4 |
2.7 |
1.3 |
*
|
24.1 |
7.1 |
3.0 |
|
Commercial
Services Retailer |
0.8 |
6.5 |
5.7 |
1.9 |
0.2 |
6.1 |
0.8 |
2.8 |
|
Vocational
Assessment Services Organization |
*
|
5.2 |
*
|
*
|
*
|
*
|
1.3 |
1.3 |
|
Other
Medical/Allied Health Services Provider |
0.6 |
1.6 |
2.7 |
2.1 |
*
|
2.3 |
*
|
1.1 |
|
Transportation
Agency |
1.8 |
*
|
0.8 |
2.9 |
0.9 |
*
|
*
|
1.0 |
|
Other
Sources |
2.3 |
1.0 |
0.6 |
*
|
*
|
0.2 |
*
|
0.7 |
|
Other
Public Agency |
1.1 |
*
|
0.5 |
0.5 |
0.1 |
0.3 |
*
|
0.3 |
|
Employer |
*
|
1.3 |
*
|
*
|
*
|
*
|
*
|
0.3 |
|
Adaptive
Equipment Suppliers |
*
|
0.6 |
*
|
*
|
0.7 |
*
|
*
|
0.3 |
|
Other
Private Agency |
0.2 |
*
|
*
|
0.5 |
*
|
0.2 |
*
|
0.1 |
|
Member of
Consumer's Family |
*
|
0.5 |
*
|
*
|
*
|
*
|
*
|
0.1 |
|
In-House
Purchase (internal to project) |
*
|
0.0 |
*
|
0.0 |
*
|
*
|
*
|
0.0 |
|
Hospital |
*
|
*
|
*
|
*
|
*
|
0.1 |
*
|
0.0 |
|
Independent
Living Center |
*
|
*
|
*
|
*
|
*
|
*
|
*
|
*
|
|
Welfare
Agency |
*
|
*
|
*
|
*
|
*
|
*
|
*
|
*
|
|
Cannot
Determine |
26.4 |
2.7 |
14.6 |
21.5 |
5.5 |
6.6 |
0.0 |
11.5 |
|
TOTAL |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
100.0 |
Note: Unless otherwise indicated, numbers are percentages. * = zero value 0.0 = less than .05 value
InfoUse’s estimates of costs per participant focused on average annual purchased services costs and average annual total costs per participant, based on a sample of consumers served during one year of each demonstration project. This method underestimates total costs because some cases may still be active. (Note that these annual figures are different from the purchased services costs reported in another section of the current report, based on total purchased services per participant, across the six years of CEEP.) Nevertheless, InfoUse’s cost findings allow for comparisons across projects.
Table 13 indicates that average annual purchased services costs ranged from $654 to $3,306 per participant across the seven Choice projects. For all projects, average annual purchase service costs per participant was $1,700. In contrast, VR’s average annual cost per consumer was $997. BCIL CEEP’s average annual purchased services costs of $3,353 are higher than all other projects. As a percentage (60%) of all annual costs, CEEP is exceeded only by the Arkansas Choice project. This indicates that CEEP spent a higher percentage of their total resources on purchasing goods and services for consumers, compared to most other Choice projects.
Table 13 also shows that total average annual cost per participant in the seven projects was $3,670, and ranged from $2,030 to $6,967 across the seven Choice projects. RSA’s VR program spent an average of $1,910 per consumer per year.
Across the projects, average per participant costs varied according to type of participant served by the projects, program practices, limitations on participant budgets, and the amount of services provided directly by project staff. Added costs frequently were incurred in providing the additional services and supports useful in expanding choice (e.g., helpers, empowerment training, extensive consumer plan-making processes). In CEEP, average annual total costs per participants were $5,552, an amount that falls in the middle of the range of all projects.
To summarize, this comparison of CEEP with the other six Choice projects shows that:
· CEEP participants reported “total choice (40.9%) or “a lot of choice” (45.9%), indicating a level of satisfaction somewhat higher than the levels reported across all Choice projects.
· 81.8% of CEEP participants reported that the project met or exceed expectations. All of the surveyed participants (100%) reported that they would recommend the project to a friend with 72.2% reporting that they actually did so. These results are higher than reported across all Choice projects.
· In CEEP, patterns of participant purchase of services varied from the overall pattern exhibited across all projects. CEEP’s participants emphasized business/vocational training (28%) and college/university training (18.9%) and obtaining computers for work and self-employment (25.3%).
· Almost all CEEP participant purchases were from educational institutions (44.7%) or retailers (21.5%).
· CEEP’s average annual purchased services costs per participants of $3,353 are higher than all other projects. As a percentage (60%) of all annual costs, CEEP is exceeded only by the Arkansas Choice project. This indicates that CEEP spent a larger percentage of resources on purchasing goods and services for consumers than most other Choice projects.
· Average annual total costs per participant were $5,552, an amount that falls in the middle of the range of all projects.
Table 13: Choice Demonstration One Year Cost Estimates
|
|
|
BCIL |
AR |
VT |
WA |
SWBIR |
TDTI |
UCPA |
Total |
|
Reporting |
From: |
Jul-95 |
Jul-95 |
Jan-96 |
Oct-95 |
Oct-95 |
Oct-95 |
Oct-95 |
|
|
Period |
To: |
Jun-96 |
Jun-96 |
Dec-96 |
Sep-96 |
Sep-96 |
Sep-96 |
Sep-96 |
|
|
Total
Participants |
104 |
122 |
215 |
140 |
152 |
55 |
18 |
806 |
|
|
Total Annual Project Replication Costs |
$577,417
|
$459,654
|
$436,393
|
$436,088
|
$596,147
|
$326,618
|
$125,407
|
$2,957,724 |
|
|
Average Annual Purchased Service Cost/Participant |
$3,353
|
$2,454
|
$654
|
$957
|
$1,941
|
$1,681
|
$3,306
|
$1,700 |
|
|
Average Annual Internal Cost/Participant |
$2,199
|
$1,314
|
$1,376
|
$2,158
|
$1,981
|
$4,258
|
$3,661
|
$1,970 |
|
|
Total Average Annual Cost/Participant |
$5,552
|
$3,768
|
$2,030
|
$3,115
|
$3,922
|
$5,939
|
$6,967
|
$3,670 |
|
|
Purchased Service Cost as a Percent-age of Total Cost |
60% |
65% |
32% |
31% |
49% |
28% |
47% |
46% |
|