RETURN TO MAIN REPORT PAGE

 

Challenges and difficulties encountered

Sponsored by and housed at a center for independent living, the CEEP created a unique model of choice, but also faced particular challenges in undertaking such a project.  There are lessons to be learned from aspects of the project that did not work optimally, as well as from the successes of the project. Difficulties encountered in the implementation of CEEP can be instructive for other organizations, especially independent living centers, that might attempt to build on the CEEP choice model.

Turnover. CEEP experienced a great deal of turnover during the project. According to the Research Triangle Institute (RTI) evaluation, project staff turned over 100% during the first two years, causing numerous other problems and delays in implementation (MSPD Evaluation Support Center, Research Triangle Institute, 1995).  During the duration of the CEEP, there were three different BCIL executive directors who served as project directors for the CEEP.  In addition, there were three different sets of personnel in the BCIL accounting department, which was critical to the disbursement of monies to consumers.  However, the project manager stayed with the project from Year 2 until the end, and there was only one change in the counselor positions during the same period.

            Difficulty defining success. Throughout the project, the criteria for success remained ill-defined and subjective.  The absence of a clear definition of success in the CEEP was noted in the Research Triangle Institute's national evaluation (MSPD Evaluation Support Center, RTI, 1995) as well as in the local evaluations for Years Two and Three (Jones & Jans, 1996a; 1996b).  Rejecting the idea that “choice” could be measured simply in terms of vocational outcomes, project staff wrestled with how to define success in a way that did justice to the alternative goals of the project.  Participant empowerment was viewed as essential, and a participant’s enhanced independence and personal well-being were seen as important project outcomes.  However, the assessment of success came down to the counselor, on the basis of a final interview, deciding whether the participant felt empowered and had made substantial progress toward goals.  Sometimes there were data to substantiate success, but other times the assessment was more subjective. This may have inadvertently created a pressure on participants to say that they had been successful, absent more objective ways of evaluating progress.  A measurable set of criteria would have enhanced the credibility and replicability of the project.

Problems with the process of approving and making payments.  CEEP complied with all federal guidelines for financial accountability, developing a complex system of approval and payment for consumer goods and services.  The accounting department reported that 95% of vendors were paid within 60 days, an exceptional record by industry standards. 

Nevertheless, the system for approving and making payments created communication problems and was perceived by many consumers and counselors as rigid, cumbersome, and far too slow.  The project had an extreme impact on the accounting department at BCIL.  One former executive director estimated that handling payments related to CEEP comprised 75% of the workload of the accounting department.  The system of payment approval created strains between CEEP staff and the accounting department and resulted in numerous delays in payment. Participants and project staff complained bitterly about how long it took for providers of goods and services to receive payment.  Consumers worried that payment delays made them look embarrassingly unprofessional and that service providers would see them as “second class citizens.”

We believe that many of these difficulties arose from unresolved conflicts between the need for accountability and the desire to provide maximum levels of funding and services.  These issues are also discussed below, in the section on case management and the dual roles of advocate and services broker.

Gaps in business expertise.  In previous evaluations, consumers with self-employment goals complained that project staff did not have enough expertise in reviewing business plans and guiding entrepreneurial activities.  During interviews in Year Two and Year Three of the project, project staff affirmed that they had a need for more training in these areas (Jones & Jans, 1996a, 1996b).  Over the course of the project, staff increasingly developed expertise in guiding small businesses, as well as resources for referring participants for consultation with other experts.  Nevertheless, it seems important for future projects with a self-employment focus to create these systems of expertise and resources before the project even begins. Networks should include people with expertise in business start-up, business plans, marketing, financial management, and other related topics.

Project isolation.  Many of the CEEP and BCIL staff whom we interviewed mentioned that CEEP was isolated from the rest of the BCIL. It is not unusual for such a project to become isolated when a relatively large amount of grant money for a specific project comes to a grass-roots advocacy organization.  Nevertheless, CEEP’s status as a project relatively separate from the rest of the CIL’s activities took its toll in terms of strained relationships within the organization.  This indirectly impacted consumers.  For example, one project director thought that CEEP participants could have made better use of the other services of the BCIL, such as IL skills training, benefits counseling, housing, and advocacy.  If CEEP had been wholly integrated within the organization as originally contemplated in the initial grant proposal, much of BCIL's strengths could have been brought to bear on the consumer's issues.

Case management and the dual roles of advocate and resources broker.  Whenever a grass-roots advocacy organization takes on a case management role, tensions emerge between the desire to provide consumers with resources and the need for fiscal accountability. In CEEP, for example, conflicts erupted between the project staff who tended to favor a system with maximal flexibility for consumers, and the accounting staff and executive directors, who tended to press for more rigorous accountability and control over funds. No system of disbursing funds can completely eliminate conflicts between the “gatekeeper” and “service provider/advocate” functions that are part of brokering services.  In the section on lessons learned, we make some recommendations on ways in which organizations seeking to replicate the CEEP model might work with the tensions between gatekeeper and service provider/advocate functions.

 

Return to Main Report Page