Organization Z organized a retreat, devised a post-assessment plan, and parceled out assignments for implementing it. In the following, we concentrate on a few significant parts of that implementation and correlate three of its most important concerns with the relevant findings of an informal CBA IPI (or health check) conducted one year after the original assessment and then with corresponding findings in a second CBA IPI assessment conducted two years after the original assessment and one year after the health check.
Concern #1 (Program Managers Need to Take More Responsibility for Software Issues and Process Improvement):
"Since program managers were responsible for all aspects of a project, they must have a greater understanding of existing key early warning indicators (don’t need more) for the software portion of their project."
Initial Actions Taken:
- Assignments were made to produce a plan to bring program managers up to speed on software early warning indicators by training and coaching them in how to integrate such indicators in the management of their projects.
- Assignments were made to produce a plan to have all program managers present early warning indicators to the executive staff at monthly program reviews.
These actions were implemented within several months of the first Level 3 assessment.
One Year Later:
Within a year after these plans were implemented, program managers were including in their monthly reports to the president and executive staff quality, cost, and schedule information associated with software early warning indicators. (The indicators were graded red, yellow, and green for threshold target expectations that were being missed by a lot, missed by a little, or met.) The project managers were required to explain the measurements on which their reports were based and also to respond to each yellow and red indicator with a plan to put the projects back on track. No longer able to claim that software was entirely the responsibility of their software managers, they began to take a hands-on role in managing the software component of their projects.
Two Years Later:
Program managers are now:
Participating in managing the software portion of their projects.
Becoming comfortable with the use of early warning indicators to better manage their programs.
Identify product and process problems earlier in life cycle.
Improve vertical communication.
Focus management attention on correct issues.
Beginning to use quantitative management.
Executive managers are also using early warning indicators to improve executive focus.
Metrics make it easier to recognize problems early.
Concern #2 (Company-Wide Processes Are Not in Widespread Use):
In the area of process tailoring, the assessment had evaluated the organization’s approach to standardizing and tailoring software processes on individual projects as inadequate because neither managers nor developers had sufficiently understood or implemented the processes the organization had formulated in the company handbook. Moreover, the assessment found that the company’s process handbook did not contain options for all types of projects and was not easy to use, nor did it provide a set of rationales for choosing options.
Initial Actions Taken:
Assignments were made to produce a plan to restructure the organization’s (web-based) process handbook to make it more user-friendly. It was recommended, for example, that it should be text-based instead of flow-chart-based, and that it should specify a set of options for each process and provide criteria for tailoring these options on different projects. The plan also specified that when undertaken all tailoring should be recorded in a project software development plan and that an SQA group was to monitor whether organization staff followed the processes they were assigned. (If not, why not?)
One Year Later:
One year after these plans were implemented, a health check determined that the new web-based handbook had made it much more effective. The health check determined that organization technical personnel had started to use the processes the new handbook contained, and that they were beginning to fulfill their obligation to inform the SEPG about which ones worked and which ones needed improvement.
Two Years Later:
The second assessment showed that the software process handbook was being accepted and used by even more people than during the first year and that process-tailoring guidelines were more widely accepted by the projects. These developments had also resulted in improvements in the quality of the processes that were being used.
Concern #3 (Defect-Detection):
In the area of early defect detection and defect prevention, the first Level 3 assessment recommended that instead of trying to "satisfy" individual CMM KPAs, the software division should focus on their real business goals: finding and fixing as many defects as possible before testing.
Initial Actions Taken:
Organization Z created a plan to make Fagan Inspection training required for all software and SQA personnel.
A plan was constructed to initiate Fagan Inspections on pilot projects and then to track their usefulness.
The software division created a plan to complete a certain number of Fagan Inspections per year. (This goal was then included in a software performance improvement plan and tracked monthly.)
Organization defect prevention teams were made part of a software performance improvement plan, and project defect prevention teams were made part of software development plans for each project.
A plan was made to train all teams in causal analysis and to track lessons learned.
One Year Later:
The software organization had been trained in Fagan Inspections, and these inspections had been made mandatory on all projects. Inspections were not required for all products but were required on select high-risk products.
Two Years Later:
"There appears to be buy-in by all levels on the value of Fagan Inspections."
"Defects are tracked by type and phase injected/detected. The numbers are analyzed and used by software managers."
"Quantitative goals on defects escaping to the field and defect detection prior to test have been established and progress toward the goals is being monitored."
"Most programs have defect prevention (DP) managers who coordinate project defect prevention activities and share DP ideas and status with other project DP managers."
Defect detection rates before test rose from 50% to 63%, resulting in a significant drop in testing cost and effort.