A Step-by-Step Case History of How Organization Z Transformed Assessment Recommendations into Action Items, Implemented Improvements, and Conducted Subsequent Assessment and Improvement Cycles over a Four-Year Period
This section traces the history of how Organization Z after a multi-year plan of disciplined periodic assessments advanced from CMM Level 2 to Level 5 and improved its productivity by 47%.
Background
When Organization Z was assessed in 1998, it fully expected to be performing solidly at Level 3 on the CMM maturity scale. After an earlier Level 2 assessment, a team of long-term consultants helped write organization-wide processes and securely predicted a rating on the next assessment of Level 3. However, corporate headquarters insisted on an independent Lead Assessor for the next assessment, and the result was an unpleasant surprise. Although the new processes that the consultants had provided were on paper, few people in the organization understood them, and fewer still had begun to use them. More seriously, too few defects were being found before testing, with the result that the company (which aimed never to deliver defect-plagued products) was spending a huge amount of time and money on testing and retesting, and many products were over budget and schedule.
The initial reaction of senior management to the Level 3 assessment results was agitation and disappointment. The organization had been told it needed a Level 3 rating to keep a large contract. The brass was not happy.
The reason that the organization’s response to the rating did not produce a general meltdown was largely due to an executive who in the wake of his superiors’ anger voiced the staff view that the assessment showed no more than most of the organization’s technical personnel already knew. The organization president, taken aback, then saw that his best course was to face into the truth and organize a real attempt to improve.
Even at that point, the president might have chosen to make a few quick changes and then look toward a Level 3 "certificate." He concluded, however, that "in for a penny, in for a pound": the most important thing for the health of the business was a program of sustained improvement. Showing real courage, he explained the results of the assessment to his sponsors and made a thorough case for addressing the organization’s weaknesses. To his delight, the sponsors responded positively. They praised the president for his forthrightness and continued the contract.
A Summary of the First Assessment’s Findings
The general results of Organization Z’s 1998 assessment were not at all that unusual for a company just short of real organizational coherence. The organization’s core problem was how to systematize, unify, and implement the first fruits of a sustained effort to introduce company-wide processes. Therefore, despite real strengths, the organization showed weaknesses, especially in managing and tracking the coordination of the new software processes and in convincing project management that the use of the processes was ultimately their responsibility and not the responsibility of their software managers.
Organization Z: Final Findings from the First Assessment
Work ethic and technical competence of staff.
President’s visibility during employee meetings was taken as very positive.
Senior management recognition of the need to improve.
Company has developed a strategic view.
Integrated teams are institutionalized and generally strongly supported.
Tight coupling between different engineering disciplines in corporation.
Software engineering directorate has taken the initiative in establishing a software engineering training program.
Capital investment in PCs and tools has had a positive impact on engineers’ ability to do their jobs.
Recently hired employees were very enthusiastic about their assignments.
Critical Concerns
Numbers of initiatives are not all tied together at company level.
Since program managers are responsible for all aspects of a project, they must develop a greater understanding of existing key early warning indicators for the software portion of their project.
Some individuals outside software engineering view software as something that other people do.
The executive emphasis on process improvement has not percolated down to project management. Process improvement is viewed by some project managers as "just one more thing to do" rather than a way of gaining effectiveness in the way they do their job.
Some processes are perceived as burdensome with little perceived ROI.
General Concerns
Employees perceive the management to be "too" accommodating to customers and therefore schedules are too aggressive.
Managers perceive that engineers have too little consideration for business goals.
Weak long-term career development path.
Limited conference attendance.
Limited planned project rotation.
Not effectively using all the people.
People are stressed.
Long hours.
Critical work keeps going to the same key people.
Recommendations
Executive staff needs to form a steering committee among themselves for process improvement. They are the only group that can fix the critical concerns. This responsibility cannot be delegated and still be effective.
Company has set reasonable improvement goals. Software needs to be reviewed in the context of those specific goals. The company should transition its emphasis from means (CMM key process areas) to ends (defect reduction).
All program reviews should include an emphasis on defect prevention (both software and hardware).
Training (at a minimum, a three-day course) should be provided to program managers, their direct reports, and their key software staff to explain the importance of software early warning indicators and their effect on cost.
Each program manager should be responsible for intervening when software early warning indicators raise a red flag.
The use of metrics needs to be re-evaluated to determine which are useful and which can be consolidated.
Specimen Technical Finding: Organization Process Definition KPA (Level 3)
Strengths
Software process handbook established and on the web (easy to find).
Software metrics and historical databases available on the web.
Weaknesses
Structure of software process handbook does not always provide sufficient options.
Not structured as menu of all acceptable options for all types of projects.
Tailoring guidelines difficult to use.
Should consist of rationale for choosing options.
Web-sensitive configuration management needs to be in place.
Software development methodologies not defined in the software process handbook.
The Assessment Findings: A Summary
Set of assessment findings can be summarized succinctly. Organization Z had attempted to satisfy the letter but not the spirit of the CMM by imposing processes written by outsiders on themselves. They should have determined which of their own provisional processes worked best and then extended them across the organization through a process of horizontal and vertical cooperation.
The assessment findings not only registered the failures resulting from this shortcutthey also pointed out the reasons why it hadn’t worked. Project managers were the only ones with the authority to require and supervise the assimilation of new processes, but they did not feel personal responsibility for the initiative’s success, nor had they been given procedures to measure and report on the success or failure of the initiative. Also, a key instrumenta software process handbookneeded to inform and educate technical personnel about the new processes was deficient. Highly un-user-friendly, the handbook provided little real instruction on how to tailor the new processes for individual projects. Thus even when engineers realized that they were expected to update their processes, they had difficulty doing so.
Finally, the measures instituted by outside consultants had been so focused on passing the assessment test they had lost sight of the real purpose of software process improvementthe real-world business objective of improving quality, productivity, and predictability. For example, peer review procedures had been installed because the CMM required them rather than because they were a powerful tool to find defects early and save the company a huge amount of effort and cost. Not surprisingly, the peer reviews were implemented in letter but not in spirit.
The bottom line of the assessment findings was that Organization Z was not functioning as an organization with the efficiency of Level 3, nor could it hope to achieve the real benefits of Level 3 until it did.