This is a article about the theory and practice of process improvement assessmentshow assessments work and what they accomplish. It focuses on software assessments because of the industry’s by now extensive experience with them, but it also implicitly addresses the kind of systems and even hardware assessments that have recently evolved along the same lines. The article is meant to help managers and engineers understand what process improvement assessments aim to do and to help them think about what assessments provide in return for a substantial cost in time and money.It is also aimed at instructing them in how best to prepare for an assessment and how to get the most out of it. Finally, it is intended as a guide for working assessors in the theory and practice of conducting assessments effectively.
For an exploration of the value of assessments, see Why Do Organizations Have Assessments? Do They Pay Off? .
On one level, assessments can be thought of simply as tools for facilitating process improvement. They analyze the strengths and weaknesses of how an organization really works by examining its business, management, and engineering processes and their analyses and results can only be understood within the larger framework of the description of structured software development articulated by a sophisticated software improvement model. The most powerful of today’s models are the SEI’s Capability Maturity Model (CMM) and Capability Maturity Model Integration (CMMI), both of which emphasize the importance of detecting defects early and then preventing them. The process improvement initiated by these models involves an organizational discipline that recognizes and deals with problems early, accepts independent quality reviews, and promotes discomfort when quality procedures are missing.
But because increasing an organization’s level of discipline involves changing people’s expectations and motivating them to make appropriate adjustments at specific stages in the improvement process, assessments amount to more than strictly analytical procedures. They also function as instruments for organizational change. Assessments, which require an organization’s active and willing involvement and which build on broad participation, are not the same as audits or external evaluations. The latter can be performed by outsiders, and they usually make insiders feel as if they are still in school and are receiving a report card.
It is true that the CMM and CMMI are associated with auditing procedures for the cases that arise when an outside organization wants to evaluate the organization being audited for the purpose of, for example, deciding whether to use the organization as a contractor. These audits are called Software Capability Evaluations (SCEs) and SCAMPI/SCEs respectively and are treated briefly in A Brief History of Process Improvement Methodologies and Assessment Methods. This article, however, is principally about assessments and will mention these other procedures only in passing.
Because assessments are group efforts at self-analysis, they have the power to effect real improvement, and the crucial differences between assessments and audits have generated rules that are critical to an assessment’s potential to motivate change. For example, one core aim of assessments is to fix problems, not people. Thus, assessments focus on how organizational structures work, not who did what in the past or who gave the assessors the lowdown. Interviews and the information they produce remain confidential. No statement made in an interview may be traceable to a given individual. These procedures result in a non-judgmental climate, which turns out to be crucial for helping to leverage an organization out of the dysfunctional patterns of a blame culture.
Because assessments are participant-based activities, they also help an organization "buy into" or "own" the improvement plans that come out of them. When proposals for change grow out of ideas generated by a collective effort rather than being imposed from above, people are much less prone to resist them.
But for organizations really to "own" the results of an assessment, the people in them must believe that an assessment has less to do with passing a test than with helping an organization get better at what it does. Audits evaluate organizations from without, which can be beneficial but is often discouraging. Because of the activities involved in the way they work, assessments help transform an organization into a more functional and more successful version of itself during the course of the assessment.