The CMMI assessment method, known as the Standard CMMI Appraisal Method for Process Improvement, or SCAMPI, was first described by the CMMI product team in 2000 and then refined in the SEI’s SCAMPI Handbook and Method Definition Document .
This article focuses first on CBA IPI and then on "staged" SCAMPI assessments. The two use slightly different terminology but are closely related in practice, with a few very significant differences. At the time of publication of this article, over 2,000 organizations have successfully completed a CMM or CMMI assessment. (These figures are available in published form in and in continually updated form on the web in .)
Differences Between the "Staged" SCAMPI and the CBA IPI Method
The most important practical difference between a CBA IPI and a "staged" SCAMPI has to do with the effect of the CMMI’s (and hence, SCAMPI’s) enlarged scope. On the plus side, this enlarged scope allows assessment teams not only to evaluate multiple (e.g., software and systems) functions across a whole organization but also provides them in the software area with a more sophisticated approach to requirements management and development and gives them a more fine-grained picture of the workings of Capability Maturity Levels 4 and 5 .
However, because the CMMI model (staged representation) contains over 60% more practices than the CMM for Software, SCAMPIs could require considerably more time than CBA IPIs.
To address this problem without diminishing the quality of SCAMPI, the CMMI Steering Group formed an Assessment Methodology Integrated Team (AMIT) to modify the definition of SCAMPI V1.0 and V1.1 to SCAMPI V1.2 . The resulting modifications prescribed more upfront document gathering than a CBA IPI and fewer activities during the onsite period of the assessment.
For example, requirements for substantiating the existence of a necessary process were changed so that the new method of verification, instead of requiring two interviews or one interview and one document to substantiate a practice, required only two documents to substantiate 50% of all examined practices. The other 50% must still be substantiated by one interview and one document. Verification, however, puts additional responsibility upon the assessed organization: Documentation requirements for SCAMPI are more rigorous than for CBA IPI, and many organizations (especially less mature ones) need assistance with the gathering of the necessary documents and information. Collecting and reviewing documentation always requires knowledgeable people, and organizations should keep the necessity for their effort in mind.
Also, SCAMPI was deemed a focused investigation in which the practices to be substantiated by interviews could be determined by a kind of triage in which an initial review of the evidence collected (e.g., documents, questionnaires, presentations) is used to determine which questions need to be asked in the interviews. This triage or readiness review is meant to produce a set of questions and a team agreement that some practices are clearly practiced (or clearly not practiced). This does not mean that no further attention will be paid to these practices, only that questions about them do not have to be asked during the interview sessions .
Through establishing stronger focus on documentation and by reducing the person-to-person interaction of interviews, both modifications create real difficulties for assessments, as described in Chapter 1. First, they increase the risk that not enough hard information is collected from practitioners about an organization, which lowers the confidence level of an assessment. Second, in inexperienced hands, SCAMPIs can resemble audits rather than true assessments. This is true not only because of SCAMPI’s reduced emphasis on interviews but also because one of the original requirements for SCAMPI was that it be designed so it could be applied either in the mode of an assessment or an evaluation (audit). The result was that certain key features of CBA IPIs were altered. For example, one of the main requirements of a CBA IPI is that at least one person from the assessed organization serves as a member of the assessment team. This requirement ensures that assessment results are transferred back into the organization’s process improvement initiative. SCAMPI, however, has dropped the requirement so that like an SCE it can be conducted by an outside team if necessary.
When the CBA IPI method was being written in 1994, there was discussion regarding the feasibility of having one method for both assessments and evaluations. Those who were experienced in both methods concluded that the motivations for the two appraisals were different enough to merit keeping the methods separate. Thus, CBA IPI was the assessment method , and SCE V3.0 was the evaluation method for the CMM for Software, both published in 1996. In considering the benefits that an organization may experience during an assessment, the attributes that are at risk for process improvement during an audit are obvious. An audit is not considered to be a collegial non-threatening activity. Some important business risks are "on the line" for the organization being assessed. An audit is a very important vehicle for organizations making business decisions based on the specific results that they seek during an audit. It’s important to understand that the purposes of the two types of methods are different and could yield different results for the assessed organization.
It should be said in defense of SCAMPI, however, that although the changes just described can produce less satisfactory assessments than a CBA IPI, this does not necessarily have to be the case. In a SCAMPI, assessors are permitted but not required, for example, to reduce the extent of interviewing and experienced assessors will make sure to arrange as many interviews as possible and to take extra measures to include a broad cross-section of the organization in the interviewing process so that the internal benefits approximate those of a CBA IPI.
Because SCAMPI is a technically more rigorous evaluation tool than CBA IPI, moreover, SCAMPIs conducted with attention to motivating process improvement are potentially as or more effective than their predecessors. The requirement, for example, that for a goal to be considered satisfied, each of its practices must be observed on all or most of an organization’s projects, means that the "wiggle room" in a SCAMPI assessment has been significantly reduced.
More Differences Between the SCAMPI and CBA IPI Methods
A further difference between the SCAMPI and CBA IPI methods involves the procedure for rating practices. In a "staged" SCAMPI, a document is required to substantiate every assessed practice, whereas interviews are required to verify only 50% of them (versus the CBA IPI requirement, which requires interviews for all practices and which recommends documents as a second source of verification wherever possible). SCAMPI also requires that practices within assessed process areas be rated as fully, largely, partially, or not implemented rather than (as in a CBA IPI) implemented or not implemented.