After interviews and document reviews are concluded and their data are consolidated into observations, a set of draft findings is synthesized. These draft findings are publicly presented to all participants to give them an opportunity to comment on the findings’ validity. After the draft findings presentations, the team conducts another consolidation meeting, where it prepares its final findings and (if desired by the sponsor) a maturity rating. A final findings presentation is then mounted for the sponsor, all assessment participants, and whoever else the sponsor chooses to invite.
Consolidating Draft Findings
"Draft findings" constitute a preliminary description of the strengths and weaknesses of the organization’s assessed processes. Draft findings are keyed to KPA/PA category and usually record on average three to four important strengths and weaknesses in that category. At this stage in the assessment, it is important for the team to start summarizing the key points of its analysis. The wording of each statement of a draft finding is crucial. It must be framed in objective, easily understandable terms, and must exclude both jargon and animus. Ratings are not considered during draft findings meetings.
Draft Findings Meetings
A full set of draft findings is presented in a meeting to all the assessment participants. It is the first time the staff members who have been interviewed will have a chance to see whether the team has understood what they have been shown and told. Draft findings serve to correct misinterpretation and to gain acceptance by the assessment participants.
An assessment team usually stages two draft finding presentationsone for practitioners and one for managers. The assessment sponsor is not present at these meetings to provide a chance for assessment participants to reflect on what they have actually said and what they want to say to senior management. Participants are actively encouraged to express their views as to the accuracy of the findings.
Draft findings are considered to be part of the data collection process and therefore preliminary to rating discussions.
Final Findings Consolidation and the Establishment of Ratings, Usually Including a Capability Maturity Level Rating
The assessment team consolidates data obtained during the draft findings presentations and during follow-up interviews and document reviews. When the team certifies that full coverage of the reference model has been achieved, the rating and final findings process begins.
Final findings are arrived at in consolidation meetings after the draft findings meetings and subsequent follow-up and revision. The final findings represent the strengths and weaknesses observed during the assessment for each KPA/PA in the assessment scope plus any other issues outside of the reference model that are relevant to the organization’s development processes.
Ratings are different from final findings because they involve translating the strengths and weaknesses of the organization into the analytical categories of the model, and they require a thorough understanding of what does or does not correspond to the model’s requirements.
Each goal for each process area in the assessment scope is rated based on consensus of the entire assessment team. For each goal, the team reviews all weaknesses that relate to that goal and asks: "Are these weaknesses in aggregate significant enough to have a negative impact on the goal?" If the answer is yes, the goal is rated "unsatisfied." If the team decides that the weaknesses in aggregate do not have a significant negative impact on the goal, the goal is rated "satisfied." For a SCAMPI, the rules are less subject to team judgmentall practices must be rated either largely implemented (LI) or fully implemented (FI) and the aggregation of weaknesses must not have significant impact on goal achievement.
A process area is rated "satisfied" when all the goals for the process area are rated "satisfied." If one or more goals for the process area are unsatisfied, the process area is rated "unsatisfied." Those process areas outside of the assessment scope are indicated in the final presentation as "not rated." (One process area that is sometimes indicated as "not applicable" is the one relating to subcontract management when an organization does not subcontract any of its work.) All other process areas are shown as either "satisfied," "unsatisfied," or "not rated" to indicate the complete scope and rating results of the assessment.
Assignment of a maturity level rating is optional at the discretion of the sponsor, but a decision about whether to record one has been made at the beginning of the assessment process. All KPAs/PAs within and below a given maturity level must be satisfied for a particular maturity level rating to be achieved. For example, if an organization is rated at Maturity Level 3, all Level 2 and Level 3 KPAs/PAs are investigated during the assessment and need to be rated "satisfied" by the assessment team.
Presenting Final Findings Informally to the Assessment Sponsor (Optional)
Managers in general do not like to be surprised. At this point in the assessment, it is a useful courtesy to show the final findings and/or ratings to the assessment sponsor. Obviously, no changes are made at that time, and confidentiality is maintained.
The Final Findings Presentation
The final findings are presented to the assessment sponsor at a meeting that includes not just the assessment participants but (depending on the sponsor’s wishes) perhaps the entire organization being assessed. This meeting represents a message of the organization as a whole to its executive manager and constitutes the final result of the assessment.
The final findings presentation usually includes not only a chart of the KPAs/PAs that have been satisfied but a more general analysis of the organization’s overall strengths and weaknesses and of the health of the organizational culture.
In addition to this analysis, the assessment team during the final findings presentation can also make global recommendations as to activities the organization could usefully focus on over the next year. These recommendations can be a powerful aid in pointing an organization toward real progress.
The final findings presentation represents the sponsor’s opportunity to thank the team, thank the assessment participants, and present his or her support for follow-on improvement activities.
The assessment sponsor then "owns" the assessment results and is free to use them as he or she sees fit.
Post-Final Findings Meeting Executive Session (Optional)
An executive session provides an opportunity for the assessment sponsor to clarify any issues with the assessment team and to receive guidance about the possible focus, timing, and priorities of recommendations and follow-on activities. Confidentiality continues to be in effect.
Assessment Team Wrap-Up
In a brief session after the executive session, the Lead Assessor collects feedback from the assessment team members. Assignments are made for the writing of a final report, if one is required. One person in the organization is asked to be the steward of all assessment observations (with attribution removed). This detailed assessment information may be used for follow-on action planning by the organization. The assessment team also prepares required feedback forms for the SEI that indicate how the assessment process might have been improved.