Although deciding on the objectives of the assessment sounds straightforward, it is not always easy to gauge what maturity levels and process areas the assessment team should appropriately examine. These consequences are significant, especially in a first assessment. If an organization thinks it is Level 3 when in fact it is really Level 1 or 2, a CMM assessment will need to look at 13 KPAs instead of 6 and a CMMI assessment at 21 PAs instead of 7. The difference translates into an added week onsite.
Just as important is the risk to the organization’s morale. It is common for senior management to think that the organization is better than it is, and when an unrealistically high standard has been selected, they may send a direct or indirect message to their people that says "Don’t fail us." The interviewees are then afraid to say the wrong thing, reverting to "audit mode." That is, they only answer the questions that are asked and don’t participate in an honest effort to examine their procedures. Thus, they get little out of the assessment other than the sense of confronting an enemy. So instead of the assessment being a positive culture-evolving experience, it ends up being a negative and stressful experience for all.
If senior managers think the organization is Level 3 and it is really Level 2 or even Level 1, the shock of looking into an assessment’s mirror can be painful. In many cases, the organization never recovers from the shock.
To ascertain what part of the model should most beneficially guide an assessment, the organization may ask the Lead Assessor to spend one, two, or three days talking to people informally. This exercise may even be elaborated into a full health check or mini-assessment.
The costs of not heeding these preliminary indications can be dire.
A managing director was sure that Company C was Level 2. When he was told both at the beginning of the assessment and during the assessment that there were areas in which no data substantiated this assumption, he still insisted that his organization was Level 2. When the final results showed that his organization was only at Level 1, at the moment when senior managers usually thank the team and the organization for their hard work, he told the audience that he did not believe the results. He then asked if anyone in the organization really believed what they just heard. Slowly the developers in the back row started to stand up, until two thirds of them were on their feet. The managing director, abashed, could only turn to his direct report managers and ask why they hadn’t listened to their developers. The next week the managing director was removed from his post. Seven years later, the organization was still Level 1.