The two recurring problems just discussed are part of a longer list that experienced Lead Assessors should anticipate during consolidation sessions. The Lead Assessor’s strategies for resolving them will play a large part in defining whether the assessment is successful.
A group of very serious problems of this sort will be discussed in regard to ratings decisions in the next chapter.
One problem that deserves to be considered in the context of daily consolidation and consensus, however, grows out of the common situation in which team members have misunderstood the nature or purpose of part of the model and as a result have begun to disagree vociferously over what has been said during interviews. Such arguments have caused many assessments to fall apart, and so the team should be prepared for them, and the Lead Assessor must handle them with great care.
Teams cannot come to consensus about whether the information they have collected corresponds to the implementation of a particular practice when they do not all agree about what the practice entails.
Example: Company P had postponed an assessment several times. The organization was competing for a multi-million dollar contract and was required to be rated at Maturity Level 3. About five years before, one project in the organization had been assessed at Maturity Level 3. Since then, however, the organization had been reorganized several times. The assessment team consisted of seven members, two of whom were internal. The two internal members moreover did not fully participate during the early stages of the assessment, a situation that interfered with full discussions and real consensus. One had difficulties remaining with the team because of family problems and ended up missing several hours of each evening consolidation session. The other had never performed an assessment before and was shy about participating, either in the interviews or in the consolidation discussions. The Lead Assessor, who had been careful to get agreement from each team member about individual observations, found himself at the end of the assessment in a difficult situation.
When the team started to characterize trends, it became clear that based on the observations, the organization would not satisfy one or two of the Level 2 KPAs. The team member who had missed a number of evening meetings spoke up and said he could not agree, even after follow-up interviews were scheduled. This part of the problem vividly illustrates the need to keep the team together from the beginning and to continuously facilitate daily consensus. Had the internal team member been available throughout the consolidation process, his disagreements would have been resolved one way or another before the assessment reached its critical point.
A more fundamental problem, though, had to do with the team’s understanding of the model. Concerning the satisfaction of one PA, the present but silent internal team member now declared that he had always disagreed with the team’s interpretation of the model and therefore now refused to concede that his organization was not performing the activity. No amount of team discussion would change his mind. Yet further interviews were scheduled, and in one of them, the organization’s process improvement manager stated in so many words that he knew the organization was not performing the relevant activity. He added that it had been his hope that the shock of the assessment would wake the organization up.
Even then, however, the internal team member refused to agree with the rest of the team.
Had the team been together and open, the Lead Assessor would have had an opportunity to discover the internal team member’s misunderstanding of the model and might have been able to present an ad hoc tutorial to the entire team, making it much more difficult for one member to hold out. By the time the problem emerged, however, it was too late. The assessment fell apart and could be concluded only by turning it into an informal health check. The lack of a formal rating, though, ended the organization’s chances for winning its contract, and its software improvement effort sputtered to a halt.