Although nothing is presented at a draft findings meeting that has not already been offered (at least twice) by members of the organization, the big picture can often come as a surprise to the organization, provoking vociferous opposition, especially from middle-level managers. In these cases, the draft finding meeting may turn into a free-for-all.
Managers sometimes have to confront weaknesses at draft findings meetings they hoped they wouldn’t have to address or thought they had already fixed. Middle managers can also be afraid that executives or corporate management will use these weaknesses against them. These managers also may fear that if the organization does not achieve an expected maturity level, their jobs could be on the line. In an organization that continues to sustain a culture of blame, these fears can have more than a kernel of truth.
Before its first assessment, the managers of Organization J thought that the organization should be rated at least Level 4, if not Level 5. However, during the draft findings meeting, management realized that the assessment team had discovered a number of unanticipated weaknesses and that the organization would probably be rated at Level 1. One of the senior managers rose up to defend his company. He said that the organization made state-of-the-art, one-of-a-kind products and that, no matter what the assessment showed, the company could not do so unless it was first-rate. The senior manager said that the assessment was clearly flawed and that its results could not be trusted. The assessment ended in frustration, and the organization did not improve. Ten years later, it still displayed Level 1 weaknesses, and several of its one-of-a-kind systems had dramatically failed.
There are ways to forestall or quiet such situations, however, and a heated but productive discussion may improve the accuracy of the assessment.
One way to head off furor at a managerial draft finding meeting is to arrange smaller feedback sessions with appropriate people through the earlier stages of the assessment. If the team has not found data to show that important practices have been implemented, it is useful to meet either with the sponsor or the organization site coordinator and explain this. The Lead Assessor at that point should ask who else might be interviewed and what other documents could be reviewed to provide evidence about a particular practice. When key individuals such as the sponsor or organizational site coordinator are kept in the loop, draft findings come as less of a surprise and produce less heat.
Also, in general it is best to conduct technical draft finding meetings before the managerial ones. Technical staff will usually discuss the accuracy of findings dispassionately, which then makes it easier to convince managers that found weaknesses are real.
In no case should assessment team members argue at the participant meetings. Instead they should explain that the findings do not represent the team’s opinions but rather evidence that has come from the people who are doing the work. The team should also eagerly elicit new evidence that might upgrade the findings, in the form of the names of additional people to interview or additional documents to review.
Because public resistance only makes it harder for the organization to make post-assessment adjustments, responsible managers should also be prepared to argue for the objectivity of the assessment process and then help their colleagues to work through whatever ratings are assigned. They may be helped to do so by remembering that although final ratings often come as a surprise, everything they are based on has been documented at least twice.
Organization X hoped to be rated Level 3 and was convinced that it would not get an upcoming contract unless it did. An assessment, however, showed significant shortfalls. Three internal members on the team consistently agreed that the organization needed to work on significant areas before it could achieve Level 3 and that it was presently not performing the practices expected in a Level 3 organization. The practitioners did not have a problem with the stated strengths and weaknesses, and the practitioner draft findings meeting went smoothly. At the management draft findings session, however, when all Level 2 PAs were reviewed slide-by-slide, heated discussion arose about what the model required. At the beginning of the Organization Process Definition PA review (associated with Level 3), it became clear that the organization had significant weaknesses, and the entire room exploded. Software managers violently disagreed with the statements their subordinates had made. Systems and project managers disagreed with each other. Everyone started talking (actually shouting). The assessment team member who was presenting the slides happened to be a manager from the organization. He tried to explain that he had taken part in all the interviews and that the draft findings statement in fact accurately reflected what the developers told the team. Remaining PA reviews produced similar shouting, and a similar defense became necessary in each case. Finally, one of the senior VPs took the floor and said, "Look, we all know that what the slides show is true. Our products still have problems. Let’s stop arguing and think about how we can make the situation better." The room calmed down, and the meeting became more constructive. The managers were asked to help with the wording of the recommendations for the final presentation to the president. Several executives, project managers, department managers, and software managers volunteered to work with the team in putting the final presentation together, making them feel a part of the assessment team. That year the organization was rated at Level 2. Two years later, it achieved Level 3, and two years after that, it achieved Level 5.
No maturity level is determined before or during a draft findings presentation, nor is one given.
Draft findings meetings end by informing participants of the time scheduled for the final findings presentation to the sponsor and reminding them that they all are invited to attend. The report at that point represents a communication from everyone who has participated in the assessment to the sponsor. The assessment participants should also be reminded not to divulge any of the draft findings prematurely, as they are still preliminary and may change by the time of the final presentation.