When consolidating collected data, the assessment team members synthesize and summarize the information already obtained during data-gathering sessions and eventually construct "observations" that relate organizational practices to the reference model.
The information involved in consolidation comes not only from interviews but also from presentations and document reviews, some of which will have been conducted early in the onsite phase of the assessment or even before.
It is essential that all determinations during consolidation be made on the basis of information obtained from multiple independent sources.
The team must come to consensus on the accuracy and validity of all "observations." Consensus, although sometimes difficult to achieve, ensures that the assessment team assumes ownership of the assessment results. One disgruntled team member who later claims that he or she did not agree about an issue can compromise the assessment results.
The process of reaching consensus takes place during every phase of an onsite assessment. Though it sounds like something that belongs to the end of an assessment, consolidation is a daily activity that an assessment team must perform immediately following every documentation review and presentation and at the end of each day’s interviewing. (Consolidation discussions even go on between interviews.) After each occasion and especially at the end of a day of interviewing, it is crucial that the team records on its data worksheets all pertinent data and makes sense of its relevance to the assessment. Otherwise, even with good note taking, information quickly blurs.
For every hour of interviewing, the team should plan 45 minutes to one hour to consolidate the interview notes onto data worksheets. The required time depends on the team members’ experience, their knowledge of the model, and how easily they come to consensus.
The Lead Assessor has the responsibility of monitoring the focus, quality, and progress of the team’s consolidation activities, both for individual team members and for mini-teams, and he or she should be actively involved in ensuring that sufficient data is being collected to satisfy all areas being investigated. The Lead Assessor provides guidance and individual feedback to team members as appropriate.
The Lead Assessor must review consolidation materials to ensure that:
- Team members are taking adequate notes.
- Notes are annotated properly.
- Observations are based on facts documented in the notes.
- Observations are properly recorded.
- Both strengths and weaknesses are identified.
- Mini-teams are properly merging observation statements and checking for coverage.
If at the end of the data-gathering process, the appropriate data has not been obtained to determine the status of an area of investigation, the Lead Assessor and the organization site coordinator must arrange to obtain the needed information. If the needed information cannot be obtained, no rating can be provided for the given KPA/PA.
Partly because it takes place each day after interviewing, and partly because it requires the team to put pieces of a large puzzle together, the consolidation segments of an assessment tend to be both challenging and contentious. The team correlates practices in the organization with the components of the model. Documentation and interviews have provided the team with an overall sense of the organization’s practices, how they are performed, and how many projects are performing the practices, but translating this overall view into specific judgments is not a quick, easy, or conflict-free task.