Final findings presentations include information such as the number and type of interviews held, the number of questionnaire participants, an account of the assessment process, a list of projects investigated in depth, strengths and weaknesses of KPAs/PAs investigated, and the KPA/PA profile.
After rating is completed, the team revises (beginning with the first KPA/PA) all the draft findings charts for use in the final findings presentation. Needed changes are made, including adding or deleting strengths or weaknesses. The team uses this opportunity to carefully describe weaknesses that were discussed during final consolidation. Any recommendations that the team has noted for improving existing processes are also noted at this time. This is done for every KPA/PA within the scope of the assessment.
A summary KPA/PA profile is completed that lists all the KPAs/PAs and KPA/PA ratings satisfied, not satisfied, not rated, or not applicable. This chart also displays the maturity level rating for the organization.
The team must clearly address the organization’s overall strengths and weaknesses (those things that cut across the organization or that have to do with the organization culture).
A set of recommendations is usually appended (though this is not mandatory). Such recommendations make it easier for the organization to begin to think about how those areas in most need of improvement can be tackled.
As throughout the assessment, the team must come to consensus on each and every statement and slide.
Preparing the final presentation also involves deciding on the order of the slides and the person or people who will present them. In most cases the presentation should consist of an introduction (on the assessment process, how many people were interviewed, which projects, etc.), an account of the strengths and weaknesses of each KPA/PA, an overview of the organization’s strengths and weaknesses, and finally a summary display of the KPAs/PAs and the maturity level rating. Recommendations are one of the last items. The presentation must conclude with an indication of how the organization is going to follow up on the assessment results, such as action planning, training, and so on.
The Lead Assessor may decide to change the sequence of the presentation, depending on the involvement of the sponsor. If a president or managing director seems not to be focused on each and every KPA/PA but is interested in what is said about the organization’s culture and how it could be improved, placing the presentation of overall strengths and weaknesses and the section with recommendations before the KPA summary might be preferred. If the sponsor becomes interested in the overall organizational comments, he or she may then be more amenable to listening to the detailed KPA/PA analysis.
Another decision that needs to be considered is when to announce the ratings. Most teams think it is best to leave this to the end of the presentation so that the audience remains attentive. However, it may be that participants neglect most of the data in their anxiety to hear the ratings. Various presidents and managing directors whom we have interviewed have told us they much prefer getting the rating out of the way so that they can concentrate on the meat of the presentation. Clearly it is useful to discuss this decision with the leaders of the assessed organization beforehand.
Although the sponsor doesn’t have to be told the results of the assessment before the final presentation, it is usually appreciated. Most senior managers do not like to be surprised. When pre-briefed, they can play a significant and useful part in the final presentation and a more involved part in the post-assessment improvement program (see, "Presenting Final Findings Informally to Senior Management [Optional]").
Choosing who presents the results is important. A particular presenter can either help an organization accept the results or provoke hostility. This choice is particularly important if the results are not what the organization had expected.
The Lead Assessor may of course give the entire final findings presentation himself. Some organizations will accept the data more easily if an outside authority presents it.
On the other hand, if respected internal assessment team members present some of the technical findings in their own words, the organization may accept the information more easily.
Final Findings PresentationA Sample Template
The following represent typical items found in a final findings presentation. Each assessment develops a presentation appropriate to the organization.
Final Findings PresentationA Sample Template
(Final findings presentation is to provide the sponsor with the results of the assessment. Typically, there is a broad audience from the assessed organization.)
Title slide:
Name of assessed organization.
Assessment dates.
Findings presentation includes agenda:
Scope of assessment.
Overview of assessment.
Reference model.
Findings.
Rating process.
Next steps.
Business goalsinsert sponsor’s business goals that motivate the assessment (optional).
Organizational scopestate the portion of the organization included in the assessment.
Assessment team and support:
Names of assessment team members listed in alphabetical order with organizational affiliation specified.
Support staffname(s) of site coordinator(s) and other support staff.
Overview of assessment:
Quantify number of interviewees, e.g., project managers, management representatives, software practitioners.
Number of questionnaires completed.
Number of observations recorded.
Number of final findings recorded.
Onsite activitiesinclude chart showing onsite activities.
Reference modelinclude chart showing process areas of reference model.
CMM/CMMI scope:
Indicate the process areas included in the assessment data collection.
Include 1-3 slides for each process area within the model scope of the assessment indicating strengths and weaknesses.
General strengths.
General weaknesses.
Indicate issues identified that impact the development processes that might not be related to the reference model.
KPA/PA profileuse a chart that indicates all of the process areas of the reference model, showing which process areas are satisfied, not satisfied, not applicable, and not rated. The maturity level may also be shown on the profile.
Recommendations.
Next stepsindicate by dates when the final report is to be delivered (optional), when action-planning activities are scheduled, and when a re-assessment is anticipated.
Examples from Final Findings Presentations
1.0 An Example of a Final Findings PA Strengths and Weaknesses Statement
(Level 3 PA = Organizational Process Focus)
Description:
The purpose of the Organizational Process Focus PA is to plan and implement organizational process improvement based on a thorough understanding of the current strengths and weaknesses of the organization’s processes and process assets.
Strengths:
A number of people are involved in process improvement:
SEPG.
CMMI project.
Various new tools and component development initiatives.
The managing director oversees all major activities.
Most people know that they can provide improvement suggestions to the SEPG.
Weaknesses:
Process improvement activities are not always well coordinated:
For example, SEPG and CMMI Project are not aware of all process improvement activities.
Plans are not clearly coordinated between all activities.
There are different steering committees for each activity (although some members overlap).
Responsibility between process improvement activities is not always clearly defined.
Most people do not submit improvement ideas to the SEPG, and if they do, they do not receive a timely response from the SEPG:
This has caused some people not to continue to submit ideas.
Most of the technical people doing process improvement do not see a need to coordinate with the SEPG.
There are no clear criteria for prioritizing candidates for process improvement (both large and small activities).
When improvement projects are done, there are few measurements, few lessons learned, and not enough information fed back to the SEPG:
This may prevent the organization from learning and improving.
2.0 Examples of Final Findings Strengths and Weaknesses Statements Having to Do with the Process Improvement Activities of the Entire Organization Rather than with a Particular Process Area
Strengths:
There is an overall sense that process improvement activities undertaken over the last two years have benefited the organization.
Quality has improved.
Production errors have been reduced.
Quality reviews have been positively received.
Peer reviews have helped to improve quality of the products.
Customer relationship is good.
In general, the procedures have made life easier.
Project Management procedures are a big strength, and people use it.
It is easier to participate in projects across different sites.
It is seen as a benefit that all people must follow the same procedures and must document what they have done (new people not totally lost, and reuse is easier).
Fewer people work late hours.
People in the organization have all said:
They have a sense of belonging and pride in working here.
There are many skilled people willing and able to go the extra mile.
People in the organization have a lot of knowledge about the business.
Weaknesses:
The message from management is perceived as:
Process improvement (CMMI) is not as important as project work.
The company has a tradition of being a "not invented here" organization.
Not many take action to improve the processes they use.
The concept of identifying defects before test is not engrained in the organization.
Some view "more processes" as only more bureaucracy.
It seems some people think that creating a firm structure kills innovation.
Need to understand mature organizational behaviors.
No culture of blaming people (picking scapegoats).
Being self-critical"I can do this better."
Realizing that looking for defects in all phases is the key to better productivity, predictability, and quality improvements.
Other Key Weaknesses:
Most people do not know the number of defects found before test, during test, and in production.
The different process improvement initiatives are not coordinated and controlled centrally.
Development methodologies are not captured at the organizational level, and in some cases, different methodologies are used on the same project.
Senior management does not consistently review summary measurements for the project processes, e.g. configuration management, design, function tests, etc.
3.0 Final Findings Presentation: Sample Recommendations
The Organization Should Implement Inspections
Purpose: To detect, measure, and fix defects before test. This approach will improve quality and ultimately lower the project cost.
Implementation: Inspections are a defined methodology, which the organization should adopt. This means training people in how to conduct inspections and collecting, regularly analyzing, and reporting results. The inspections will replace some of the known peer reviews that are currently being conducted.
Target audience and areas: Developers should conduct inspections for requirements, design, code, test plans, and test procedures.
Number of defects per review are categorized by severity and should be reported into the Measurement and Analysis Tool as a basis for statistical analysis.
The Organization Should Restructure Its Process Improvement Program for Greater Benefits
Purpose: To ensure that all process improvements are collected, evaluated, and implemented consistently throughout the whole organization.
Implementation: New structure should be created and put in place to drive this effort.
Target audience and areas: The whole organization will be involved in this effort.
The Organization Should Provide Support to the Developers
Purpose: Good and reproducible education.
Facilitate and support the learning process.
Facilitate job rotation within the organization and thereby increased possibility for personal development.
Implementation: Developer cookbook with underlying technical methodologies should be created.
A mentoring program should be reestablished for IT specialists and IT professionals.
Target audience and areas: Developers in all application areas and the IT specialist focus group.
Senior Management Needs to Improve Reporting Procedures
Purpose: To provide management with early warning indicators.
Implementation: Monthly project reviews that include data on quality, productivity, and predictability (max 10 key measurements per project in a stoplight chart).
Target audience and areas: Senior manager and executive staff.