Choosing a Time
A time needs to be chosen to conduct the assessment, based on the answers to the questions:
Is the organization ready?
Will the time be intrusive in regard to deadlines and deliverables?
Has enough time elapsed since the last assessment?
Appointing an Organization Site Coordinator
After an assessment sponsor is designated, he or she will then delegate the day-to-day practical arrangements of the assessment to someone such as a process improvement manager in the organization.
This person is known as an organization site coordinator, who will coordinate both the planning phase and the onsite phases of the assessment. Along with the Lead Assessor, he or she is responsible for building the assessment plan.
The Responsibilities of the Organization Site Coordinator
The organization site coordinator helps the Lead Assessor make practical choices and ensures that all team members are available and that all interviewees for the assessment are available, scheduled, and able to appear for their sessions. The organization site coordinator also ensures that rooms are scheduled, that logistic requirements are planned and completed, and so on.
Besides coordinating the assessment logistics (such as scheduling interviewees, booking appropriate rooms for assessment activities, etc.), the organization site coordinator must arrange lodging, food, and beverage for the assessment team members, obtain documentation needed by the team, and make plans to return documentation to appropriate people when the assessment is concluded. (Before the onsite assessment begins, rooms must be booked, interviews must be scheduled, lodging for non-site personnel must be arranged, food must be organized, equipment must be planned, etc.)
The Assessment Plan
The assessment plan is the organization’s map through the assessment. A provisional big picture plan is generated immediately after the executive and the Lead Assessor meet and decide on what elements constitute the organization to be assessed, who will be the official assessment sponsor, and the reference model scope to be investigated. During the three phases of the assessment, this plan will be continually refined.
To begin building an assessment plan, it is necessary that the Lead Assessor understands the organization’s key strategic objectives and that the assessment sponsor understands the parameters of the assessment.
An assessment plan will include answers to these questions:
Who should participate in the assessment and at what stage?
What are the outcomes of the assessment planning process? (For example, what outputs does the organization need from the assessment to plan further improvements?)
When should the major milestones for the assessment’s completion be scheduled?
An assessment plan must include documentation of the assessment’s scope, the identification of assessment participants, a detailed schedule of assessment activities, and all other relevant information. The assessment plan must be agreed to and signed by the assessment sponsor, the Lead Assessor, and in most cases the organization site coordinator and the assessment team.
Creating an Assessment Team
The Lead Assessor helps create a competent assessment team, choosing key people in the organization capable of understanding organization-wide problems, concerns, and suggestions for improvement.
Generally an assessment team contains members internal to the organization as well as some from the outside. The assessment team needs to have organization-specific knowledge, but it is also necessary to have members who do not have a vested interest in the results.
The selection of assessment team members is important for the organization not only during the assessment but also after the assessment. The organizational team members are usually responsible for follow-on process improvement activities. Team members should be highly respected opinion leaders who have experience in the areas being assessed. Selecting assessment team members should not be taken lightly. Team members should not have a high personal stake in the outcome of the assessment. For an assessment to work properly, this rule needs to be followed in spirit as well as letter. People who are intimately involved with managing an activity may be too heavily involved (both technically and emotionally) with the assessment outcome and may have a difficult time listening to outside critiques. Organizations sometimes think the right people to put on the team are quality assurance staff because they are trained in performing audits. This, however, is only a part of the truth. Assessments are not audits, and assessment interviewing also requires considerable interpersonal skills.
An assessment team is composed of from four (a required minimum for SCAMPI and CBA IPI) to ten members. Fewer than four does not provide enough "group think" to provide a valid consensus. More than 10 members becomes difficult to manage, difficult to schedule, and difficult to bring to consensus. Seven, including the Lead Assessor, is an ideal size.
Assessment team members must understand that being part of the assessment team is a full-time job during the course of the onsite period.
Defining the Final Assessment Products
The precise outputs of the assessment need to be defined in the assessment plan. A final findings presentation, for instance, may or may not produce a formal maturity level rating, depending on the choice of the sponsor. A final report may or may not be requested by the sponsor.
Selecting Projects to Be Assessed
An assessment can realistically make an in-depth sampling of only three to five projects. However, it is possible to interview selected personnel from other projects to augment the in-depth part of the assessment. Projects being considered by the in-depth part of the assessment, however, must be represented at all the interview sessions. That is, after a project has been selected for an assessment, its project manager and all appropriate project members must be available to participate in interview sessions.
Selecting People to Be Interviewed
The most time-consuming responsibility of the assessment team is interviewing organization participants. These include individual or group interviews of project, middle, and senior managers. In group interviews, practitioners representing different parts of the development cycle provide breadth of coverage. For example, one group interview might include people who develop and refine requirements, a second group might include people who design products, another group might include testers, and so on. Group interviews allow people on different projects that do the same job to hear how others do their task in other parts of the organization. It is best to include as many individuals involved in the chosen projects as possible in the appropriate interview sessions. They will come away with a sense that they have been allowed their say and that they form part of the long-term improvement process.
Representatives of senior and middle management connected with the software development process also should be interviewed. They are usually interviewed as a group in order to get a management perspective on the organization, its problems, and desired improvements.
All assessment participants should ideally attend an assessment participants briefing, which will explain why the organization has undertaken an assessment, the basic principles behind process improvement, and what usually happens before, during, and after an assessment. (This is especially important for a first-time assessment.)
Distribution of Questionnaires
For a CBA IPI, the project managers from the projects chosen for in-depth investigation (in conjunction with selected project team members) must fill out a maturity questionnaire. These need to be distributed and their results summarized for the assessment team’s use. They will later guide the assessment team’s questioning during interviews.
Assessment Team Training
The Lead Assessor will train the assessment team members in the logic and phases of the assessment process. The team members must already be familiar with the reference model and should have already taken required courses of instruction. Assessment team training addresses the CBA IPI or SCAMPI assessment procedures, the assessment plan, site-specific organizational data, and the tools and techniques to be used during the assessment. Assessment team training also plays an important part in team building and establishing team norms.
The Assessment Team’s Last Pre-Onsite Activities
After the assessment team members have been trained, they need to review appropriate questionnaire responses. These responses are used to identify patterns and help focus interview questions.
The assessment team also inventories and examines initial documents. The organization site coordinator or team librarian is responsible for beginning to put these into a library. They are then reviewed for the purposes of establishing context and determining areas to probe.
For a SCAMPI, the organization is expected, prior to the assessment, to collect an entire set of documentation for review. Finally, before the official assessment begins, the Lead Assessor, the organization site coordinator, or one of the internal team members stages an assessment participants briefing to prepare assessment participants for what is to come.