As part of their SEI authorization, Lead Assessors are taught how to train an assessment team and conduct associated team building. The Lead Assessor walks the assessment team through every step of the assessment process and explains the team’s duties and how they are to be performed. Training includes instruction in what kinds of questions to ask and how to ask them, the scheduling of practice interviews, data-consolidation exercises, and so on.
Team training usually takes two and a half to three days and emphasizes situations that are likely to be encountered by team members. (With an experienced team, less time may be required.) CBA IPI and SCAMPI team training are both usually provided at the beginning of a specific assessment. However, the SEI allows SCAMPI training to be taken in one of two ways, in no case earlier than 60 days before the start of the assessment:
If one or more team members have had assessment experience and have received CBA IPI Team Training within the past six months, they can be waived from all or part of the team training at the Lead Assessor’s discretion.
- Team training specific to the assessment
- Team training delivered to a large group of potential future team members who are not currently engaged in an assessment
Team training delivered to groups of potential future team members covers the complete set of tailoring options and allowable variations for the method to prepare them for a range of situations they are likely to encounter on future assessments.
SCAMPI assessment team training materials should be tailored to fit team needs and objectives of the assessment. Tailoring provides opportunities to:
- Provide insight into the context, objectives, and plans of the particular assessment
- Communicate team members’ assigned roles and responsibilities
- Identify tailoring of SCAMPI for the upcoming assessment
- Acquaint the team with the organization characteristics and documentation
- Focus on skills that may be more critical to the upcoming assessment, such as the ability to facilitate interviews or the ability to identify alternative practices
SCAMPI team training must be provided within 60 days of the assessment. The Lead Assessor typically provides method training. Regardless of how method training is delivered to the team members, opportunities for team building should be provided to bring the team together and up to speed on the specifics of the assessment planned.
SCAMPI team training and team building provide good opportunities to establish team familiarity with the assessment plan. This includes such items as assessment objectives, organizational scope, reference model scope, and the schedule, resources and constraints for conducting the assessment. Team member input can be obtained to refine or complete the contents of the assessment plan.
Analysis of the objective evidence provided by the assessed organization, such as questionnaires, responses, or worksheets summarizing objective evidence, can follow or can be part of assessment team preparation and training.
Team members will become familiar with the instruments to be used during the assessment (e.g., questionnaires, process implementation indicators [PII] database). Exercises using the data collection tools and methods planned for the assessment should be used to provide assessment team members with an opportunity to practice techniques for data recording, verification, and analysis. This may include mechanisms such as wall charts, spreadsheets, or data reduction tools. The more familiar and comfortable the team is with these tools before the assessment, the easier the assessment will be.
The SEI Appraiser Program specifies additional requirements about delivering training to people who are not already members of an assessment team.
Team members who have participated in previous assessments are not automatically qualified to participate on subsequent ones without first attending SCAMPI team training. In such cases, the Lead Assessor is required to understand the nature of the training delivered previously and the adequacy of that training for the assessment at hand. This requires that the previous assessment be compared with the planned assessment. For example, if the team member assessment focused only on software engineering, using the continuous representation, and the planned assessment is focused on SE/SW/IPPD using a staged representation, there may be some important new concepts to cover.
There are significant advantages to having all members of an assessment team undergo team training together. Team training offers significant opportunities for team building. Also, team training courses can be tailored for the particular assessments. If the training of some team members is waived because they have had prior assessment experience, other team activities should be scheduled for team-building purposes and to bring the team up to speed on the specifics of the particular assessment being planned. At a minimum, there must be at least one event where the team gathers as a group for the purpose of establishing team norms and operational decisions about how the team will work for the upcoming assessment.
When training is provided to an assessment team for the purpose of performing a particular assessment or series of assessments, the Lead Assessor will tailor the training to the team’s needs. Modifying the team training exercises is the primary tailoring technique. Some exercises might even include data that relates specifically to the organization being assessed, although a separation of training material and actual assessment data collection must be maintained.
In tailoring team training, the Lead Assessor uses training to:
- Provide insight into the context, goals, and plans of the organization’s assessment that the team will undertake.
- Assign team member roles and responsibilities in the upcoming assessment process.
- Identify the manner in which the Lead Assessor has tailored the CBA IPI assessment method for the upcoming assessment.
- Acquaint the team with the assessed organization’s characteristics and documentation.
- Focus on skills that have a greater criticality in the upcoming assessment (such as the ability to facilitate group interviews or the ability to identify alternatives to CMM key practices that satisfy KPA goals).
Many teams have found it useful to follow team training in the same week with examination of questionnaire responses, initial document review, and scripting of interview questions.
Some organizations have established a capability to perform assessments with very limited preparation effort through the use of a pool of trained assessment team members. Drawing from an established group of experts who are accustomed to working together clearly provides a savings over time for organizations that conduct frequent assessments.
Team training provides a time to review the assessment plan with team members. They should be sent the plan in advance of their arrival so that they can think about their roles.
Experienced teams usually use the early stages of training to assess the organization’s readiness for the assessment and to validate the reasonableness of assessment estimates. Where organizational artifacts such as procedures, plans, and so on exist, they should be used in the training. Just-in-time training can also be used at appropriate points in the assessment process to re-emphasize method concepts.
During team training, it is useful to arrange an interview with a project manager whose project is not expected to be selected for the assessment. In theory this is a practice interview, but as all projects reflect the organization’s management practices, the interview also contributes real information.
For most assessment teams, the first interview is the most difficult, partly because the team is not used to working together and partly because the team doesn’t know what to expect from the organization personnel. Will they be overly chatty or try to not answer the questions? Will they only answer the questions they are asked? This practice session shows the team what to expect during actual assessment interviews.
The risks of too little assessment team training are obvious. Consider Organization X, which requested a Level 3 assessment after several years of preparation. The organization had never experienced an assessment. During the assessment, although it became clear that the organization had fulfilled a few Level 3 goals, real concerns surfaced concerning many Level 2 areas. In the consolidation period, the team attempted to identify interview responses to help it demonstrate the performance of necessary Level 2 and Level 3 activities. Much arguing ensued. At that point, it became clear that not all team members understood what Level 2 or Level 3 were about. It was therefore necessary for the Lead Assessor (starting at 10:00 p.m.) to conduct an on-the-spot tutorial about the CMMI. In the end, the organization was rated at Level 1. Better pre-assessment training would not only have cut down on the stress of the assessment but also might have helped the organization to schedule a more appropriate assessment in the first place.