An assessment can realistically make an in-depth sampling of only three to five projects. However, it is possible to interview selected personnel from other projects to augment the in-depth part of the assessment. Projects being considered by the in-depth part of the assessment, however, must be represented at all the interview sessions. That is, after a project has been selected for an assessment, its project manager and all appropriate project members must be available to participate in interview sessions.
Selection of projects is critical to helping the organization move forward. The criteria involved for selecting representative projects include:
- Do they produce typical products?
- Are the processes they use representative?
- Do the projects represent a quality mix? (Some should represent the organization’s best-managed projects, others should be average, and one project should represent a project that is or has been in trouble.)
- Are they important in the current and future business of the organization?
- Are they representative in terms of their product size, staff size, and project length? (Normally the projects selected should not have durations of fewer than six months. Selected projects should be at varying points in the life cycle, with emphasis on later stages.)
- What kind of application types and domains do they involve?
- What kinds of technology do they employ?
Assessed projects should represent the kind of business the organization has and expects to have in the future. Project selection should also take into account the impact of the project on the business in terms of revenue, profit, and strategic value. Factors to consider are not only the current business value but also future business value, size, and safety criticality. Looking at a project in a business area that is not strategic for the organization will not necessarily benefit the organization. It is better to include a project that represents the future direction of the organization because the assessment team can help the organization determine if the processes chosen will work on future projects.
Projects selected should be at different stages of their life cycles (or development) because the team needs to understand the processes used throughout development and have the project members think about whether they would use the same processes again or how they would improve those processes. A typical mix might include one project that has almost completed its development and is in the last stages of testing (or in the maintenance phase), a second project at the beginning of its life cycle in, say, system requirements or software requirements, and a third project in the middle of its life cycle. (A project at the beginning of its life cycle can be improved by learning positive and negative lessons from projects currently being assessed.)
The portion of the assessed organization represented by the projects (this may be calculated in terms of the number of projects assessed or by the number of staff involved in the projects assessed) affects the confidence of the assessment results.