Different jobholders are interviewed in different ways. Project managers and the assessment sponsor are interviewed individually. Most others (requirements developers, design developers, system engineers, middle managers, and executive managers) are interviewed in small groups of two to ten people. (The optimum number may be four to six.)
The rule of every interview should be that everyone being interviewed gets a chance to answer all questions.
Project Manager Interviews
Most project manager interviews come early in the assessment schedule. Other project manager interviews are interspersed with the group interviews so that coverage of the practices of the reference model can be obtained.
Before being interviewed, project managers are encouraged to gather examples of their "day-to-day" project management activities and give them to the organization site coordinator.
Each project manager interview typically lasts one to one and a half hours. If the organization runs a large number of projects, after the key project manager interviews are concluded, an additional group interview of the remaining project managers may be conducted.
In most assessments, the Lead Assessor will facilitate the project manager interviews. The team physically arranges the room to make the project manager comfortable. Eye contact with team members should be maintained.
The Lead Assessor opens the interview. Open-ended questions are asked about the organization and its processes, with mini-teams asking questions about their particular KPAs/PAs. The project manager being interviewed should be encouraged to bring data that will help him explain his job, how he plans his project, and how he monitors his project.
After the Lead Assessor completes the first set of questions, he or she will ask the team: "What other information do you need to know?" The Lead Assessor reviews the documents requested during the session with the project manager and adds them to the document list. The librarian makes note of any documents requested during the interview and gives a copy of the notes to the interviewee at the end of the interview.
After the project manager leaves, each team member updates his notes immediately. The team also identifies any significant information that still needs to be gathered and prepares for the next interview.
Unexpected and/or Sensitive Situations
The following disconcerting situations illustrate problems to watch out for:
Organization F selected four project managers for individual interviews without consulting with the Lead Assessor. When the team interviewed the first "project manager," it turned out that he had responsibility only for requirements on a particular product. The second project manager was responsible for the design of the same product. The organization, it turned out, used the title of "project manager" for development specialists. Discovering this only at the end of the first day, the Lead Assessor needed to adjust and prolong the assessment schedule, causing no small inconvenience. Moral: A project manager in one organization may not be a project manager in another.
A similar example: In Company A, two project managers were scheduled to be interviewed for the same project. The organization explained that each product development had two project managers, one responsible for requirements collection and management and a second with equal responsibility for development and testing. Together, the organization sponsor suggested, the two would be able to give a complete picture of the most important project in the company. At some point, however, it became clear that the person ultimately responsible for delivering a quality product on time and within budget was someone else entirely with the title of VP. When the Lead Assessor asked to set up an interview, however, the VP said he would prefer to be interviewed with his two subordinate project managers. He had just been promoted to his position, he explained, and he did not feel comfortable speaking about the project alone. On intuition and against assessment guidelines, the Lead Assessor made an on-the-spot decision to agree. In this case, it was the right one. By participating with his subordinates, the VP had a chance to learn not only about how the project had been run but also about how other companies ran similar projects. He made comments during the interview such as, "That is a really good question. I am going to go back and ask my team that question." After the interview, he immediately set up meetings with his staff and discussed a number of the items raised in the interview session. These meetings continued throughout the week of the assessment. It was true that the two subordinate project managers were made uncomfortable during the interview. Therefore the team set up follow-up interviews with them. But the benefit to this VP and the company was substantial. (The VP several years later became managing director of the company.) The story illustrates the positive impact assessments can have on an organization, above and beyond ratings.
Sometimes the assessment team becomes frustrated and begins to treat interviewees harshly. This has serious consequences and should be anticipated and resisted.
In Company X, which was matrix-organized, project managers were responsible for the entire product, hardware and software. During one project manager interview, the assessment team asked standard project planning, monitoring, and control questions. This particular project manager, however, despite initially being very amiable, knew nothing about the planning and tracking of his product. The team did not pick this up and continued to ask for the same information in a variety of different ways. By the end of the interview, both team members and the project manager were frustrated. The project manager went away from the interview feeling defensive and embarrassed. It took over a year for him to feel positive about process improvement again.
This kind of situation frequently occurs in immature organizations. Assessment teams should anticipate this and react accordingly. In these cases, it is useful to end the interview gracefully and then set up an additional interview with the software manager. (As an organization becomes more mature, the likelihood of such situations decreases.)