Roger Steele, the facilitator of the event, attributed this unusual dynamic to the influence of the PMD Pro1 certification examination that all the learners were scheduled to take at the end of the training week. He observed that the combination of interactive, case-based facilitation with a certification-focused examination served as a game-changer for the training paradigm. On the one hand, rich facilitation techniques (through learner participation, contextualized cases, and small-group work) increases the learner’s enthusiasm for subject matter and helps him or her apply the learning to her work context – “Did I like the training? Was it relevant?” On the other hand, including a certification examination at the end of the workshop increases the emphasis on learner accountability – “Am I studying the materials and wrestling with the often complex concepts included in the training?” Together, these two elements help increase the likelihood that the training content is learned, retained and applied in the learners’ organizational working contexts. There are, however, considerations that organizations should keep in mind when deciding whether to link their training events to certification tests. For example, one important consideration is whether the certification examination is connected to a valued/recognized credential. In the case of the PMD Pro1, the certifcation and exam are accredited through the APM Group – the same group which manages the Prince2 certification for project management. As a result, the learners’ motivation to succeed on the exam is enhanced by the credential’s reputation and credibility.
A second consideration is the cost of the effort (in terms of time, people and money). While the PMD Pro1 employed a number of strategies to constrain costs, the upfront work in creating the credential was substantial. Some of the strategic approaches the PMD Pro1 group employed to reduce costs included the decision to make the test a a multiple-choice, knowledge-based, on-line exam. This approach automated the distribution and grading processes for the exam, making it much more affordable to manage. The PMD Pro1 also benefited from economies of scale that result from multiple organizations working together, and from its strategic relationship with the APM Group. A third consideration when developing certification tests is reducing the risk of examination bias. For an exam to be fair, test-takers need to be working on a “level playing field.” There can be no inherent (dis)advantages to a particular group of test takers. Once again, in the case of the PMD Pro1, the working group that developed the certification (including representatives of LINGOs, World Vision International, Catholic Relief Services , Oxfam and others) took a number of steps to hedge against the risk of examination bias and to ensure that the test was fair. More specifically, some of the areas of concern they work to address included the following:
- Question Bias: Are the test questions clear? Do unambiguous answers exist? Is there a baseline document against which the test questions are written? To address these challenges, the PMD Pro1 developed a baseline guide from which all the questions were developed. The questions were developed with the assistance of instructional designers and editors who checked to ensure that the questions were clearly written and linked to clear learning objectives. Furthermore, all the materials were studied by a panel of over 30 reviewers (comprised of a mix of project management experts, learning professionals, development sector representatives and others).
- Language Bias: Do native-English speakers have an advantage? The experience of the PMD Pro1 exam is that test takers who identify English as a second language (or third) have had lower test passing rates. For that reason, all test takers taking the English language exam on line have the option of extending the time limit from 60 minutes to 90 minutes if they require more time to navigate the exam language. A longer term solution is to make provisions for translations. The PMD Pro1 guide and exam are currently being translated to French, Spanish and Portuguese with the intent of making the credential more accessible (and fair).
- Audience Bias: This is one area where the PMD Pro1 examination needs to focus. Very preliminary data indicates that the staff of international NGOs (both the HQ-hires and country office-hires) have pass rates that are higher than the passing rates of staff of local NGOs. This data is new and will need to be tracked and analyzed further over the coming months. That said, some initial ideas for addressing this area of potential bias might include any of the following approaches:
- Ensuring that there are a variety of learning tools that can reinforce the face to face instruction . The PMD Pro1 for example, provides access to the Guide to the PMD Pro1, practice examinations, and instructional videos. An on-line course sequence is also in the works for the coming year.
- Provide the option of delaying the examination. The data comparing pass rates is drawn from learners who sat the examination on the last day of a five-day workshop. This might mean that some training participants need time to study. Remember, many of the learners come from backgrounds that might not have a formal test-taking culture, or it might have been years since they have taken and examination.
So, while it is clear that challenges exist when developing certification mechanisms that are linked to F2F training events, the initial data from the World Vision workshops and other PMD Pro1-focused training events around the world indicates that a mix of contextualized, interactive facilitation with a focus on test-based certification results in learning events that combine the benefits of participation AND accountability.