Detailed Debriefing Guidance from the Trenches
This article was originally published on August 11, 2016 in The Legal Edge
Gordon Kyle has been providing procurement advisory services in the Atlantic Region for more than 20 years, guiding public-sector entities through more than 80 procurement-related projects. As many of you might remember, Gordon partnered with NECI and the Province of Nova Scotia in the development and roll out of the Public Sector Procurement Program (PSPP). Now semi-retired, Gordon continues to oversee procurement advisory projects for his firm, Gordon Kyle & Associates and participates as an online course facilitator and occasional instructor for PSPP courses in Atlantic Canada.
In a recent sit-down with Gordon, we discussed the nuances of a successful debriefing, and how he approaches this thorny issue with his clients on major infrastructure and other public-sector procurement projects. Gordon describes one of his key objectives for implementing a robust evaluation, documentation and debriefing process as “happy losers.” In other words, the objective is to have unsuccessful respondents eager to compete for the next opportunity, rather than feeling dejected, confused, and unable to understand where their proposal fell short.
Like most project success indicators, a good debriefing experience starts with procurement planning. The evaluation criteria and weighting section of the RFx should very clearly explain to respondents how their proposals will be evaluated. It should also provide similar guidance for the evaluation committee. The evaluation team first scores the proposals independently, and then comes together for the consensus meeting. Gordon’s process requires that, if a respondent falls below 80 percent of the allocated points for a category, consensus notes are created to document why they fell short. If the project is particularly sensitive, it might even be necessary to document the rationale for scores above 80 percent, although this of course slows down the process considerably.
When it comes to documenting reasons during the consensus meeting, Gordon encourages participants to focus on two key areas:
• What would be helpful for respondents to know so that they can improve future responses?
• What would the auditors need to see in order to be satisfied that the process was fair and effectively conducted?
A key difference with Gordon’s suggested approach is that, during the debriefing, the respondent is actually provided with a copy of the detailed consensus scoring notes for their proposal. These notes reflect the scoring on each of the evaluation criteria, providing guidance for respondents to the granular level. It has been Gordon’s experience that the more detail provided, the lower the likelihood is of any Freedom of Information and Protection of Privacy applications, legal challenges, or further enquiries. During the debriefing session, his team will review some of the high-level comments, leaving the respondent to review the more detailed feedback at their leisure.
While the process of documenting detailed consensus notes is both time-consuming and not without legal risk (all comments must be objective and defensible), in Gordon’s experience, respondents to these larger, more resource-consuming competitive processes love the detailed notes and sincerely appreciate any pointers provided. He regularly has respondents thank him profusely for the helpful debriefing – suggesting that indeed it is possible to have “happy losers” in the world of procurement.
Readers are cautioned not to rely upon this article as legal advice nor as an exhaustive discussion of the topic or case. For any particular legal problem, seek advice directly from your lawyer or in-house counsel. All dates, contact information and website addresses were current at the time of original publication.