MP5301.9001(b)
Clearance – Multi-Functional Independent Review
[Revised November 7, 2012]
1. Policy. The Clearance Approval Authority (CAA) must use Multi-functional Independent Review Teams (MIRT) as an integral component of the clearance process in AFFARS 5301.9001(a) by validating each critical decision point (CDP) as described in paragraph 3.2 of this mandatory procedure.
2. Applicability. This procedure is required for all competitive acquisitions meeting the requirements of AFFARS 5301.90 when contract values are $50M or more and where the CAA is the Senior Contracting Official (SCO), the Senior Center Contracting Official (SCCO), the Associate Deputy Assistant Secretary (Contracting) (ADAS(C)), or the Deputy Assistant Secretary (Contracting) (DAS(C)). This includes task orders for services issued against multiple award indefinite delivery/indefinite quantity contracts (to include GSA schedules). At the discretion of the CAA, this mandatory procedure may be used for competitive acquisitions below $50M or for non-competitive acquisitions at any dollar threshold.
3. Independent Review. The CAA must establish an independent and objective process, employing cross-functional subject matter experts (SME) to constitute a MIRT. The MIRT must operate in an advisory manner in order to foster frank and candid discussions regarding the soundness of the business and contracting approaches employed in the acquisition. The results of these discussions (solutions, lessons-learned, etc.) should be shared with acquisition organizations across the Air Force.
3.1. Multi-functional Independent Review Team (MIRT). The MIRT is formed at the beginning of each competitive acquisition with membership approved by the CAA. The CAA may use existing Independent Review Teams, peer reviews, Acquisition Center of Excellence (ACE) established review teams, or other established processes to satisfy this requirement. Note 1 To promote consistency, it is desirable that the same MIRT members participate in each critical decision point review for the duration of the acquisition unless otherwise approved by the CAA. The members of MIRTs must not be members of the source selection/competitive acquisition team. Note 2 The team will be comprised of government personnel, to the maximum extent practicable.
The MIRT will have representation from:
-- Technical or Requirements Activity (e.g., engineering, program management, A1, A5, etc.)
-- Legal (e.g., other program counsel, SAF/GCQ, AFLOA/JAQ, etc.)
-- Contracting (.e.g., Contracting Officer, Chief of Policy, Technical Director/Advisor, etc.)
-- Others as applicable (e.g., finance/cost, small business, contract management (DCMA) etc.)
The inclusion of peers from other bases, MAJCOMs, or other DoD Agencies provides a broader approach to the independent review process and facilitates the sharing of lessons-learned and best practices. IG 3.1.2.1
3.1.1. Independent Review Structure. The CAA must structure the independent review in a way that best achieves the objectives of the business and contract clearance process and tailored to the needs of the acquisition. The process must focus on reviewing CDPs in a thorough and substantive manner where SMEs are granted the level of access to acquisition/source selection documents necessary to provide the CAA with a clear assessment of the soundness of the approach and methodology employed. The CAA should ensure that the established review process encourages discussion among MIRT members concerning their observations and recommendations.
3.1.2. MIRT Work Product.
The MIRT will provide an assessment to the CAA on the state of the source selection/procurement. At a minimum, the MIRT will out brief the source selection/acquisition team at the conclusion of each CDP conducted. Documentation of MIRT reviews is required for the official contract file IAW paragraph 4. All MIRT and ACE comments must be adjudicated by the CAA. The SSA will be provided feedback either by the CAA or the source selection team.
3.2. Critical Decision Points (CDP). CDPs are precursor actions required to obtain approval for formal clearance events (e.g., to issue the solicitation, to award without discussions, to request final proposal revisions, or for the SSA source selection decision). These CDPs are conducted by the MIRT prior to obtaining the appropriate business or contract clearance from the CAA.
3.2.1 CDPs.
3.2.1.1 Pre-Business Clearance CDP.
3.2.1.2. Pre-Contract Clearance CDP.
-- Review draft Competitive Range Brief or review of draft Award without Discussions Brief (Review of these draft briefs may include review of supporting documentation and evaluation notices or interim ratings, etc.)
-- Review draft Final Proposal Revision (FPR) Request (Review of this draft brief may include review of pre-FPR brief including interim ratings after discussions, etc.)
-- Review draft Source Selection Decision briefing (Review of this draft briefing may include review of pre-source selection decision brief, PAR, etc.)
3.2.1.3. Areas of Special Interest (ASI). The CAA may require the MIRT to assess ASI in addition to the CDPs discussed in the previous paragraphs. The MIRT also has the discretion to review ASIs that may impact their assessment of the CDPs. ASIs may include review of source selection plans or debriefing charts/script, the conduct of mock debriefs, review of model contracts and all attachments, etc.
4. Documentation. Documentation that independent reviews were conducted IAW this procedure will be included in the contract file under the business and contract clearance tabs. Also include the actions taken to adjudicate the MIRT comments by the CAA. The documentation will be considered “Source Selection Information” and/or “For Official Use only” and will be handled IAW FAR 2.101 and 3.104.
5. Waiver. The CAA may waive the use of MIRTs in support of a specific clearance action for individual acquisitions or specific CDPs based on acquisition/source selection history and procurement/source selection experience of the acquisition team (i.e. recurring nature of the requirement, no history of sustained protest in a competitive acquisition, and successful experience with evaluation criteria in a competitive acquisition).
********************************************
INFORMATIONAL GUIDANCE
for
MP5301.9001(b)
3.1.1 - MIRT Other Considerations
1. Operational Considerations
a) Identification of content of required documentation such as briefing charts, record of reviews, adjudication/disposition of review comments;
b) Notification procedures for MIRT reviews;
c) Method of feedback to source selection team, to the CAA, and if applicable to the SSA;
d) Procedures for acquisition team and the MIRT to share and exchange information;
e) Appropriate forum to conduct the MIRT reviews i.e. roundtable, telecom, video conferencing, etc; or
f) Procedures for the source selection team to provide the CAA with its response to the findings of the MIRT.
2. Teaming/Membership Considerations
a) Whether to establish standing or ad hoc MIRTs or to assign members based on unique circumstances of each acquisition. Beware of the possibility that continuous membership may lead to members losing their independence or objectivity over time and getting pulled into the “group think” scenario;
b) Whether the team should be led by one individual or co-chaired; or
c) Whether to partner less experienced personnel with senior SME to foster OJT and mentoring.
3.2.1.3 The review process may include the following areas:
1. Pre-Business Clearance CDP
a) Pre-ASP
Source Selection Team Membership
-- Are there appropriate, knowledgeable, and qualified source selection team members, to include a sufficient number, assigned?
-- Did the briefers effectively articulate their decisions/justifications?
ASP Brief/Draft Acquisition Plan
-- Is the requirement captured by performance-based objectives?
-- Is the requirement defined to capture the minimum program objectives and can it be realistically proposed to?
-- Are the requirements documents (including operational capability requirements) translatable into appropriate, evaluative criteria?
-- Does the market research support the requirements, acquisition strategy and recommendations?
-- Does the risk analysis include contractors’ risk as well as government’s?
-- Does the risk analysis address and support the recommendations of contract type, pricing structure, type of source selection?
-- Were other contractual vehicles considered, e.g., GSA, multiple award contracts, other DoD contracts, etc.? Was a cost/benefit analysis performed?
-- Are there special contract requirements?
-- Are potential Organizational Conflict of Interest (OCI) issues addressed?
-- Are the milestones established by focusing on the key events and with realistic dates to accommodate accomplishing the actions required at the key events? (e.g., is the source selection schedule event driven and not schedule driven?)
-- Do the milestones include review/participation by a MIRT and OSD Peer Review, if required?
-- Does the briefing support the recommendations, e.g., type of contract, pricing structure, type of source selection, etc.
-- Is the acquisition strategy executable?
-- If using ASP as an acquisition plan – are there sufficient details, either in the brief or in the notes – that the ASP can be approved as the acquisition plan?
-- If there has been any consolidation of contracts, was the bundling and consolidation analysis performed prior to deciding on an acquisition strategy? Is the basis for bundling or consolidation justified?
-- If this is a single-award IDIQ, do any of the exceptions at 10 USC 2304a(d) apply?
b) RFP Sections L and M
-- Are the individual elements to be evaluated under each factor/subfactor stated in a clear, non-complex manner?
-- Are evaluation criteria appropriate (e.g., limited key discriminators based on risk analysis) in order to select the best value contractor?
-- Are cost considerations appropriate for strategy? If one of the criteria is to evaluate Most Probable Life Cycle Costs (MPLCC) this is a “red flag” – were lessons-learned from previous source selections where MPLCC was used discussed before recommending it as a criteria? Was required approval obtained? Was rationale and methodology thoroughly explained and executable?
-- If most probable cost (MPC) is being evaluated – is it only for the instant contract?
-- Was a side by side comparison of Sections L & M made to ensure how what is being asked for will be used in the evaluation process?
-- If using a trade off source selection approach, does the evaluation identify and prioritize specific trade space or performance thresholds for which the government is willing to pay more? Is it clear from the evaluation criteria what the offeror would provide that the government would constitute more advantageous?
-- Are minimum thresholds and maximum performance objectives clearly defined?
2. Pre-Contract Clearance CDP
a) Pre-Initial Evaluation/Pre-Competitive Range (Award w/o Discussions)/Pre-release of Interim Ratings
Evaluation
-- Did the evaluation team follow/comply with:
--- Sections L&M of the RFP?
--- Source Selection Plan?
-- Was there consistency and fairness in evaluating each offer against Section M (evaluation criteria)
-- Was there consistency in applying the ratings across the offerors?
-- What clarifications and/or communications, if any, were conducted?
--- Were they appropriate clarifications and/or communications?
--- Were meaningful and open communications conducted in which the offeror and government clearly understood each other’s position and assumptions? (It is not sufficient to handle communications in a way that is it only for the benefit of the government to understand the offeror’s proposal.)
-- Did the evaluation team apply any standards or undisclosed evaluation criteria to their evaluation of proposals or past performance?
Documentation
-- At a minimum, were deficiencies, weaknesses, significant weaknesses, and adverse past performance described in the evaluation notices? Were assumptions addressed as appropriate or acceptable?
-- Is supporting documentation describing details of the evaluations readily available during the briefing?
-- Does the supporting documentation fully, completely, and contemporaneously document the entire evaluation? Does the briefing accurately compare to the official documentation?
Briefing Charts
-- Does the briefing tell the story of proper evaluations from receipt of proposals, through the technical, past performance and cost evaluations, through the type of clarifications and communications conducted, through formations of evaluation notices, to the initial ratings/standings?
-- Does the content and quality of the briefing reflect Contracting officer and SSEB Chairperson(s) or equivalent review?
-- Does the briefing support the recommended competitive range (or to award without discussions)?
-- Did the briefers effectively articulate their decisions/justifications?
b) Pre-Final Proposal Revision Request Brief
Discussions
-- Were discussions conducted IAW FAR 15.306(d)?
-- Did the discussion process utilized achieve meaningful and transparent communications?
Documentation
-- Does the supporting documentation adequately describe the basis and justification for the ratings?
-- Do the Interim Ratings prior to final evaluation support the degree of discussions held with each offeror? (e.g. there is no misunderstanding by the offeror that the Contracting Officer clearly discussed the issues that resulted in the identification of significant weakness and/or any deficiency.)
-- Does the supporting documentation, fully, completely, and contemporaneously document the entire evaluation? Does the briefing accurately compare to the official documentation?
Briefing Charts
-- Does the content of the briefing bring the SSA up to date from approval to establish competitive range to completion of meaningful discussions (with or without releasing of Interim Ratings to the offeror)?
-- Does the briefing content confirm that meaningful discussions were conducted?
-- If interim ratings were not released prior to Final Proposal Revision request, has the Contracting Officer clearly articulated his/her rationale in the briefing charts?
-- If any offeror’s proposal should no longer be in the competitive range, does the briefing support the recommendation to eliminate it from consideration for award?
-- Did the briefers effectively articulate their decisions/justifications?
c) Pre-Source Selection Decision Brief
Documentation
-- Did the SSEB complete proposal evaluation, incorporate information provided through discussions and Final Proposal Revisions IAW section M?
-- Is the Proposal Analysis Report (PAR) reflective of the evaluations and justification documents?
-- If a SSAC was used, are the Comparative Analysis Report and recommendations complete?
-- Does the supporting documentation fully, completely, and contemporaneously document the entire evaluation? Does the briefing accurately compare to the official documentation?
Briefing Charts
-- Do the briefing charts clearly summarize and justify the evaluation results?
-- If Proposal Analysis Report is not used, does the briefing contain adequate details of the evaluation results and a comparative analysis (cost/price, past performance, technical risk, and a source selection recommendation) of the competitive offers?
-- Can the SSA make an integrated assessment best value decision based on the information presented in the briefing?
-- Can the information in the briefing be used, as applicable, for the source selection decision document?
-- Did the briefers effectively articulate their decisions/justifications?
3. Areas of Special Interest
a) Pre-Debriefs
-- If Sections L&M were complied with; if meaningful discussions were conducted with the offeror; and if offeror aware of its standings (via initial & interim ratings) throughout the evaluation process, the majority of the debriefing will focus on the final evaluation.
-- The content of the debriefing slides should be taken from those used for the offeror during the competitive range, interim ratings, and final proposal revision requests briefings. (The offeror should not be surprised with the content of this information during a debriefing.)
-- Purpose is to put the “debriefing” team in a mock situation to be able to handle offeror’s questions and comments.
-- Can the members of the debriefing team effectively articulate their findings and their justifications?
-- Does the debriefing focus on the government’s evaluation of the offeror’s proposals instead of the selected winner?
-- Will the debriefing assist/instruct the offeror on what it did well and what it could have done better?
********************************************
Note 1: Personnel participating as members of or advisors to source selection teams or procurement teams may not be an acceptable substitute for the independent review due to their lack of objectivity and independence. However, the CAA has the discretion to use “advisors” as members of a MIRT.
Note 3: In the event the contents of ASP do not align with FAR Part 7 and its supplements or an ASP is not held, review of written AP or equivalent documentation will be conducted.
Note 4: Applicable to Life Cycle Management Plan (LCMP), Acquisition Plan, and other similar documentation.
Note 5: References to RFP Sections L and M include equivalent solicitation provisions for the acquisition of commercial items.