Common Pay for Success contracting pitfalls and how to avoid them
by Grace Edinger & Dr. Timothy Male
Pay for Success (PFS) contracts are becoming more common for environmental projects, particularly in ecological restoration. These contracts are structured such that payment is based on successful delivery of outcomes (acres of stream restored, amount of nutrients reduced, etc.). From a public perspective, this allows the government to get fair prices from bidders on outcomes that matter to taxpayers. From a bidder perspective, it gives businesses and non-profits more opportunity to innovate in where and how they do the work as long as they deliver the agreed upon result.
However, because this is very different from traditional contracts for a set of services and tasks, drafting a Request for Proposals (RFP) and deciding on specific contract language can be difficult for government agencies, especially if this is the first time they are using Pay for Success. If done properly, PFS can be a fantastic way for government money to be spent more effectively. If done wrong, it can create tension, lack of interest from potential awardees, and ineffective projects.
Montgomery County, Maryland, has recent experience with a Pay for Success RFP gone wrong and - hopefully - a new one going right.
Associated with these RFPs, we interviewed multiple for-profit restoration firms who have extensive experience with responding to government PFS RFPs. In the interviews, we dug into what works, what doesn’t, and what are their major red flags when deciding to submit a bid.
Montgomery County issued an RFP that did not result in an award in 2018. The original motivation for carrying out these interviews was that we had heard the opinion expressed that the RFP failed because no businesses were interested in doing outcome-focused work. We found that the absence of bids had nothing to do with business interest - instead there were requirements in the RFP that made participation enormously risky and potentially costly to experienced restoration professionals.
In 2022, the county restructured that RFP to address many of the problems with the 2018 RFP and is currently in the process of selecting winning bidders.
This restructure presented a unique example to compare concrete language from both project solicitations to explore interviewees’ interests/opinions on each RFP. The new solicitation was very clearly an improvement on the old one in a number of ways (see below), and also included creative provisions like allowing projects that delivered more water quality benefits to be paid more for them (at the same unit rate).
Based on the difference between these RFPs and our experience with other Pay for Success procurement efforts, there are at least 4 major factors in a successful RFP that a government agency might underappreciate:
Allow contractors to determine their own sites within an ecologically-relevant area rather than having government predetermine exactly where projects must go
Structure RFP evaluation criteria to be clear and adequately weigh project cost
Structure performance triggers and payment schedules so they are truly dependent on outcome delivery, not process milestones
Minimize bonding requirements if no government capital is at risk until project is completed
| Old RFP | New RFP | |
|---|---|---|
| Overall Structure | Would issue 1 contract with multiple projects | Would issue multiple contracts for separate projects |
| Site Selection | Must include at least 10 projects from predetermined sites included in the RFP | Can choose project sites |
| Evaluation Criteria | ● Two, 500 point phases ● Somewhat ambiguous categories ● Project cost = 20% of criteria | ● Out of 100 points total ● Clearer categories ●Project cost = 35% of criteria ● Bonus for projects located in minority and low income communities |
| Budget Structures & Payment Triggers | ● Hybrid of outcomes-based and traditional procurement based on timelines ● Requires itemized billing for each project | ● 100% of payment comes after full delivery of outcomes ● Requires only total project cost and number of outcome units |
| Bonding Requirements | $20 million bonded over the course of the project | Bonding requirements removed (due to change to outcomes approach) |
Predetermined Site Selection
The old RFP required the contractor to include 10 sites from a predetermined list provided by the agency. The list includes 38 sites, with varying degrees of planning completion.
All interviewees told us that this requirement was a nonstarter. Restoration providers felt that choosing sites themselves was needed in order to take on the risk involved with Pay for Success contracting. They felt that there was a reason the projects on the County’s list were not completed previously, thinking they presented particularly difficult challenges.
One interviewee stated, “From our point of view, they were trying to take sites that weren’t able to be successfully implemented previously. We didn’t have confidence that those sites would be successful moving forward.”
The County recognized this was an issue, and has removed this predetermined site selection language from its 2022 RFP.
RFP Evaluation Criteria & Structure
The original evaluation criteria and scoring matrices were a point of confusion for interviewees. Scoring was broken into two phases each with a 500 point total. The first round score was based on the written proposal, and applications that scored 350 points or higher would move onto a verbal interview.
Over half of the written score was based on 5 ambiguous categories that all revolve around how the proposal meets the objectives and qualifications listed in the RFP (320/500). Only 100/500 points were based on the total project cost. Pay for Success contracts are effective because competition drives down cost; this evaluation structure deemphasizes it to the point where a score of 0/100 in the cost category could still have an applicant moving to the interview round.
Interview scoring criteria reflected the same priorities as the written scoring. 350/500 were again dedicated to how well the proposal meets vague objectives and team experience. Project cost also represented 100/500 total.
This is likely to be resolved in the revised iteration. Scoring has been simplified to be out of 100 points for each phase, cost has been bumped up to 35% of the available points, and categories are presented more clearly. We also applaud the prioritization for projects located in minority and low income communities in the new RFP criteria.
Going one step further, multiple interviewees suggested that an RFQ (Request for Qualifications), rather than an RFP, would be more appropriate for any government agency using Pay for Success for the first time. An RFQ outlines large goals and objectives, including aspects like desired socio-economic co-benefits, fixed pricing, and credit outcomes. Specific projects and tasks are left up to the bidders, who respond by describing their qualifications, and explaining the kinds of conditions the government would need to set in place for them to carry out the desired work. This should be considered as an option in future contracting opportunities.
Budget Structure & Payment Triggers
The original contract was structured as a hybrid of traditional, time-based billing and Pay for Success. The contractor would receive payment based on set milestones if completed on time (annually). Upon completing the first year of the contract, the contractor needed to have a certain number of acres designed, meaning “a project that is technically complete, has all required Permits and approvals, has all required easements, and is ready to start construction.” Following years were similarly structured, with an increase in the number of designed acres as well as construction acres which were “project[s] which ha[ve] all physical work completed, all Permits closed out, and all documents, including As-Built plans, submitted to the [community]”.
The 2018 RFP referenced budgeting constraints and needed to pay out the contract in a consistent manner throughout the five years, which was likely the reasoning for the hybrid milestone-time structure. However, we heard that the associated reporting and annual milestones created undue strain on the contractor who knew their work wasn’t going to happen in identical one-year increments.
One of the most fundamental flaws that we have seen repeated in other Pay for Success attempts by agencies is a requirement that contractors itemize all costs. In the 2018 RFP contractors were required to create an itemized budget for each project for the first year. The RFP stated that they would be held to this budget. Our interviewees saw this as contrary to the whole value of Pay for Success contracting. The point of these contracts is that bidders can expect a predetermined payment, regardless of whether their costs are higher or lower than that payment amount. If agencies are worried about paying too much for an outcome in one year, that problem is likely to fix itself in future years’ contracting (which has been the experience of Anne Arundel County, Maryland, which has the longest running environmental Pay for Success program in the country).
Bonding Requirements
As written in the original RFP, a total of $20 million (broken into two-$10 million pieces) was to be bonded for the duration of the project. With a total estimated contract cost of $35-40 million, the county was asking the contractor to bond roughly half of the entire project.
Interviewees representing smaller, more localized restoration firms expressed that this was not feasible for them. This language has since been revised, removing this high bonding requirement, making smaller firms eligible to submit a bid.