By Nick Ansley, Partner, Wipfli
CECL is no longer new. Credit unions have implemented it, reported under it, and been through the audits and exams. But running the model successfully for a few years isn’t the same as having perfected it.
The question now is different: Not “are we compliant,” but “how well do we actually understand our own model?” A CECL estimate that hasn’t drawn examiner or auditor pushback isn’t necessarily a well-understood one. It may just mean no one has tested it yet.
The practices below aren’t about getting CECL off the ground. They’re the next steps: backtesting, sensitivity analysis, benchmarking, and validation. These are practices many credit unions still haven’t built into their process, whether their model is built in-house or licensed from a vendor. And outsourcing the model doesn’t outsource accountability for the results.
Backtesting compares your credit union’s past credit loss forecasts to what actually happened, so you can see whether your model is systematically over- or underestimating losses and, more importantly, whether you can explain why.
There are three types worth considering: outcome, assumption, and decision backtesting. Together, they answer three different questions. What happened? Why did it happen? And did we respond to it the right way?
You don’t need all three running at once, and they aren’t equally urgent. If your credit union hasn’t done any backtesting yet, start with outcome backtesting. It’s the most accessible entry point, and it doesn’t need to be sophisticated to be useful.
Outcome backtesting
Outcome backtesting is a retrospective comparison. Pull your ACL estimates from two to four years ago and compare them to the actual cumulative net charge-offs that followed. It’s a directional test, not a precision test. You’re looking for trends and magnitude, not an exact match.
Is the model consistently running high or low? By how much? And can you isolate why? If qualitative factor overlays were a major driver of a higher-than-actual reserve, that’s useful information. If you can’t explain the divergence at all, that’s a gap worth closing.
Outcome backtesting doesn’t need to be elaborate on day one, and it doesn’t need a fixed schedule right away either. Many credit unions start by running it once a year, often alongside their annual model review, and build from there as the process matures: Breaking results out by loan pool, extending the lookback period, tightening the definition of “actual” losses used in the comparison. The goal is to start somewhere and add sophistication over time, not to wait until you can do it perfectly.
Assumption backtesting
Where outcome backtesting looks at the final number, assumption backtesting looks at the judgments that produced it: the inputs you controlled, not the economic conditions you didn’t. It’s a logical next step once outcome backtesting is established, and you want to understand why a gap exists.
Pick a handful of assumptions that actually move the model, such as prepayment speeds or loss emergence timing, and compare what you assumed to what happened. Did your prepayment assumptions match actual experience? Did your qualitative factor adjustments move in the right direction relative to observable conditions?
Document what you observed, how it compares to your original assumptions, and a simple conclusion. A useful format is, “If our prepayment assumptions had matched actual experience, our ACL estimate would likely have been approximately $X lower.” That kind of statement is concrete, defensible, and demonstrates real model understanding, not just model operation.
Decision backtesting
Decision backtesting is the most strategic of the three. It’s also the least common. It asks whether the actions management took in response to CECL outputs were the right ones.
Your model isn’t just producing a reserve number. It’s informing decisions. If the model signaled rising losses over the next several quarters and leadership tightened credit standards in response, did that play out as expected? If it projected higher prepayment speeds and you staffed up for refinance volume, was that the right call?
This isn’t where most credit unions should start, and few will need to formalize it right away. But for institutions with a mature backtesting program already in place, it’s a natural extension.
Used together, these three types of backtesting do more than satisfy regulators. Outcome backtesting tells you whether your model is running high or low. Assumption backtesting tells you why. Decision backtesting tells you whether the actions you took because of that information were the right ones. Most credit unions get real value from stopping at the first two; the third is worth building toward once those are routine.
Sensitivity analysis means changing one key input at a time, such as prepayment and curtailment assumptions, loss emergence timing, forecast period length, or qualitative factor adjustments, and measuring how much it moves your ACL. If you move your projected unemployment rate from 4% to 5%, how much does your reserve change?
This is worth doing well, because it feeds directly into everything else on this list. If you know where your model is sensitive, you know where to focus your resources. The assumptions that move the needle most deserve the most rigorous support, the most documentation, and the closest attention from the board and examiner. The ones that barely move it don’t need the same level of scrutiny.
Sensitivity analysis should also shape your assumption backtesting. Use your sensitivity results to decide where to start. Test the assumptions that move your reserve the most, before spending time on the ones that barely matter. That keeps your review focused instead of spread evenly across everything, and it means you can walk your board and examiners through what’s actually driving your reserve, not just report the number.
Quarterly call reports are public information, which means you can pull data from credit unions similar in asset size, portfolio mix, and geography, and compare your ACL ratios to theirs.
Benchmarking doesn’t provide a target, and it doesn’t replace your model. But if your reserve is meaningfully higher or lower than your peers’, that’s worth digging into. The question isn’t just “why are we different?” It’s “can we explain why we’re different, and does that explanation hold up?”
Useful metrics for comparison include:
ACL as a percentage of total loans
ACL as a percentage of nonperforming loans
Delinquency and nonaccrual ratios
Charge-off coverage ratios
Choose your peer group carefully, and don’t cherry-pick institutions to justify a high or low reserve. If your numbers differ meaningfully from peers, document the reason and make sure it holds up.
Independent validation means bringing in a qualified third party, that’s independent of the ACL calculation and credit approval process, to evaluate your model’s assumptions, data inputs, process controls, governance and documentation.
There’s no single rule that dictates exactly how often a CECL model must be validated. That doesn’t make it optional. Model validation is a routine area of inquiry for both federal and state examiners, and given how central the ACL is to your financial statements, most examiners and auditors expect to see it happening on some defined, risk-based cadence, even without a rule that names the interval.
That makes the real question not “do we need to validate,” but “how often and how deep.” Not every validation should look the same. The right scope and frequency depend on the nature of the model and the risk it carries, not a one-size-fits-all schedule.
Higher-risk situations, such as black-box third-party models, complex PD/LGD or DCF methodologies, or heavy reliance on peer or macroeconomic data, generally warrant validation every 12 to 18 months.
Simpler, more stable models, such as an open-formula WARM calculation, can often run on an 18- to 36-month cycle.
The results of your own backtesting and sensitivity analysis are useful inputs here, too. A model showing instability in backtesting, or one that’s highly sensitive to a small number of assumptions, may warrant more frequent or more targeted validation than the calendar alone would suggest.
Scope should match the model just as much as frequency does. A WARM model’s validation will likely emphasize data integrity, assumption support, and governance documentation. A vendor model will typically need a deeper review of how peer and economic data are being applied and may call for loan-level recalculations. Before engaging a validation provider, make sure the proposed scope actually matches your model’s complexity and risk. There’s no reason to pay for procedures that don’t apply to your situation.
Have questions about strengthening your CECL process? Wipfli has a team that can help your credit union build a defensible, regulator-ready compliance program.
Connect with Wipfli to learn more.