
85: Can Borrowing Ever Increase Power?
In this episode of "In the Interim…", Dr. Kert Viele, Director of Research at Berry Consultants, takes up a requested topic: published results showing that Bayesian borrowing of historical control data cannot increase power when type I error is controlled. The key is the definition of type 1 error control. Borrowing methods can increase power and lower the probability of false positive conclusions when the historical data is near the current trial parameters, but not when the historical data and current trial begin to separate. If “type 1 error control” means controlling the worst case upper bound on the chance of a false positive conclusion, borrowing indeed can offer no benefit. In a world where all past data was generated to deceive you (the worst case), you shouldn’t use the past data. But, if type 1 error control means the actual probability of a false positive conclusion, borrowing can produce benefits in power and type 1 error commensurate to the match between history and the present trial. As such, these papers revisit an old tension between minimax methods and expected value approaches. When data is informative and when we borrow from historical situations most like our current trial, we can be optimistic the historical data will lead to better inferences.
Key Highlights
Why we naturally want to use past data in our current trials.
What does type 1 error protect against? Why we can’t directly compute this probability from a frequentist perspective.
Type I error as a worst-case upper bound.
Why borrowing cannot gain power under standard type I error control.
The fundamental tradeoff of historical data, with lower type I error and higher power where history matches, increased error rates where it does not.
A review of FDA guidance on considerations for historical data, including recency, comparable sites, and matching inclusion and exclusion criteria.
Minimax rules versus expected value, and the cost of guarding against unlikely parameter values.
