Why does it matter if drug development is inherently unpredictable and not simply high risk?
The question strikes many in an industry with an end-to-end success rate of between 2% and 3% as (a) obvious and (b) irrelevant. Yet I would argue that the failure to distinguish between high and inherently unpredictable risk is a principal cause of pharma’s potentially catastrophic decline in productivity .
Many of the best practitioners persist in operating as if drug development were simply high-risk–with lamentable consequences. Through skill or scale they believe that they can tip the odds in their favor. More insight and effort, new tools, smarter researchers—e.g. someone who had a blockbuster drug approved in the past–will produce the growth the industry needs.
They are wrong. After 40 years, the innovation crisis shows no signs of abating, despite remarkable advances in both knowledge and technology.
What evidence is there that drug development is inherently unpredictable?
The performance of a business is measured by return-on-investment (ROI), sales minus costs. Returns in pharma are approaching zero and headed negative. Investors provide capital in the expectation of returns. The probability of achieving those returns is risk. To understand risk in an industry, one must look not only at the magnitude of uncertainty but its nature. Different types of risk call for different operating strategies.
The height of the bars, (Y-axis), corresponds with the probability (frequency) that a drug will produce the level of sales shown on the X-axis. Tall bars indicate lots of drugs with a given level of sales; short bars, very few. The PDF of pharma sales shows large quantities of poorly selling drugs on the left and a few blockbusters in the long right tail.
The shape of the PDF of drug sales reflects the nature of the underlying process that generated them. Here is an abbreviated interpretation of the above chart by Dr. Arthur De Vany, Professor Emeritus of Economics, University of California Irvine:
The shape of the histogram [bar chart] for 2012 sales (Figure 1) shows [that]…the data are clustered to the left – far from the sample average which stands at $ 1.06 billion – and skewed to the right with a “heavy tail”, [meaning that there are more blockbusters than one would expect from a “normal” or gaussian distribution—e.g. a “normal” PDF of people’s height would include roughly equal numbers of tall and short on each side of the average, but no 75-footers, the human equivalent of a blockbuster.]… An analysis of the upper [right] tail shows that the top 10 drugs (3.2% of population) collected 21% of sales and the top 20 collected 35%, underscoring the critical importance of the upper tail (blockbusters) to the performance of pharmaceutical companies.
Blockbusters exert an out-sized effect on industry returns:
Compare average [expected returns in a normal system] and median [mid-point on the X-axis with equal numbers of drugs above and below] sales. In 2012, they were $ 1.06 billion vs. $ 546 million respectively. Three-fourths of sales were below the average, which underscores the fact that the “expected returns” are distorted by a few large outliers and not representative of the population.
Other years show the same shape, with many small drugs and an extreme skew to the right (figure 2).
The shape of the PDF of drug sales has been remarkably stable over the last three decades. Blockbusters provide most of the value in the industry.
With the average cost of bringing a new drug to market in 2014 running at $ 2.6 billion, more than twice the average or expected return, pharma is not going to solve its productivity problem by producing more “average” drugs. Without the possibility of blockbusters, the pharmaceutical industry could not afford to develop new drugs.
But why is the occurrence of these rare-event products inherently unpredictable, rather than simply low-probability, a problem that could theoretically be managed by scaling-up?
Examining the PDFs of sales for the years 1986 to 2012 in Figure 2, Dr. De Vany focuses on a parameter called alpha (α), which is a measure of the “skewness” of the chart and has specific implications for the expected (average) returns and the probability (risk) of achieving them. He observes,
For each year in our data set, α < 1, which implies that pharmaceutical sales have an indeterminate mean [expected return] and an indeterminate variance [level of risk]. This reflects the inherent unpredictability of drug sales and explains the impossibility of forecasting them.
To understand generally the implications of a PDF, scholars look for the best match among a selection of well-characterized standard curves, as Dr. De Vany demonstrates below.
For comparison purposes, …we look at three different reference curves for best fit—normal (gaussian), lognormal and Pareto… The normal distribution is a poor fit. The lognormal and stable [Pareto] are much better fits but differ in important ways.
The best overall fit is the lognormal curve, which would reflect a low-probability, as opposed to an unpredictable, process. However, as explained above, the portion of interest to drug developers is the upper tail or far right section of the curve where blockbusters are found. Here the best fit is one called a Stable Pareto curve.
By amplifying the all-important upper tail, Dr. De Vany shows the critical difference between the lognormal curve (best overall fit) and the Pareto curve (best fit for the upper the tail).
Magnifying the tail end of the curves (Figure 4) shows that:
• The normal curve (blue) bottoms out at about zero around $ 6.5 billion, with virtually no probability of larger values
• The lognormal curve (green) plunges underneath the stable distribution (red), which means that it has a slimmer tail, that allows for fewer blockbusters
Although the lognormal distribution is a reasonable fit for the data, it is not an appropriate model for pharmaceutical sales. …While a lognormal process converges toward the lognormal distribution as the sample size increases, under the Stable Paretian distribution, the scale of deviations [uncertainty] of drug sales continues to grow with sample size, which is consistent with the data…
The Stable Paretian distribution has important implications for understanding pharmaceutical market dynamics. Scale and diversification do not improve the probability of success [i.e. returns from larger portfolios are less certain—riskier—than those from smaller portfolios, though in a good way, since the uncertainty is caused by the higher probability of a blockbuster], and therefore cannot be used to mitigate risk. It is impossible to forecast drug sales from past sales, which are a function of portfolio size and are disproportionately influenced by outliers…
The drug developer’s dilemma: To increase productivity, he doubles his portfolio and his costs. Since blockbusters are extremely rare, it is not likely that either the old or the new halves, taken separately, contain one. When combined, the average or expected return increases slightly, because the probability of finding a blockbuster in the larger portfolio goes up, but the gain is small relative to the step-up in costs. The increased possibility of a blockbuster brings greater uncertainty, because their occurrence is rare and unpredictable. Expenses have gone up substantially; expected returns increased slightly; and the outcome is less certain. Pharmaceutical portfolios do not scale up in a “normal” or predictable manner.
Dr. De Vany summarizes,
• Blockbusters dominate revenues.
• Most drugs sell modest amounts…
• Rare and unpredictable, blockbusters cause the mean and variance of drug portfolios to fluctuate so much that a stable mean [expected return] does not exist and the variance [uncertainty, risk] is infinite
• The sales distribution of drug portfolios indicates that scale cannot be used to mitigate risk. The bigger the portfolio, the higher the risk.
• The focus of pharmaceutical development plans should be to maximize the likelihood of discovering blockbuster (upper tail) drugs and minimize the cost of mediocrity.
If blockbusters cannot be predicted regardless of portfolio scale, how can pharma increase productivity?
Dr. De Vany advises the industry to align the operating model with the nature of the risk.
Another important consequence [of unpredictability] is that the size and long-term growth of the firm [and the industry] is directly related to size of its portfolio, since the larger it is, the greater the odds of having [blockbusters]… Growth cannot be achieved by other means, such as increasing the proportion of blockbusters in the product mix in any predictable way.
High costs and increasing uncertainty prevent drug makers from increasing the size of their portfolios. Since they can’t scale-up operations, researchers have focused on improving the efficiency of the process, relying on knowledge and technology to find more blockbusters. It hasn’t worked, because they are implicitly assuming that drug development is a “normal,” high-risk process. In fact, outcomes are inherently unknowable, regardless of how skilled or well-equipped the drug developer. Managers have structured the industry for high-quality, high-cost production in the expectation of “increasing the proportion of blockbusters in the product mix in …[a] predictable way,” a strategy at odds with the nature of pharmaceutical risk.
The only way to find more blockbusters is to test more drugs, and the only way to do that is to reduce costs until they align with expected gains (modest) in sales and increased uncertainty. That requires changing the high-cost, limited-production business model to one that places equal importance on quality and quantity.
Inherently uncertain means inherently expensive. Whether pharma can reduce costs sufficiently to solve its innovation crisis remains to be seen. Society might simply decide that it cannot afford new drugs and impose price controls, accelerating the decline of already failing profitability. In the meantime, pharma must try new “quantity” models. Any improvement in costs and scale will increase the likelihood of survival for both the industry and patients.
This blog-post was largely extracted from unpublished research I did several years ago with Bernard Munos of Innothink/FasterCures and Dr. Arthur De Vany of De Vany Consulting, Professor Emeritus of Economics, University of California Irvine. The impetus for the study came from Munos and the technical analysis from Dr. De Vany. Since then, Dr. Kenneth Fernald of Evolvalor has confirmed many of the findings. All the data was provided by EvaluatePharma. Any errors in the presentation are mine.