Throwing Darts

A recent study by analysts from the management consulting company, McKinsey & Co., and published in Nature Reviews Drug Discovery (Cha et al. 2013) attached numbers to a skeleton that many in the biopharma industry keep in the closet:  the accuracy of forecasting future sales of new drugs is poor, so poor they may be useless.  The authors studied forecasts on 260 drugs launched 2002-2011 (1700 in all) made by sell-side analysts (analysts for brokerage firms whose livelihood is advising clients on what stocks to buy).  They compared the consensus forecast for a drug (an average of the forecasts of up to six analysts) with its actual reported sales for a period from two years before launch to seven years after and found a wide distribution of variance, both over- and under-estimates (+160 to -80%), with more than 60% of the forecasts being either over or under by more than 40% of the actual revenues.  The authors also found that, while the accuracy of the forecasts increased as a drug progressed through trials and onto the market, the sales forecasts were still unreliable, decreasing from being off by 75% prelaunch to 40% or so three to six years after launch.  Not so surprising, they noted that the estimates made for drugs developed by big pharma were more accurate than those for smaller companies (the former being underestimated prelaunch and the latter always enthusiastically over-estimated).

I found these findings rather astounding given that the analysts presumably are working from the same data (and post-launch, actual sales information) and have access to their fellow analysts’ forecasts and given the authors excluded from study 53 “outlier” forecasts that had over-estimates of more than 160%.  And what was not considered by the McKinsey team, but would have increased the under-estimate side, were the approximately half of drugs that fail in Phase III clinical trials.  These likely had prelaunch forecasts and, of course, revenues of zero; for a list of five such failures for this year, see the FierceBiotech special report.  But then I’m not really astonished since I had seen a report last April by EP Vantage, a division of the bioindustry data and analysis firm, Evaluate Pharma (Evaluate Pharma), and had read a fair number of market forecasts while working at Wyeth.  The EP report (accessed by an EP guest login) found that the revenue forecasts for ten drugs predicted to be blockbusters (annual sales more than $800 million) were off by 60-96% lower and for another ten, 3-183% higher.

So forecasting sales is difficult and the results suspect; why does this matter?  As noted in the McKinsey paper and by industry commenters (e.g., David Shaywitz in his columns in Forbes, e.g., Forecasts are Fragile), forecasts are critical to decision-making in the pharma industry especially because it has long product development cycles and high costs.  These data are used by companies to guide decisions on what basic research programs to pursue and which to jettison, the design of clinical trials, the deployment and size of sales forces, and what assets to license-in or companies to buy and at what price.  Early-stage forecasts are made and used by equity/venture investors since their returns upon exit depend on the potential value of their investments.  As noted by Mr. Shaywitz, forecasts of future revenue is central to companies:  “Forecasting enables an entire methodology, a structure, an approach to thinking about your business.”

So, what’s to be done about unreliable forecasting?  Mr. Shaywitz advised that companies focus on opportunities where the potential upside vastly outweighs the downside (like a cure for cancer) and on many opportunities that can be pursued cheaply and discarded early (the Silicon Valley approach).  The McKinsey authors noted the question is unanswered and suggested improving forecasting methodology through internal assessments and consideration of unforeseen circumstances like adverse clinical data or new competition (noting that some big pharmas make better forecasts than others).  They also said “beware the wisdom of the crowd” since well-compensated and experienced professionals are often wrong.

So, what’s this got to do with global health?  The wisdom of the biopharma crowd, the conventional wisdom, is that the potential revenues for drugs and other products aimed at the non-major markets are too low, and will remain too low, to justify investing in their development.  Moreover, the analysts say that there are too many unquantified risk factors, like whether a government is going to come up with the funds to pay for a needed drug, to make meaningful estimates.  But then to me the factors, data, and methods they use for quantifying risk for major market drugs looks wanting.  There is no denying that the major markets are where the money is now so it’s easier to assume a new drug will be profitable.  A report last year by the IMS Institute for Healthcare Informatics, a not-for-profit division of the pharma industry data company, IMS Health, noted that 63% of the about $1 trillion spend on medicines is in the major markets (US, most of the EU, Canada, and Japan), 20% in the “pharmerging” countries (China, Brazil, India, Russia, Mexico, Turkey, Poland, Venezuela, Argentina, Indonesia, South Africa, Thailand, Romania, Egypt, Ukraine, Pakistan and Vietnam), and 7% in the rest of the world (ROW) (Medicines Outlook Through 2016).  With cost containment in the major markets and increasing wealth in the others, by 2016 the IMS put the pharmerging share increasing to 30% and the ROW share to 8%, or about $460 million or $10 to $100 per person.

Of course, the IMS’s data is on sales of existing drugs and not the value of existing drugs deployed in new markets or those developed expressly for diseases and conditions existing in those markets.  I posit that the industry analysts, venture capitalists, and company market researchers should be throwing their darts at these markets, too, and making the results public to rewrite the conventional wisdom and encourage investment.  I’ve found few attempts to forecast revenues for these types of products.  Several years ago, BIO Ventures for Global Health published a report on the tuberculosis vaccine market (BVGH reports) before the organization got off its track of industry proselytizing and wandered off into not-for-profit land.  More recently, as I wrote about in my post, “The Price is Right”, Dow and Mora 2012 estimated the market for a new drug to treat dengue fever by estimating its economic burden in individual countries and making the not-illogical leap that a country’s government (possibly with donor aid) would be willing to pay 50% of that cost for an efficacious new drug.  In their model, they also accounted for potential competition from a successful vaccine of which several are in the works.  I suspect there are other market studies for specific existing or hypothetical drugs needed in the non-major markets, or, at least, there should be given the billions that are spent on drug development (including $4 billion per year on drugs for the neglected diseases).  My suggestion to analysts looking for new clients and VC firms and bio-pharmaceutical companies looking for new revenues:  make the dart board bigger and don’t expect to hit the bullseye.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s