All of M Khurana's Comments + Replies

AMA: Mark Khurana, author of The Trajectory of Discovery, creator of Untold Health

Definitely an enormous issue! I’m not as familiar with the Long-COVID data, but the issue applies to a lot of other fields/areas.

I argue in the book that one of the most prevalent issues is linked to how researchers are incentivized to publish quickly and often. This means that studies will often tend to be under-powered (i.e., too few participants to show the ‘right’ level of statistical certainty) because it’s time-consuming to include more patients and because it’s often more expensive. The result is, as you point out, that we’re inundated with studies ... (read more)

1delton1372yThat makes sense, thank you. "studies suggest that the true rate of fraud among published studies lies somewhere between 0.01% and 0.4%". Even 0.4% seems drastically too low - perhaps 10 times too low. I'd be curious to see the source for this claim. An analysis by Elizabeth Bik [https://journals.asm.org/doi/10.1128/mBio.00809-16] and others found problematic image duplication in 3.8% of studies. Some of that may have been accidental, but I suspect most were intentional fraud. If ~3.8% percent of papers have this one specific type of fraud, that suggests an even larger percentage contain fraud in general. It's extremely hard to know, though. I doubt it's over 10% but I could easily see it being 5%, which is obviously still a massive problem.
AMA: Mark Khurana, author of The Trajectory of Discovery, creator of Untold Health

Good question! I think we seem to be going at a steady pace, but that depends on who you ask. Ultimately, it probably depends on what your expectations are of progress; my hunch is that people have higher expectations for cancer than for other diseases, particularly since it’s received so much attention historically, and that sets us up for inevitable failure when these expectations fail to materialize – much like the ‘war on cancer’ in the latter part of the 1900’s.

Broadly speaking, one can approach this from a treatment perspective and a prevention persp... (read more)

AMA: Mark Khurana, author of The Trajectory of Discovery, creator of Untold Health

Speed of information transfer: There’s good reason to believe that social media rapidly increases the speed at which science can be disseminated. Ideally, this increases the rate of medical discovery by a) making us aware of what others are doing and that we can build on and b) exposing us to alternative approaches and methods from other disciplines that we can integrate into our own work. I’ve certainly benefitted greatly from being exposed to ‘random’ articles from other fields.

Epistemic disorientation: In contrast to the first point, there are potential... (read more)

AMA: Mark Khurana, author of The Trajectory of Discovery, creator of Untold Health

Two chapters spring to mind - for different reasons!

The very first chapter, “Citations as Currency", was probably the most fun to write, mostly because friends and colleagues that have read it can identify with the themes. The chapter is concerned with how researchers attempt to accumulate ‘scientific capital’ by publishing papers and getting citations, but this ends up distorting the types of research projects we choose to pursue. I enjoyed the colleagues telling me: “yes, this is exactly how I feel!” – validating that this really is an issue.

Chapter 15, ... (read more)

AMA: Mark Khurana, author of The Trajectory of Discovery, creator of Untold Health

A few different issues! I’ll preface my answer by saying that there is certainly some evidence that ‘good ideas are becoming harder to find’, meaning that the marginal effort required to discover a new drug is increasing. This isn’t an excuse for the pharmaceutical industry, but it is worth noting.

Structurally, large pharmaceutical companies take too few risks during drug development, meaning that the onus is on smaller companies and universities to develop novel products. Why so? Well in more recent times, large pharma companies have essentially offloaded... (read more)

AMA: Matt Clancy, Open Philanthropy

If you were given a million dollar grant, which question in innovation policy would you want to answer and what might that dataset look like?

2mattclancy2yI will cheat and list two ideas, plus one irrelevant one. First, let's start with some large-scale descriptive statistics of practices! 1. How do different labs organize and manage their staff? 2. How do different editors manage their journal and peer review process? 3. How do different grant reviewers pick grants? The goal is to see if there are obvious best practices; what kinds of stuff is correlated with good outcomes? This would be some kind of large-scale survey. Second, I've always wanted to know more about the political economy of R&D. How do governments decide how much to spend on R&D? I'm not sure the best way to study this though. Third, my dream research project, after we have sorted out the more important stuff, is to use the decadal Sight and Sound Greatest Movies of All Time poll to study how people's perception of artistic greatness changes over time [https://twitter.com/mattsclancy/status/1605728149051801601?s=20&t=zfIMQCBG3-O1SAVZwNCaQg] . Do people change their minds? Or is it all about new (younger) critics embracing newer films?