Recent Discussion

Hi everyone. I'm Mark Khurana, a medical doctor and epidemiologist currently working as a research fellow at the University of Copenhagen, primarily in infectious disease epidemiology. I am the creator of the 'Untold Health' Substack and podcast, which explores topics within the field of health that are significant but overlooked. Also, I am the author of the upcoming book, The Trajectory of Discovery: What Determines the Rate and Direction of Medical Progress?, due to be released in the US in May 2023:

Medical research works in trajectories. Scientists and researchers must choose to pursue certain scientific pathways and omit others, limited by resources, attention, and time. The trajectory of medical progress is therefore characterized by two crucial characteristics: rate and direction. These two components form the foundation for

...
2delton13711hWhy are there so many medical studies using sloppy research methods, and how big a problem do you think this is? I noticed this when trying to figure out how common Long Covid is - most of the studies being reported in the media, at least early on, did not have a control group. On the basis of these studies, the media was saying that Long Covid affects 30, 50, or even 60% of people who get Covid. Many of the studies also use methods which suffer from responder bias like surveying online support groups. Studies which track cohorts over time and have a good control group find more modest figures like 10-15% of patients experiencing greater than expected symptoms at 3 months. However nearly all of these are retrospective studies which as I understand it are not as good as prospective studies. More recently a study [https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2799116] came out which does what should be done all along - it compares outcomes of Covid patients with patients who got symptomatic non-covid upper respiratory infection. They found more symptoms in the control group than the Covid group at 3 months. This calls into question whether Long Covid is actually an actual phenomena in its own right or just another iteration of post-viral illness / post-viral chronic fatigue syndrome (see Vinay Prasad's video [https://vinayprasadmdmph.substack.com/p/long-covid-vs-long-uri]). I wonder, if low quality studies can be so misleading, is it worth doing them at all? It seems to me we should be pooling resources to do more high quality studies rather than many low quality ones.
1M Khurana11hDefinitely an enormous issue! I’m not as familiar with the Long-COVID data, but the issue applies to a lot of other fields/areas. I argue in the book that one of the most prevalent issues is linked to how researchers are incentivized to publish quickly and often. This means that studies will often tend to be under-powered (i.e., too few participants to show the ‘right’ level of statistical certainty) because it’s time-consuming to include more patients and because it’s often more expensive. The result is, as you point out, that we’re inundated with studies that don’t show much of an effect size, or at least not enough to conclude anything meaningful. Ultimately, this wastes research resources on a systems level, because one big study would have sufficed for fewer resources overall. But because it’s a publishing game, researchers aren’t incentivized to collaborate as much as we’d like from a progress perspective. In the book, I call this ‘artificial progress’, where we think we’ve learnt something new about the world (through the publishing of these studies), but ultimately we’re just misleading ourselves and need to use even more resources to clarify studies that should have been clear from the outset. One could argue that it should ‘cost’ more for authors to submit under-powered studies to journals, since journals often accept their research despite the methodological flaws, and therefore authors aren’t penalized for this type of behavior. The journals might also prioritize interesting results over the study size being adequate – meaning that too many of these articles get published. Authors and journals ultimately both ‘win’ from this behavior. I think this issue of sloppy research methods is probably MUCH more prevalent than we think, but I haven’t been able to find reliable sources. In the book I talk about research misconduct and fraud, where some “studies suggest that the true rate of fraud among published studies lies somewhere between 0.01% and 0.4%.” I’d

That makes sense, thank you. 

"studies suggest that the true rate of fraud among published studies lies somewhere between 0.01% and 0.4%". Even 0.4% seems drastically too low - perhaps 10 times too low. I'd be curious to see the source for this claim. An analysis by Elizabeth Bik and others found problematic image duplication in 3.8% of studies. Some of that may have been accidental, but I suspect most were intentional fraud. If ~3.8% percent of papers have this one specific type of fraud, that suggests an even larger percentage contain fraud in genera... (read more)

1tandkott11hHow do you think the role of physicians would be affected by AI?

The Swiss Existential Risk Initiative (CHERI) is excited to announce the launch of its 2023 Research Fellowship, an 8-week in-person program focused on global catastrophic risk research. 

The fellowship will take place from July 3rd to August 27th, 2023, in the beautiful city of Geneva, Switzerland. This year, we are particularly eager to host fellows interested in exploring governance issues related to global catastrophic risk mitigation, given our close connections with policy organizations in the field. That being said, we also warmly welcome and anticipate having several fellows engaged in more technical research questions.

Apply here before April 16th, 11:59PM CET.

By participating in the CHERI Research Fellowship, you'll have the opportunity to:

  • Strengthen your research portfolio and test your fit for research in global catastrophic risks.
  • Receive mentorship from experienced researchers who will
...

“Consumerism” came up in my recent interview with Elle Griffin of The Post. Here’s what I had to say (off the cuff):

I have to admit, I’ve never 100% understood what “consumerism” is, or what it’s supposed to be. I have the general sense of what people are gesturing at, but it feels like a fake term to me. We’ve always been consumers, every living organism is a consumer. Humans, just like all animals, have always been consumers. It’s just that, the way it used to be, we didn’t consume very much. Now we’re more productive, we produce more, we consume more, we’re just doing the same thing, only more and better….

The term consumerism gets used as if consumption is something bad. I can understand that, people can

...

To see the other perspective, try replacing "consumption" with food and "consumerism" with obesity. We only have 1 earth (for the foreseeable future), and rampant consumerism leads to a very inefficient conversion from its resources to value.

Also, you can still be anti-consumerism while agreeing that the global south would ideally see higher consumption. Reducing obesity doesn't mean we shouldn't feed the starving.

[Excerpt from https://davidlang.substack.com/p/costs 
I am not the author. David Lang is.]

My commentary below:


What should science cost?

Hacking the economics of scientific equipment.

David Lang

"Direct costs dictate research directions.

For personnel costs, the influence is straightforward. Grant availability can push more principal investigators to undertake a problem or enable the hiring of additional post-doctoral researchers in a lab.

The costs of tools are just as important, but much more difficult to pin down. Scientific equipment is an essential part of discovery. New technologies are enablers of new ideas and perspectives, and vice versa. Throughout history and across disciplines — from telescopes to microscopes, synchronized clocks to automated genomic sequencers — technology sets the pace for knowledge and insight. It's personal for scientists, too. Access to cutting-edge tools can make or break careers by...

I would agree with you about growing costs for equipment in trendy "big science" (dark mater, hot fusion, gravity waves, accelerators), and I see this trend in military domain, like in IT in old-fashioned companies, like in nuclear industry... It is aggravated by a growing increase of regulation.
It seems that some domains push providers to improves performances to the point nobody can buy the product, but it is really perfect. I've heard that for nuclear reactors (they can resist to anything, but nobody can afford them), for tanks (they are smart, agile, p... (read more)

1David Lang2dThanks for posting and considering. I agree. It would be great if more people researched this. A simple way to start would be to study the open science hardware momentum. A few anecdotes from that scene over the past decade: OpenROV (us) — made ROV prices >10X cheaper Open qPCR (Chai Bio) — made qPCR multiple X cheaper OpenTrons — made liquid handling multiple X cheaper Cubesats (not really OSH, but similar idea) — made satellites 10x cheaper They are all orders of magnitude more affordable, and many have completely rearranged who uses the tool, sometimes opening up big new markets. The lack of demand pull assumption should be tested. -David

As a Torontonian, when I visit East Asian cities metropolises, I envy the superiority of their metro systems, including:

  • More comprehensive subway lines
  • Trains with automated signalling that rarely break down/are late
  • Platform screen doors that make queuing easier
  • Signs by the screen doors that tell me when to expect the next train
  • Fares based on distance travelled
  • The ability to pay my fare on my phone (or by credit card)
  • Screens in the trains that tell me what side I will be exiting
  • Wifi and cell service available in the trains

Despite the envy, I can accept all of these deficiencies because on some level, they relate to infrastructure and building, something that my society has lost the ability to do effectively — while tragic, I have accepted.

What I find much more troubling are the...

Have you considered printing off a few sheets of paper, getting some glue, and just adding a few signs yourself? ;-)

Do you think fast progress is dangerous? Use the seat belt test. 

What if the seat belt had been invented ten years earlier? If progress in vehicle safety had been faster, thousands of lives would have been saved.

Some tech, like seatbelts, are almost pure good. Some techs, like nukes are almost pure bad. Some, like cars, we might want to wait until we develop seatbelts and traffic lights for before we use widely. It depends on the technology. 

To get the best posts emailed to you, create an account!
Subscribe to Curated posts
Log In Reset Password
...or continue with

Summary: I’ll be teaching the sixth and seventh cohorts of The Foundations of New York, a seven-week class to give New Yorkers the history, political theory, and political strategy they need to conduct the politics of progress. Applications ARE OPEN through April 12, but seats are limited—applying sooner is better. The next class will likely not be offered until September.

Summary details:

  • Cohorts 6 and 7 will run concurrently, but meet separately.
  • Cohort 6 will meet Sunday afternoons, 2-4pm, a three-minute walk from the Montrose L stop in Brooklyn. Class will run from April 16—May 28. June 4 will be a make-up day if needed.
  • Cohort 7 will meet on a weeknight evening from 6:30-8:30pm, on the Upper East Side. The meeting night will be finalized soon, and in accordance with
...

Something a little bit different today. I’ll tie it in to progress, I promise.

I keep noticing a particular epistemic pitfall (not exactly a “fallacy”), and a corresponding epistemic virtue that avoids it. I want to call this out and give it a name.

The virtue is: identifying the correct scope for a phenomenon you are trying to explain, and checking that the scope of any proposed cause matches the scope of the effect.

Let me illustrate this virtue with some examples of the pitfall that it avoids.

Geography

A common mistake among Americans is to take a statistical trend in the US, such as the decline in violent crime in the 1990s, and then hypothesize a US-specific cause, without checking to see whether other countries show the same trend. (The crime...

Great points. In (good) science, scope matching is one of the most important concerns. I've always wondered why it doesn't have a (widely used) name.

Scope matching failures really do come up constantly in modern criticisms of new technologies, whether it's social media or AI. Probably happened centuries ago too

For anyone who's interested: I'll be teaching the next cohorts of The Foundations of New York soon! It's an accelerated introduction into NYC government/law that also touches on dependencies at the state and federal levels. Class begins in mid-April and goes through May.

https://maximumnewyork.substack.com/p/the-foundations-of-new-york-applications