ryan_b

Wiki Contributions

Comments

AMA: Jason Crawford, The Roots of Progress

Do you have any methods of analysis or threads of scholarship that you think are definitely wrong, or seriously misleading, and so should be avoided?

Distinguishing the impact and value of an idea

This is an interesting post, and the arguments make sense to me. Upvoted.

I did find one idea which is very popular in economics thinking that I want to push back on:

Some amount of time that the second group spent will have been duplicative and so wasted.

I claim none of the effort spent by the second group is wasted: all of the duplicative effort pays out as reduced time to understand (and therefore use) the discovery. In cases where two groups are very close, that amount of time is basically zero; in cases of multiple discovery it is actually zero. I strongly expect that having multiple groups with a good understanding of a discovery increases the likelihood of successfully getting to downstream discoveries, and I suspect it would disproportionately increase the likelihood of leapfrogging and branching into new areas as the multiple groups look to differentiate themselves.

Separately and not directly related to the post, I claim that situations of multiple discovery are the most valuable events for the study of progress, because they give us n>1 experiments in how much information is required to make the discovery in question. An example of what I mean here is that if the same thing was discovered three independent times, and we look at what each person or group knew when they made the discovery, then:

  • Stuff known by all groups tells us what is necessary
  • The group that knows the least stuff gives us an idea of what is sufficient
  • Stuff that one or more groups was wrong about can be either dismissed as irrelevant, or if another group had it right they could be compared to see what influence that part had on how far they got

The way the second point relates to the first is that I believe the analytical lens which looks at the efficiency of a single discovery - whence the duplicated effort is wasted idea - is fundamentally mistaken. A single discovery doesn't make sense to me as a unit of analysis for this because they are not independent; they depend on the discoveries that came before them and are in turn depended on for later discoveries. If we shift from the abstract discovery level to concrete ones like steps in the chain of producing products, this becomes much more stark: what sense does it make to compare the efficiency of an automated truck in a Uranium mine to the efficiency of an additive in paint manufacturing? In order for the efficiency numbers to make sense we need the context of the process of which they are a part.

Turning at last back to the actual subject of the post - that the value of a discovery by a person or group should be considered in light of the duplicated effort - feels to me like carrying the the same frame of analysis one step farther and applying it to the groups in the research process. If we want to identify which groups we should look to for lessons on progress (which I realize was not identified in the post) then it feels like my intuitions about this point in the opposite direction of yours.

The Progress Dashboard (Concept)

This is a cool project! I am going to spend some time poking around in there, and if I think of anything in the way of improvements I’ll let you know.

The Book of T: An Exegesis of The Greatest Paper in Economic History

This is a fantastic write-up, and I would like to see more like it.

I always enjoy seeing love lavished on important intermediate artifacts, like a particular paper, or particular tool, or particular storytelling technique. For papers in particular I would love to see piles and piles more highlights, walkthroughs, and appreciation of great papers across all fields.