All of jasoncrawford's Comments + Replies

Tyler Cowen did something like this for covid: https://marginalrevolution.com/marginalrevolution/2020/03/1-million-plus-in-emergent-ventures-prizes-for-coronavirus-work.html 

I liked Vitalik's post and generally agree

OK, sorry!

Good point, I agree! Something important to creating the right research lab team and culture.

Hmm, I don't agree with how you are characterizing my assumptions about human nature. I'm not assuming that scientists are after money or prestige. I assume most of them, or at least the best of them, are motivated by curiosity, the desire to discover and to know, and the value of scientific knowledge for humanity.

Re accountability, I frankly think we could do with a bit less of it. Accountability is always in tension with research freedom.

Re people performing for their superiors: I actually think scientists performing for their managers would be a much healthier model than what we have today, which is scientists performing for their grant committees. I have another piece on this that I plan to publish soon.

Curiosity is already a very strong motivator, we just need to enable it and get out of the way. Give scientists funding without making them narrowly constrain their goals, dial down their ambition, or spend half their time writing grants. Then give them the research freedom to pursue that curiosity wherever it leads. It's not easy but it is pretty simple.

Good question, I don't know. People have been talking about “progress studies” or the “progress movement” or “progress community”, and others have talked about the “abundance agenda”, but none of those lend themselves to personal labels/identities…

I have never particularly liked the term “techno-optimism” anyway. “Optimism” on its own is confusing enough. “Techno-optimism” implies that not only do you think we can solve all problems, but that technology will be the solution to all of them, which is not really true.

Thanks, good point about the flow here.

Thanks Robert. I think progress studies needs a more well-defined value system. I have gestured at “humanism” as the basis for this, but it needs much more.

I agree that Rand's ideas are important here, particularly her view of creative/productive work as a noble activity and of scientists, inventors and business leaders as heroic figures.

I wonder if there is also a psychological factor at work. Something along the lines of: a major world crisis, especially one that causes a lot of deaths, makes you think about the ephemeral nature of life and what's really important, and you either decide to have kids or to stop putting it off. Imagine someone reading about deaths among the elderly and thinking, “I really want Mom and Dad to meet their grandkids—better get on it.”

Curious if Claudia Goldin's work is relevant here?

The Shirky Principle: “Institutions will try to preserve the problem to which they are the solution.”

Thanks. What I see is that this paper specifies “a non-naive precautionary principle” or “an intelligent application of the precautionary principle,” which implies something about what the precautionary principle might end up being in practice without those qualifiers…

FWIW, this Jones & Romer paper names “accelerating growth” as one of the key stylized facts that growth models should explain. See pp. 13–16. 

One example of accelerating progress they give is from Nordhaus's famous “price of light” paper:

Between 38,000 B.C. and 1750 B.C., the real price of light fell by a total of about 17%, based on the transition from animal or vegetable fat to sesame oil as a fuel. The use of candles and whale oil reduced the price by a further 87% by the early 1800s, an average annual rate of decline of 0.06% per year. Between

That said, I agree that it's possible that the future could hold something different, and the demographic transition is a good reason why. Chad Jones has drawn attention to this. We need more people and more brains to continue growth. I tend to think we will solve this by (1) solving the fertility crisis and getting back to high rates of population growth, and/or (2) using AI to substitute for human researchers.

No, you're looking at too short a timescale [edit: to be clear, this is referring to your specific point about constant growth rates over the last 200 years]. Zoom out to the last few thousand, or few tens of thousand years.

See this from Paul Romer: https://paulromer.net/speeding-up-and-missed-opportunities-evidence/

Key excerpt:

My conviction that the rate of growth in GDP per capita at the technological frontier had to be increasing over time sprang from a simple calculation. Suppose the modern rate of growth of real GDP per capita (that is the growth rate

Absolutely true that new subdomains open up new areas of low-hanging fruit. It is the “stacked S-curve” model.

Not immediately clear whether what this means for β>0. I think this model may be addressed in Bloom et al, or maybe in an earlier paper by Jones. I vaguely recall that it doesn't make a difference whether you analyze things in terms of the subdomains or the economy at large, but I don't have the exact reference at hand.

No. The Precautionary Principle doesn't just mean “take precautions when warranted.” No one would be against that. It has become more like a bias towards inaction, regardless of cost/benefit calculations. See Ridley's quote above, about how this “superficially sensible idea” was transformed into something irrational.

Haha, fair point! (Although I would suggest that the most productive way to do that would be to pen an opposing manifesto.)

To be exact, what he said was:

Our present society has been subjected to a mass demoralization campaign … under varying names like … “Precautionary Principle” [etc.]

I interpret that to mean not that he's against precaution, but that he thinks terms like that are being used to promote bad ideas.

Also, the Precautionary Principle is objectively bad:

As we look back on the failed civilizations of the past, we can see that they were so poor, their technology was so feeble, and their explanations of the world so fragmentary and full of misconceptions that their ca

Which specifics?

I've heard people criticize this for lacking nuance, not engaging with critics, and not citing sources. I feel this misunderstands the genre. It's a manifesto. It's not supposed to be nuanced or appeal to critics; it's supposed to be even a little divisive, drawing a line in the sand, recruiting those who are already sympathetic and ignoring or even repelling those who are not. It's not supposed to argue for its claims, it's supposed to stake out some beliefs and declare them.

If you just don't like manifestos of any stripe, then fine; but it never makes sense to criticize a piece for not being in a different genre.

I am pretty convinced that β>0, even though I also think there is some contribution to slowdown from other factors. Some reasons:

• Bloom et al observe β>0 not only in the economy at large but in various subdomains, such as transistors and Moore's Law. If you want to argue that this is all caused by society's resistance to innovation (which is real), then you have to argue that society also resists innovation in integrated circuits (which… doesn't seem like a thing).
• There are simple intuitions for β>0 based on low-hanging fruit and burden of knowledg

What about land cost for solar? At what point does that become a significant part of energy cost? Solar is less diffuse than wind but more diffuse than any fuel-based energy technology.

If it's not significant now, surely at some point on our way to becoming a Kardashev Type 1 civilization it becomes a problem?

You will hear a lot about Georgism which advocates a tax on the (unimproved) value of land. This might be a starting point: https://www.astralcodexten.com/p/your-book-review-progress-and-poverty 

Hmm, I honestly don't know whether progress studies can be applied to any random job or company. I think of it more about applying at a society-wide level. Of course, it might inspire some people to take jobs at more ambitious / cutting-edge companies (or start such companies!) But that also doesn't mean there's anything wrong with companies that aren't cutting-edge—it takes all kinds of companies to make a functioning economy.

If anything, maybe progress studies can help remind you all of the moral value of economic growth. To the extent you all do you job... (read more)

Update: I’m already planning to give brief remarks at a few events coming up very soon:

• Thurs, Aug 24: Recur Club founders meetup in Indiranagar. Register/apply here
• Sun, Aug 27: LessWrong / Astral Codex Ten meetup at Matteo Coffea

If you’re in/near Bangalore, hope to see you there!

Fascinating, thanks for the pointer!

This book is “for babies” but it's probably just about right for a 3yo. It is the best “STEM for babies” book I have ever seen, maybe the only one I really like: https://computerengineeringforbabies.com/ 

I don't know exactly how seriously to take it—but I know Michael Dearing, whose site that is, and he is 100% in favor of capitalism, so… at least partially serious?

Thanks Gergő. We're doing this as a “work made for hire” meaning that the rights belong to us and we can then license it however we want.

Thanks a lot, Zvi.

Meta-level: I think to have a coherent discussion, it is important to be clear about which levels of safety we are talking about.

• Right now I am mostly focused on the question of: is it even possible for a trained professional to use AI safely, if they are prudent and reasonably careful and follow best practices?
• I am less focused, for now, on questions like: How dangerous would it be if we open-sourced all models and weights and just let anyone in the world do anything they wanted with the raw engine? Or: what could a terrorist group do wi

Certainly. You need to look at both benefits and costs if you are talking about, for instance, what to do about a technology—whether to ban it, or limit it, or heavily regulate it, or fund it / accelerate it, etc.

But that was not the context of this piece. There was only one topic for this piece, which was that the proponents of AI (of which I am one!) should not dismiss or ignore potential risks. That was all.

This essay was written not written for the doomers. It was written for the anti-doomers who are inclined to dismiss any concerns about AI safety at all.

I may write something later about where I agree/disagree with the doom argument and what I think we should actually do.

Yes, certainly! But that point isn't relevant to the point I'm making here. And emphasizing that point as a way of arguing against AI risk itself is one of the things I'm discouraging. It would be like responding to concerns about drug safety by saying “but drugs save lives!” Yes, of course they do, but that isn't relevant to the question of whether drugs also pose risks, and what we should do about those risks.

Not just “safety is good”, but: (1) safety is a part of progress, rather than something opposed to it and (2) optimists should confront risks and seek solutions, rather than downplaying or dismissing them.

I think what Allen probably added was a more quantitative investigation of this idea. He gathered the price data for fuel, labor, capital, etc. and did the analysis of rates of profit and return on investment.

Added a little bit in the revised version to try to clarify this. Thanks again for the feedback

Not sure if this is quite what you are looking for, but I've been keeping a list of progress-related museums that I have visited or want to visit, large or small, including:

• Antique Gas & Steam Engine Museum in Vista, CA
• Charles River Museum of Industry and Innovation in Waltham, MA
• Henry Ford Museum in Dearborn, MI
• Davistown Museum in Maine
• Museum of Craft & Design in San Francisco
• History of Science museum in Harvard (one room)
• Jenner Museum, Gloucestershire; also a statue of Edward Jenner in Kensington Gardens?
• Bibliotheque de la Faculte de Medecine in

Thanks! Yes, this is definitely part of Allen's argument (maybe I should make that more clear).

I've been meaning to read that Devereaux post/series for a while, thanks for reminding me of it.

However, I don't you think can argue from “the Industrial Revolution got started in this very specific way” to “that is the only way any kind of an IR could ever have gotten started.” If it hadn't been flooded coal mines in Britain, there would have been some other need for energy in some other application.

I see it more as: you develop mechanization and energy technolo... (read more)

Was supposed to be “before products are launched”. Fixed, thanks

Related: The Long Now Foundation's Manual for Civilization

“What books would you want to restart civilization from scratch?”

The Long Now Foundation has been involved in and inspired by projects centered on that question since launching in 01996. (See, for example, The Rosetta Project, Westinghouse Time Capsules, The Human Document Project, The Survivor Library, The Toaster Project, The Crypt of Civilization, and the Voyager Record.) For years, Executive Director Alexander Rose has been in discussions on how to create a record of humanity and technology for

Why do you think we don't have more people starting ambitious genetic engineering projects?

What are the best near-term/foreseeable applications of genetic engineering? What is the low-hanging fruit here that we can see and define and should go after first?

Thanks.

Rather than asking how fast or slow we should move, I think it's more useful to ask what preventative measures we can take, and then estimate which ones are worth the cost/delay. Merely pausing doesn't help if we aren't doing anything with that time. On the other hand, it could be worth a long pause and/or a high cost if there is some preventive measure we can take that would add significant safety.

I don't know offhand what would raise my p(doom), except for obvious things like smaller-scale misbehavior (financial fraud, a cyberattack) or dramatic technological acceleration from AI (genetic engineering, nanotech).