All of jasoncrawford's Comments + Replies

Accelerating science through evolvable institutions

Good point, I agree! Something important to creating the right research lab team and culture.

Accelerating science through evolvable institutions

Hmm, I don't agree with how you are characterizing my assumptions about human nature. I'm not assuming that scientists are after money or prestige. I assume most of them, or at least the best of them, are motivated by curiosity, the desire to discover and to know, and the value of scientific knowledge for humanity.

Re accountability, I frankly think we could do with a bit less of it. Accountability is always in tension with research freedom.

Re people performing for their superiors: I actually think scientists performing for their managers would be a much healthier model than what we have today, which is scientists performing for their grant committees. I have another piece on this that I plan to publish soon.

1NicholasGruen2moThanks Jason, I don't think you've understood what I was trying to get at.
Accelerating science through evolvable institutions

Curiosity is already a very strong motivator, we just need to enable it and get out of the way. Give scientists funding without making them narrowly constrain their goals, dial down their ambition, or spend half their time writing grants. Then give them the research freedom to pursue that curiosity wherever it leads. It's not easy but it is pretty simple.

2Michael Frank Martin2moI agree with that. But having seen IBM ARC up close in person in the 1990s, my gut is that there is some critical mass of curiosity -- a threshold number of curious researchers all working in the same place -- that leads to a kind of magic you don't see when the same people are more distributed geographically.
Marc Andreessen pens “Techno-Optimist Manifesto.” Discuss

Good question, I don't know. People have been talking about “progress studies” or the “progress movement” or “progress community”, and others have talked about the “abundance agenda”, but none of those lend themselves to personal labels/identities…

Marc Andreessen pens “Techno-Optimist Manifesto.” Discuss

I have never particularly liked the term “techno-optimism” anyway. “Optimism” on its own is confusing enough. “Techno-optimism” implies that not only do you think we can solve all problems, but that technology will be the solution to all of them, which is not really true.

1Yarrow Bouchard3moWhat’s a good alternative word for someone who has a strong conviction in the past, present, and future benefits of technology?
Neither EA nor e/acc is what we need to build the future

Thanks Robert. I think progress studies needs a more well-defined value system. I have gestured at “humanism” as the basis for this, but it needs much more.

I agree that Rand's ideas are important here, particularly her view of creative/productive work as a noble activity and of scientists, inventors and business leaders as heroic figures.

Baby busting myths about shortages

I wonder if there is also a psychological factor at work. Something along the lines of: a major world crisis, especially one that causes a lot of deaths, makes you think about the ephemeral nature of life and what's really important, and you either decide to have kids or to stop putting it off. Imagine someone reading about deaths among the elderly and thinking, “I really want Mom and Dad to meet their grandkids—better get on it.”

Curious if Claudia Goldin's work is relevant here?

2Tina Marsh Dalton4moYes, the psychological factor is often cited for discrete events that bring people closer together or highlight a stark idea of what is important in their life. But did COVID initially present a more troubling future? That might work against this idea, because you are pessimistic about the future of a world subject to a global pandemic. However, your point might hold differently for the women highlighted here, since they are in a much more secure place than their peers subject to exposure and uncertainty about their employment. I've also seen discussion about how the opportunity cost of time-- what else women could be doing during this period-- fast-forwarded plans. Nothing much to do with my free time- might as well have a baby! That could speak to Claudia's work because her thesis about women's late fertility has to do with the cost of establishing a career. The time cost of this delays having a family. In the COVID period, many time costs were slashed- i.e. commuting, meetings, besides most social obligations. Might have seemed more feasible to start families with a 2000/2001 view of the balance of time available for both pursuits.
Commensal Institutions

The Shirky Principle: “Institutions will try to preserve the problem to which they are the solution.”

2Sable4moYep. I wanted to lay out a somewhat more detailed accounting of it, as a basis for future work on how institutions are designed - and how they should be designed, if we want them to be more effective.
Marc Andreessen pens “Techno-Optimist Manifesto.” Discuss

Thanks. What I see is that this paper specifies “a non-naive precautionary principle” or “an intelligent application of the precautionary principle,” which implies something about what the precautionary principle might end up being in practice without those qualifiers…

Where I have landed on “ideas getting harder to find”, in a nutshell

FWIW, this Jones & Romer paper names “accelerating growth” as one of the key stylized facts that growth models should explain. See pp. 13–16. 

One example of accelerating progress they give is from Nordhaus's famous “price of light” paper:

Between 38,000 B.C. and 1750 B.C., the real price of light fell by a total of about 17%, based on the transition from animal or vegetable fat to sesame oil as a fuel. The use of candles and whale oil reduced the price by a further 87% by the early 1800s, an average annual rate of decline of 0.06% per year. Between

... (read more)
Where I have landed on “ideas getting harder to find”, in a nutshell

That said, I agree that it's possible that the future could hold something different, and the demographic transition is a good reason why. Chad Jones has drawn attention to this. We need more people and more brains to continue growth. I tend to think we will solve this by (1) solving the fertility crisis and getting back to high rates of population growth, and/or (2) using AI to substitute for human researchers.

1SamHughes4moYes, that’s essentially the stylised model I use – i.e. I understand the long-run history of GDP per capita growth at the frontier as a transition from stagnation (/a very low rate) into sustained growth at a roughly constant rate. And it is very stylised (and I allow that the take-off may have been quite gradual), but I still think it works quite well as a basic framework. And I agree that Romer’s backwards projection implies that the rate of GDP per capita growth at the frontier has increased over time; but it doesn’t prove that this took the form of a constant acceleration across all of history, rather than a roughly discrete acceleration (described above). I don’t yet think that the Maddison data supports the idea of accelerating frontier growth across millennia. I think we need better country and year coverage to establish that claim. Better country coverage because the country at the frontier changes over time. Even if we see constant acceleration in country X’s GDP per capita growth rate between (say) 1-1800AD, it is unlikely that it was consistently at the frontier. We need to splice together data from various countries to get a timeseries of frontier growth. And better year coverage to avoid us relying on data points which may just so happen to be at a low or high point in a fluctuating cycle. We might have a higher estimate of GDP per capita in country Y for AD1000 than AD1, but I’d need more convincing to interpret that as long-run growth rather than our data point for AD1000 incidentally being a good year (or at the high point of a cycle which may span generations) and/or our data point for AD1 incidentally being a bad year (/low point in a cycle).
Where I have landed on “ideas getting harder to find”, in a nutshell

No, you're looking at too short a timescale [edit: to be clear, this is referring to your specific point about constant growth rates over the last 200 years]. Zoom out to the last few thousand, or few tens of thousand years.

See this from Paul Romer: https://paulromer.net/speeding-up-and-missed-opportunities-evidence/

Key excerpt:

My conviction that the rate of growth in GDP per capita at the technological frontier had to be increasing over time sprang from a simple calculation. Suppose the modern rate of growth of real GDP per capita (that is the growth rate

... (read more)
1jasoncrawford4moThat said, I agree that it's possible that the future could hold something different, and the demographic transition is a good reason why. Chad Jones has drawn attention to this. We need more people and more brains to continue growth. I tend to think we will solve this by (1) solving the fertility crisis and getting back to high rates of population growth, and/or (2) using AI to substitute for human researchers.
Something Is Getting Harder To Find But It's Not Ideas

Absolutely true that new subdomains open up new areas of low-hanging fruit. It is the “stacked S-curve” model.

Not immediately clear whether what this means for β>0. I think this model may be addressed in Bloom et al, or maybe in an earlier paper by Jones. I vaguely recall that it doesn't make a difference whether you analyze things in terms of the subdomains or the economy at large, but I don't have the exact reference at hand.

Marc Andreessen pens “Techno-Optimist Manifesto.” Discuss

No. The Precautionary Principle doesn't just mean “take precautions when warranted.” No one would be against that. It has become more like a bias towards inaction, regardless of cost/benefit calculations. See Ridley's quote above, about how this “superficially sensible idea” was transformed into something irrational.

Marc Andreessen pens “Techno-Optimist Manifesto.” Discuss

Haha, fair point! (Although I would suggest that the most productive way to do that would be to pen an opposing manifesto.)

Marc Andreessen pens “Techno-Optimist Manifesto.” Discuss

To be exact, what he said was:

Our present society has been subjected to a mass demoralization campaign … under varying names like … “Precautionary Principle” [etc.]

I interpret that to mean not that he's against precaution, but that he thinks terms like that are being used to promote bad ideas.

Also, the Precautionary Principle is objectively bad:

As we look back on the failed civilizations of the past, we can see that they were so poor, their technology was so feeble, and their explanations of the world so fragmentary and full of misconceptions that their ca

... (read more)
1NunoSempere4moYou might want to Ctrl+F here for mentions of the precautionary principle: https://www.sciencedirect.com/science/article/pii/S0169207020301230 [https://www.sciencedirect.com/science/article/pii/S0169207020301230]
1Jelle Donders4moThe precautionary principle is objectively bad? That's a massive assumption that only holds if you are somehow confident that nuclear war, engineered pandemics, advanced AI derailing society etc. are all impossible, right?
2NunoSempere5moE.g., "Our enemy is the Precautionary Principle", unqualified
Marc Andreessen pens “Techno-Optimist Manifesto.” Discuss

I've heard people criticize this for lacking nuance, not engaging with critics, and not citing sources. I feel this misunderstands the genre. It's a manifesto. It's not supposed to be nuanced or appeal to critics; it's supposed to be even a little divisive, drawing a line in the sand, recruiting those who are already sympathetic and ignoring or even repelling those who are not. It's not supposed to argue for its claims, it's supposed to stake out some beliefs and declare them.

If you just don't like manifestos of any stripe, then fine; but it never makes sense to criticize a piece for not being in a different genre.

1Jelle Donders4moThis works both ways imo. You can boldly state things in a manifesto, and people can boldly criticize it.
Solved and Unsolved Externality problems?

Some negative externalities and other problems of progress that may or may not be considered solved:

  • how technology can make war more deadly
  • how it can assist oppressive/authoritarian regimes
  • technological unemployment
  • unclean/unsafe food, water supplies
  • resource depletion
  • pollution of all kinds
  • carcinogens

Positive externalities are like, almost all of human wealth?

Yale economist William Nordhaus estimates that innovators capture a mere 2.2% of the total “surplus” from innovation. … If it is anywhere close to being an accurate estimate, the implication is that “s

... (read more)
Something Is Getting Harder To Find But It's Not Ideas

I am pretty convinced that β>0, even though I also think there is some contribution to slowdown from other factors. Some reasons:

  • Bloom et al observe β>0 not only in the economy at large but in various subdomains, such as transistors and Moore's Law. If you want to argue that this is all caused by society's resistance to innovation (which is real), then you have to argue that society also resists innovation in integrated circuits (which… doesn't seem like a thing).
  • There are simple intuitions for β>0 based on low-hanging fruit and burden of knowledg
... (read more)
2Maxwell Tabarrok4moI basically agree with everything you've said here. On the subdomains point, you can have decreasing returns within each subdomain but constant returns overall if you keep finding new subdomains. I think this is an accurate model of progress. It captures ideas like paradigm shifts and also integrates the intuitions for low-hanging fruit and burden of knowledge in a way which still allows rapid progress. My favorite example is the Copernican revolution. There were huge obstacles from burden of knowledge and low-hanging fruit in Ptolemaic astronomy. It took so much extra data and education to improve the epicycles of Mercury by a few decimal points. But once astronomy moved to a new model, there was a whole new grove of low hanging fruit and almost none of the investment in Ptolemaic astronomy was necessary to make progress so the burden of knowledge was reset.
Radical Energy Abundance

What about land cost for solar? At what point does that become a significant part of energy cost? Solar is less diffuse than wind but more diffuse than any fuel-based energy technology.

If it's not significant now, surely at some point on our way to becoming a Kardashev Type 1 civilization it becomes a problem?

3AnthonyC5moRight now land costs are on the order of $1k-$2k/acre/yr (1 acre ~ 4000m2, but I find it a convenient metric because an average acre receives an average of just over 4MW of sunlight if you spread it across the full 8760 hours in a year, which gives an average of ~1MW output at current efficiencies if you had 100% panel coverage). and with current efficiencies in typical regions that's something like 2000-8000 MWh/yr depending on local weather and panel layout, so <$1/MWh. If we move towards tandem or other multijunction cells (which seems plausible in the 2030s) that power density could double. In addition there are some slower trends that should start to support things like agrivoltaics (dual use of land without decreasing crop yields) and comparably cheap or cheaper non-silicon semitransparent panels (which can actually be used in greenhouses or over crops, selectively absorbing wavelengths plants can't use while providing shade to reduce water consumption). In other words, there are lots of options to address this. World electricity consumption would have to increase by at least 3 orders of magnitude before land use even started to become a consideration. I do think the OP is overestimating the rate at which energy storage and synthetic fuel costs will fall, and that that is a bigger consideration than land use. I also think resistance to early retirement of existing assets will slow down the later stages of the move away from fossil fuels, both in electricity generation and in transportation fuels. But I doubt that shifts the overall timeline by more than 5-10 yrs.
6Casey Handmer5moSolar panels are more economically productive than any unused land, forestry, or agriculture, and even some land uses in built up areas, such as car parking. What this means is that deploying solar upgrades utility. There is a question whether there is enough land. The short answer is yes, easily, it's not even close. Something like 4-5% of Earth's land surface with solar can provide enough energy for 10 billion people to live at current US levels of energy consumption, and more than 35% of Earth's land surface is essentially uninhabited deserts, mountains, swamps, forests, etc. The longer answer is that we can provide the food needs of our civilization with about 20% of Earth's land surface area under more-or-less intense cultivation, our civilization consumes roughly 100x more energy in the form of electricity, oil and gas, than food, and solar energy is about 1000x more productive, per unit area, than plants. Strictly speaking, Kardashev Level 1 would require the entire surface, land and water, of Earth to be paved with solar. This is not particularly desirable nor necessary, in my opinion!
Elle Griffin's Shortform

You will hear a lot about Georgism which advocates a tax on the (unimproved) value of land. This might be a starting point: https://www.astralcodexten.com/p/your-book-review-progress-and-poverty 

1Elle Griffin5moThank you!
Does the Progress Movement agree with Singer's Drowning Child Argument?

I don't think there would be broad agreement within the progress community about the Singer argument, or more generally about utilitarianism.

Personally, I am neither a utilitarian nor an altruist, and I don't agree with the drowning child argument as I understand it.

I think how much to spend on yourself vs. charity or other causes that you believe in is a personal decision, based on what is meaningful and important to you.

2Isaac Benson6moI am curious what you think about it. Do you have any refutation of the argument? Is the opinion one your willing to share?
How to help if you work in average company?

Hmm, I honestly don't know whether progress studies can be applied to any random job or company. I think of it more about applying at a society-wide level. Of course, it might inspire some people to take jobs at more ambitious / cutting-edge companies (or start such companies!) But that also doesn't mean there's anything wrong with companies that aren't cutting-edge—it takes all kinds of companies to make a functioning economy.

If anything, maybe progress studies can help remind you all of the moral value of economic growth. To the extent you all do you job... (read more)

Jason Crawford in Bangalore, August 21 to September 8

Update: I’m already planning to give brief remarks at a few events coming up very soon:

If you’re in/near Bangalore, hope to see you there!

Matt Ritter's Shortform

This book is “for babies” but it's probably just about right for a 3yo. It is the best “STEM for babies” book I have ever seen, maybe the only one I really like: https://computerengineeringforbabies.com/ 

6 Minute Capitalist Meditation

I don't know exactly how seriously to take it—but I know Michael Dearing, whose site that is, and he is 100% in favor of capitalism, so… at least partially serious?

Wanted: Technical animator and/or front-end developer for interactive diagrams of invention

Thanks Gergő. We're doing this as a “work made for hire” meaning that the rights belong to us and we can then license it however we want.

If you wish to make an apple pie, you must first become dictator of the universe [draft for comment]

Thanks a lot, Zvi.

Meta-level: I think to have a coherent discussion, it is important to be clear about which levels of safety we are talking about.

  • Right now I am mostly focused on the question of: is it even possible for a trained professional to use AI safely, if they are prudent and reasonably careful and follow best practices?
  • I am less focused, for now, on questions like: How dangerous would it be if we open-sourced all models and weights and just let anyone in the world do anything they wanted with the raw engine? Or: what could a terrorist group do wi
... (read more)
1Zvi Mowshowitz8moI think it's an important crux of its own which level of such safety is necessary or sufficient to expect good outcomes. What is the default style of situation and use case? What can we reasonably hope to prevent happening at all? Do our 'trained professionals' actually know what they have to do, especially without being able to cheaply make mistakes and iterate, if they do have solutions available? Reality is often so much stupider than we expect. Saying 'it is possible to use a superintelligent system safely' would, if true, be highly insufficient, unless you knew how to do that, were willing to make the likely very very large performance sacrifices necessary (pay the 'alignment tax') in the face of very strong pressures, and also ensure no one else did it differently, and that this state persists. Other than decelerationists, I don't see people proposing paths towards keeping access to such systems sufficiently narrow, or constraining competitive dynamics such that people with such systems have the affordance to pay large alignment taxes. If it is possible to use such systems safely, that safety won't come cheap. I do think you are right that we disagree about the nature of such systems. Right now, I think we flat out have no idea how to make an AGI do what we'd like it to do, and if we managed to scale up a system to AGI-level using current methods, even the most cautious user would fail. I don't think there is a 'power-seeking' localized thing that you can solve to get rid of this, either. But yeah, as for the crux it's hard for me to pinpoint someone's alternative mindset on how these systems are going to work, that makes 'use it safely' a tractable thing to do. Throwing a bunch of stuff out there I've encountered or considered, in the hopes some of it is useful. I think you're imagining maybe some form of... common sense? Satisficing rather than pure maximization? Risk aversion and model uncertainty and tail risk concerns causing the AI to avoid disrup
A plea for solutionism on AI safety

Certainly. You need to look at both benefits and costs if you are talking about, for instance, what to do about a technology—whether to ban it, or limit it, or heavily regulate it, or fund it / accelerate it, etc.

But that was not the context of this piece. There was only one topic for this piece, which was that the proponents of AI (of which I am one!) should not dismiss or ignore potential risks. That was all.

1Max Görlitz8moThanks, this is useful! I know the Convergent folks but wasn't aware of Speculative Technologies. The article looks interesting but is, unfortunately, paywalled/sign-up-walled.
A plea for solutionism on AI safety

This essay was written not written for the doomers. It was written for the anti-doomers who are inclined to dismiss any concerns about AI safety at all.

I may write something later about where I agree/disagree with the doom argument and what I think we should actually do.

A plea for solutionism on AI safety

Yes, certainly! But that point isn't relevant to the point I'm making here. And emphasizing that point as a way of arguing against AI risk itself is one of the things I'm discouraging. It would be like responding to concerns about drug safety by saying “but drugs save lives!” Yes, of course they do, but that isn't relevant to the question of whether drugs also pose risks, and what we should do about those risks.

1Moritz Wallawitsch8moWhy would it not be relevant to the question? What's the value of only looking at eliminating the potential risk? Regulating a technology is not just about eliminating the risks of it but about reducing the risks to some extent while still enabling the upside. the upsides need to be clearly analyses and acknowledged.
A plea for solutionism on AI safety

Not just “safety is good”, but: (1) safety is a part of progress, rather than something opposed to it and (2) optimists should confront risks and seek solutions, rather than downplaying or dismissing them.

What if they gave an Industrial Revolution and nobody came?

I think what Allen probably added was a more quantitative investigation of this idea. He gathered the price data for fuel, labor, capital, etc. and did the analysis of rates of profit and return on investment.

What if they gave an Industrial Revolution and nobody came?

Added a little bit in the revised version to try to clarify this. Thanks again for the feedback

Progress Travel Guide

Not sure if this is quite what you are looking for, but I've been keeping a list of progress-related museums that I have visited or want to visit, large or small, including:

... (read more)
What if they gave an Industrial Revolution and nobody came?

Thanks! Yes, this is definitely part of Allen's argument (maybe I should make that more clear).

I've been meaning to read that Devereaux post/series for a while, thanks for reminding me of it.

However, I don't you think can argue from “the Industrial Revolution got started in this very specific way” to “that is the only way any kind of an IR could ever have gotten started.” If it hadn't been flooded coal mines in Britain, there would have been some other need for energy in some other application.

I see it more as: you develop mechanization and energy technolo... (read more)

2jasoncrawford10moAdded a little bit in the revised version to try to clarify this. Thanks again for the feedback
Machines: Global Village Construction Set

Related: The Long Now Foundation's Manual for Civilization

“What books would you want to restart civilization from scratch?”

The Long Now Foundation has been involved in and inspired by projects centered on that question since launching in 01996. (See, for example, The Rosetta Project, Westinghouse Time Capsules, The Human Document Project, The Survivor Library, The Toaster Project, The Crypt of Civilization, and the Voyager Record.) For years, Executive Director Alexander Rose has been in discussions on how to create a record of humanity and technology for

... (read more)
2jonathan.starr10moAlso related: HOW THINGS WORK VOLUMES, I TO IV [HTTPS://WWW.AMAZON.COM/HOW-THINGS-WORK-VOLUMES-IV/DP/1114119512/REF=SR_1_11?KEYWORDS=HOW+THINGS+WORK&QID=1682268221&S=BOOKS&SPREFIX=HOW+THINGS%2CSTRIPBOOKS%2C72&SR=1-11] by Roger Jean Segalat
When will AutoGPT like software be able to write Enviromental Impact Assessments?

I bet GPT-4 could already do a lot of this work, perhaps with some fine-tuning and/or careful prompt engineering.

The problem with automating compliance documents is not just the time/effort to prepare them. It's also the time spent waiting to get a response, and in some cases, “user fees” paid to the government to review them. If everyone started using GPT to do compliance, I suspect that the various agencies would just start to build up an ever-growing backlog of un-reviewed applications, until they're all like immigration and they have decade-long wait times.

1ErikSchmidt10moAlso, if GPT-4 raises the "productivity" of environmental impact statement, my guess is that they could only increase productivity so long as there is not broad awareness and acceptance of the technology in writing the statements. If GPT-4 becomes the standard in writing statements, then expectations for the length and breadth of the statements have the potential to rise as well.
AMA: Bryan Bishop, Biohacker & Founder of Custodia Bank

Why do you think we don't have more people starting ambitious genetic engineering projects?

3kanzure1yThere is definitely a bizarre social taboo surrounding the pursuit of some of these projects. Another constraint is that even if someone is doing the work, they can't exactly be public especially in germline because the privacy of the child is of utmost importance. Researchers in academia are mostly focused on grants for curing various diseases because that's what appeals to the appetite of federal funding agencies and the philanthropic organizations. The academic biologists tend to be extremely sensitive to public opinion because the public controls much of the federal funding. As a result, they have felt the burn from the anti-GMO people and the attempts at stopping embryonic stem cell research. They absolutely do not want further prohibitions on research and they worry about people outside of academia doing things that cause a backlash on federal funding of researchers. Thankfully, you don't need to do this work inside of academia. Focusing on diseases will never lead to extremely cheap interventions; there's simply not enough sick people with the same problem. Nobody really focuses on enhancements. As a result, costs are going to remain high because the market for a specific disease can be incredibly small. Meanwhile the market for general broad spectrum mass market enhancement has a potential population of almost 8 billion people. I also think that biologists don't paint that interesting of a future. They usually talk about curing diseases but don't have any vision beyond that point. What are we going to do after we cure all diseases? The silicon people have visions of computronium painting the universe. The biologists don't really promote visions of a flourishing biosphere across the entire cosmos or some other moral vision for progress. There's some good news though. Since not everyone is working on the ambitious projects, there's lots of low-hanging fruit available. I think there's enough people working on curing all diseases or ending aging/anti-aging/
AMA: Bryan Bishop, Biohacker & Founder of Custodia Bank

What are the best near-term/foreseeable applications of genetic engineering? What is the low-hanging fruit here that we can see and define and should go after first?

2kanzure1yI have a few other genetic interventions and modifications in the comments, but see also https://diyhpl.us/wiki/genetic-modifications/ [https://diyhpl.us/wiki/genetic-modifications/] for a list.
Wizards and prophets of AI [draft for comment]

Thanks.

Rather than asking how fast or slow we should move, I think it's more useful to ask what preventative measures we can take, and then estimate which ones are worth the cost/delay. Merely pausing doesn't help if we aren't doing anything with that time. On the other hand, it could be worth a long pause and/or a high cost if there is some preventive measure we can take that would add significant safety.

I don't know offhand what would raise my p(doom), except for obvious things like smaller-scale misbehavior (financial fraud, a cyberattack) or dramatic technological acceleration from AI (genetic engineering, nanotech).

1niplav1yTrue, I was insufficiently careful with my phrasing.
Load More