Hi everyone. I am Allison Duettmann, president and CEO of Foresight Institute. Recently, I co-authored the book, 'Gaming the Future: Technologies for Intelligent Voluntary Cooperation' on our own Foresight Press with Christine Peterson and Mark S. Millar.
Also at Foresight, I direct the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Groups, Fellowships, Prizes, and Tech Trees. I also share Foresight’s work with the public, for instance at the Wall Street Journal, SXSW, O’Reilly AI, WEF, The Partnership on AI, Effective Altruism Global, and TEDx. Additionally, I founded Existentialhope.com, co-edited Superintelligence: Coordination & Strategy, and co-initiated The Longevity Prize. I advise non-profits, companies, and individuals. I hold an MS in Philosophy & Public Policy from the London School of Economics, focusing on AI Safety, and a BA in Philosophy, Politics, Economics from York University.
Ask me anything! I will be here to answer your questions Monday, March 20. Use the comments below to add questions, and upvote any questions you'd like to see me answer.
How does Foresight Institute allocate its time/effort across projects? Do you have a way of thinking about how much attention to spend on different speculative areas?
Foresight Institute was established in 1986 on the ideas discussed in Engines of Creation, Drexler, published by Eric Drexler, co-founder of Foresight. The book lays out a network of technologies that have the potential to significantly enhance the human condition, including nanotechnology, biotechnology, information technology, and cognitive science, which are interconnected with other important technologies like robotics and space exploration in complex ways.
Given the broad technology stack Engines considered, the book, and Foresight, became an early Schelling point for scientists and technologists who wanted great futures across the board of technologies.
So within this broad technology stack, we decide on our focus by weighing how much attention an issue we think of as important is already receiving with how much our community is in a position to contribute.
For instance, since our inception until today, the general field of molecular nanotechnology remains undervalued, and our community has a unique potential to contribute to it, so generally advancing the field in a beneficial direction is still where the bulk of our fellowship, prizes, workshops, and seminar strength lies.
Then, within our other technology focus areas (bio, neuro, space, secure human AI cooperation), there are often specific subdomains that are still too niche, exotic, ambitious or interdisciplinary for the mainstream of that field to address.
For instance, when potential AI race dynamics first became an issue, we used to hold annual workshops after the Bay Area EAGs, focused on AGI coordination across great powers and corporations:
At the time a lot of governance work popped up so we refocused our annual AGI-focused workshops on bridge-building between the security and AI safety communities, as a currently undervalued area that we can meaningfully contribute to given our existing strong security and cryptography community:
That being said, we’re currently reviewing whether to take up the AI coordination workshops again given timelines coming down, leading to new interest in revisiting those meetings.
Another area we’re taking up given shortening AI timeline that we think we have a comparative advantage in helping with given our AI and neurotech community is revisiting Whole Brain Emulation as a potential strategy for AI safety, leading to this 2023 workshop, chaired by Anders Sandberg, co-author of the original WBE roadmap in 2007:
Do you see most of the value/impact/benefit that the Foresight Institute produces as coming from a few key outputs, or from a larger list of projects each of which only produces a fraction of your total value? If the first, what are your key outputs?
I think Foresight's value comes from a larger list of projects each of which has a small chance at creating a large impact. This comes mostly from the fact that we focus on advancing the beneficial use of a variety of undervalued technologies, including nano, bio, neuro, computing, and space, whose trajectory is harder to predict. We do this through early ecosystem development in these areas, that usually includes tools like our fellowships, prizes, workshops, and virtual seminars. Given that different technologies impact Given that many of the technologies are influenced by the relative speed of other technologies, they will be advancing at varying rates, and tools to accelerate them are differently useful at different stages.
For instance, for driving progress in Molecular Nanotechnology, from 1986 onward, Foresight started hosting annual technical conferences, published research papers, developed a Nanotechnology Roadmap, and launched the Feynman Prizes to award work toward molecular manufacturing.
The road was incredibly bumpy, but in 2016, Sir Fraser Stoddart was finally awarded the Nobel Prize for his work “for the design and synthesis of molecular machines” (https://www.nobelprize.org/prizes/chemistry/2016/press-release/) just nine years after he received Foresight’s Feynman Prize for the exact work: https://foresight.org/foresight-feynman-prizes/
Today, molecular nanotechnology progress is accelerating faster, largely enabled by new AI simulation tools, such as AlphaFold, Rosetta, Samson, CanDo, and more. Simulation tools, combined with progress in newer approaches to molecular nanotechnology, such as DNA origami, led tech-analysts such as Eli Durado declare that it’s Nanotechnology’s spring: https://worksinprogress.co/issue/nanotechnologys-spring
To streamline progress across tool builders, in 2022 we hosted a Design Tool for Molecular Machines Systems-focused workshop https://foresight.org/molecular-workshop, whose 2023 iteration will focus on opportunities for combining insights across tools to work to the design of more complex molecular machinery: https://foresight.org/foresight-molecular-systems-design-workshop-2023
We aren’t a leading driver in each of the technological areas in particular but by providing a container that enables for multidisciplinarity across fields such as ML and molecular machines, we hope to facilitate insight and tech transfer across them.
What are some example decisions that the Foresight Institute's work has helped influence?
Given that our main effort is to kindle beneficial innovation in undervalued technical domains of importance for the long-term future, such decisions are sometimes hard to trace but are mostly in the area of founding and funding such projects.
Through Foresight matchmaking, members have started companies (such as a carbon drawdown company co-founded by a Foresight Fellow who met their co-founder at a Foresight event and recently raised $30M in follow-on funding), new research projects (such as a major research project building LLM-enabled preference simulations of groups of people which was founded and funded at a Foresight workshop), and existing organizations receiving government funding (more than $30M for a water filtration company, and $15M for a molecular nanotechnology simulation project at a university through Foresight workshops).
Other decisions we shape involve early career path choices, with individuals joining organizations, including a neurotech FRO, several major longevity companies, and security companies, through Foresight events. In rare cases, aid career decisions more actively, for instance by providing J1 visas to promising researchers seeking to move to the US. This more tailored support is particularly prominent with younger applicants who have little default exposure to senior researchers, funders, and entrepreneurs in their domain.
What does Eric Drexler think about the Foresight Institute? If I recall correctly, he was one of the founders?
I think this is a question that is better directed at Eric himself :) I can confirm that he was one of Foresight’s co-founders, and that he did present at a few more Foresight recent events, such as the Decentralized AI workshop (https://www.youtube.com/watch?v=pClSjljMKeA, https://www.youtube.com/watch?v=hNDD-ZbEsJA) and a Molecular Machines workshop (https://www.youtube.com/watch?v=HjgjtAk-lws&t=1s).
I can also definitely say that our community remains excited about his outstanding work, such as Comprehensive AI Services, the Open Agency Architecture, Paretotopian Goal Alignment, and Molecular Nanotechnology.
It looks like the next major technological wave will be AI. How might this change Foresight's plans or focus areas? Would you focus more on AI? Or, can AI help us with nanotech, longevity, etc. (and how exactly)?
In a few previous comments here, I point out how we integrate ML as a major driver of progress in our areas, e.g. such as molecular machines simulation tools, and how it affects our focus with respect to whole brain emulations. I give a longer review of how computing and AI progress affects each of our technical domains in this Breakthroughs in Computing Series by Protocol Labs: https://www.youtube.com/watch?v=lBvkFZycXRQ
With respect to Foresight’s role in safe AI progress, I think Foresight’s comparative advantage lies in bringing computer security inspired lens to AI development:
This is largely due to Foresight Senior Fellow Mark Miller, who, in 1996, gave this talk on Computer Security as the Future of Law (http://www.caplet.com/security/futurelaw), and together with Eric Drexler, published the foundational Agoric Open Systems Papers (https://papers.agoric.com/papers/), laying out a general model of cooperation enabled by voluntary rules, that applies not only to today’s human economy, but may be transferable to a future ecology, populated by human and AI intelligences.
Mark built on the Agoric papers by following the computer security thread as a necessary condition for building systems in which both humans and AIs could voluntarily cooperate. Recently this thinking culminated in Mark, Christine Peterson (Foresight’s co-founder) and me co-authoring the book Gaming the Future, focusing on specific cryptography and security tools that may help secure human AI cooperation on the path to paretotopian futures: https://foresight.org/gaming-the-future-the-book.
I think Miller’s and Drexler’s work on reframing the traditionally singleton-focused AI safety in terms of secure coordination across human and AI entities that relies on the respect of boundaries is now more relevant than ever, given A infosecurity risks, that have become a larger focus within AI alignment. I have a longer Lesswrong post on this coming next weekend.
What areas of research or technology are most underrated by the broader research world, and why?
Here are a few across different Foresight focus areas:
Security & AI:
What are some new projects you are working on but haven't published yet?
What are some projects you want to see but wish someone else or another org was working on?
Hi Sam, here are two previews of projects we're working on but which aren't published yet.
Throughout 2022, we have been building technology trees to map our five interest areas; molecular nanotechnology, longevity biotechnology, neurotechnology, secure human AI interaction, and space. The goal is to help onboard new talent and funders into the fields by sketching out which required capabilities are required for the long-term goals of the field, who is working on them, and which open challenges are left to be tackled. The trees contain 50k+ nodes but the current interfaces are still pretty clunky and hard to navigate for outsiders: https://foresight.org/tech-tree
What’s new is that we’ll likely be launching a Discourse graph-enabled tech tree edition, which allows natural language question-based navigation of the trees, making the main info much easier to digest for users. In addition, a gpt integration in the tool itself can automate parts of the research process by populating entire paths of the tree automatically. For instance, when prompting the gpt integration questions such as “what are the ten main labs working on autophagy” or “what are the main technical challenges we need to solve to make progress on privacy-preserving ML?” replies relatively well matched human-generated replies, even though there is still fact-checking and completion to do. This means our tech tree architects can function as reviewers and editors, rather than research assistants combing the web from scratch, making the roadmaps more long-term sustainable.
The discourse graph editions of the trees scheduled to go live by July would allow individuals to contribute to the main trees and fork their own AI-assisted tech trees. They would also enable users to advance progress on highlighted challenges via an integrated bounty tool. Thanks to the amazing Discourse graph team for building the tool and allowing us to use it. More about how the tool works: https://protocol.ai/blog/discourse-graph-qa/
We’re currently working on a book proposal on Existential Hope to highlight alternative futures to the currently en vogue doomerism. It’s early stage but may discuss various great future scenarios, plus “eucatastrophes”, i.e. positive turning points, technologies and strategies to get there. Many of the people and resources that inspire the book can be found on: https://www.existentialhope.com
How do you think that framing a discussion about the effects of future technologies as potentially leading to scenarios of existential hope as opposed to existential risk can be helpful? Or is positing a dichotomy between existential hope and risk sort of missing a key point?
That’s an interesting question and I would love to know more about what key point you think it’s missing.
I’m the meantime, here’s two things I’d say:
Very interesting, thanks for the thoughts!
I realize now that my questions were a bit unclear. I tend to think about the world in terms of trade-offs. So my first question was really about the trade-off of thinking about the future in terms of existential hope vs existential risk.
You already addressed a key upside of thinking in terms of existential hope that I hadn't thought of with your first point, which is that thinking of the future can create a self-fulfilling prophecy, so it's better to have a positive vision of the future than a negative one.
My second question was mostly about my own reticence to posit trade-offs everywhere since I do it too much probably. Sometimes, there is a false dichotomy in thinking about things in dichotomous ways ("both/and" instead of "either/or"). So perhaps it's not best to think of thinking about existential hope vs existential risk as a trade-off at all. That's what I was getting at, about whether I was missing a key point about the way you think about this topic by trying to frame the discussion in terms of a dichotomy.
By the way, I love the idea of existential hope and think it is a beneficial concept, in part to help avoid doomerism. =)