Dreams and VR could be two underexplored levers in unlocking human potential.

Recently, have been fascinated by the rehearsal of embodied cognition during REM sleep, or the ability of the human body and brain to continue codifying, practicing, and mastering skills via dreams. This interest was triggered by a specific incident last week:

• Swam ~40 laps at 9pm
• Fell asleep and dreamt of swimming laps all night (slept ~8 hours)
• More accurately, had a memory of swimming laps for hours, recollecting specific individual, unique laps
• Woke up feeling 1) tired, and 2) like I’d learnt a new way of swimming freestyle.

During the next swim, I found that I was faster, and somehow better — as if a dream-generated syntax had produced semantics. Or, as if subconscious cognitive processing induced a degree of muscle memory.^1[1]

To say cognition is embodied means that it arises from bodily interactions with the world. But do interactions necessarily have to be situated within the physical world? In the above case, the world in which embodied cognition occurred was a high-fidelity mental visualisation — a dream.

Experiencing incidental embodied cognition through dreaming is exciting — it features a strange feeling of déjà vu, of knowing but not remembering. Of having the illusion of a memory; a sense of familiarity with and confidence in a skill, accompanied by an eagerness to test it out in the physical world.

If we’re able to render high-fidelity mental visualisations - sometimes at will, by lucid dreaming - can learning and experiencing skills in this dream-state lead to elevated performance in the real-world? Research indicates that this might be possible — studies suggest that lucid dreaming could be as effective as some types of physical practice & wakeful visualisation techniques among athletes.

“Dreams may be the most vivid and multi-sensorial form of embodied cognition” - Domhoff

Lucid dreaming could be an effective, unique technique to rehearsing and developing embodied cognition in dreams-worlds — but learning via dreams seems to be mostly restricted to physical skills and motor movements. Furthermore, stats suggest that the incidence of people who are able to conjure up lucid dreams reliably & frequently (at least once per month) is low (<20%).

All this to suggest that while mastering a skill through incidental or intentional dreaming can be thrilling, as a learning technique it may be limited in scope, unreliable, inaccessible, and sometimes simply undesirable.

Calibrating moonshots

In Political debates 50 years from now I proposed that by the 2070s, it’s possible that would have figured out a way to 1) perform effortful and intentional cognitive activity while dreaming, and 2) tangibly capture dreams through consumer-facing devices.

These were moonshot claims—but thought it might be interesting to plot out if, or how, each of these realities could be achieved within the current (2023) paradigm of science and technology.

On claim #1 — It’s possible that would have figured out a way to perform effortful and intentional cognitive activity while dreaming.

• Implication: Explores the possibility that one day, learning or encoding new information while simultaneously reaping the benefits of REM sleep might be possible
• Bottom line: likely not possible to organically “learn” novel content while sleeping; but can envision a world where memory prosthetics (a form of BCI) stimulate the brainwaves needed to create an optimal knowledge consolidation & rehearsal environment.
• Explanation:
  • Sleep supports memory consolidation. Memories formed during the day are weak and temporary: during sleep, the brain makes an active decision to discard a memory, or strengthen and integrate it into existing knowledge networks.
  • Encoding new information (i.e. absorbing novel info and stimuli) cannot occur in parallel with consolidation (i.e. processing existing cognitive information into a long-term store) — so, simplistically, the daytime is reserved for encoding new stimuli, and sleep-states are reserved for consolidating them into long-term memory
  • It is incompatible for encoding and consolidation to occur simultaneously since the processes compete for shared neural resources—for now, a satisfying explanation for why we suspend consciousness & the absorbing of novel stimuli while sleeping.

On claim #2 — It’s possible that would have figured out a way to tangibly capture dreams through consumer-facing devices

• Bottom line: Preliminary research suggests that the visual experience of dreaming may be capturable (e.g., in the format of a video), and that decoding brainwaves could allow an external observer to predict or analogously view the contents of someone else’s dream
• Explanation:
  • Detecting dreams: Detecting the presence of a dream is the first step to capturing it. Activity in a specific “posterior cortical hot zone” in the brain is an indicator that a dream or subjective experience is occurring.
  • Identifying parts we can map out: While dreams are multi-sensory, it seems that the easiest part to decode and record is vision. Additionally, it is the most dominant modality in dreaming across cultures and ethnicities. Some visual aspects of dreaming can be captured in the form of a video or movie in low spatial and temporal resolution — but today, this decoding is restricted to research studies and labs
  • Capturing and re-running dreams: If a dream is a program executed by your brain, it should be theoretically possible to re-run it an another time. Neuroscientist Yukiyasu Kamitani is exploring this possibility using machine-learning algorithms, with the following methodology:
    • Frequent categories (e.g., key words or objects) from subjects’ dream reports were documented
    • Then, he showed subjects visuals of these categories in wakeful states, and measured brain activity corresponding to these visuals
    • Neural networks were then trained to recognise those brain-wave patterns when similar images occurred during sleep-states
    • Using this method, researchers could reliably predict the contents of the subjects’ dreams

Alternate mindscapes

To say cognition is embodied means that it arises from bodily interactions with the physical world.

In another case, replace ‘the physical world’ with ‘virtual space’: can interactions with and within virtual reality lead to embodied cognition? Rephrased — can physical movement in the real world, reflected as visual representations in an immersive digital environment with sensory feedback, lead to embodied cognition? Several studies suggest this may be the case. What are some levers that could mediate embodied cognition in virtual spaces?

(1) If effective learning occurs through bodily and sensory interactions with the environment, then having granular, precise control over various aspects of a virtual environment could augment learning. 

(2) Gall et. al showed that a bodily representation in virtual spaces increases embodied cognition: owning a virtual body provides means to modulate emotional responses in virtual spaces, and that the illusion of embodiment intensifies the emotional processing of the virtual environment. Immediate implications in tech:

• The presence of artificial bodies & avatars can increase the effectiveness of immersive applications.
• Representative avatars can increase emotional processing in virtual environments - emotional response is an important component of identity, suggesting that effective, immersive VR could contribute to the formation of a preliminary conception of digital identity.

Is it possible to have a persistent sense of self in a digital space, in the same way we do in the physical world? Perhaps not — but is this necessary for embodied cognition? In dream-states we arguably don’t have a persistent self across moments; fractured experiences, it seems, are sufficient for embodied cognition.

It prompts the question — if embodied cognition can occur across modes of existence (physical world, VR, dreams), then how can the difference in subjective experience across these modes be explained in terms of consciousness?

For each mode of existence, we can examine the nature of the space within which consciousness is embedded, and the impact of the environment on phenomenal and access consciousness:

(1) In the physical world, humans have:

✔️A complete, rendered environment

✔️Access consciousness (beliefs, thoughts, & feelings about mental states)

✔️Phenomenal consciousness (a persistent feeling of what it is like to be something)

(2) In non-wakeful dream states, humans have:

➖ An incomplete environment (a model rendered just well enough to convince you that you’re experiencing the world)

❌Access consciousness is minimal or altered - we experience mental states, but exhibit limited metacognition, self-awareness, or logical reasoning based on them, so dream sequences are disjointed & surreal^[2]

✔️Phenomenal consciousness is intact - often dreams allow a fractured but undeniable experience of being an experiencing subject, as they can involve vivid, likelike sensations and emotions

(3) In virtual spaces:

✔️ VR focuses on re-creating high-fidelity, hyper-realistic aspects of the external world (eg: environments, avatars, 3D graphics & vectors, animations, laws of physics)

✔️Humans can engage in immersive virtual experiences - responding to mental states generated through interactions with an immersive env. can indicate Access consciousness.

❓ TBD on phenomenal consciousness — I think the following questions’ answers will inform the conclusion of whether or not we experience phenomenal consciousness in virtual spaces:

• Do we have an emerging persistent digital identity in virtual spaces?
• Are virtual experiences as valuable, meaningful, and authentic as real experiences?

But answering those is for another day.

1. ^{^}

   Still don’t know which specific aspects of my technique changed; just that a qualitative difference was observed in the experience of the action. Perhaps syntax is indeed sufficient for semantics.

2. ^{^}

   Seems like Access consciousness requires identity/having a persistent sense of self through time?