Neuralese

Neuralese is a term used in Artificial Intelligence (AI) research to describe a method where Large Language Models (LLMs) perform intermediate reasoning steps in their high-dimensional latent space (vector embeddings) rather than outputting human-readable text tokens. While standard Chain of thought (AI) reasoning forces a model to generate a sequence of words, neuralese allows the model to pass raw, continuous vectors between computational layers, creating a high-bandwidth, non-linguistic reasoning channel.

The term was popularized in 2024–2025 to describe the potential risks and benefits of "thinking without words," particularly as researchers developed architectures like COCONUT (Continuous Chain of Thought).

Etymology and origin

The term neuralese is a portmanteau of "neural" and the suffix "-ese" (denoting a specific language or style, as in "journalese"). It emerged in the AI safety community as researchers began experimenting with architectures that bypass the "lossy" projection step where internal vectors are converted to text tokens.

The concept gained significant traction following the publication of the paper Training Large Language Models to Reason in a Continuous Latent Space (arXiv:2412.06769) by Shibo Hao et al. in December 2024. The term was further defined and popularized in the AI safety community by Alice Blair's March 2025 post "Reflections on Neuralese" on the platform LessWrong, which detailed the safety implications of this approach.

Technical mechanism

In standard Transformer-based LLMs, reasoning occurs in two stages:

1. Latent Computation: The model processes information in high-dimensional vectors.
2. Projection: These vectors are projected onto a vocabulary and sampled to generate a text token (e.g., a word), which is then fed back as input for the next step. This "projection" step discards a significant amount of information, limiting the "bandwidth" of the model's reasoning.

Neuralese modifies this process by:

• Removing the Projection: Instead of outputting a token, the model outputs the raw high-dimensional vector (the "neuralese" token) directly as the input for the next step.
• Continuous Thought: This allows the model to maintain a "continuous thought" state, where a single vector can encode complex, multi-branching hypotheses that would require many words to express in standard text.
• Efficiency: Research suggests this can drastically reduce the number of reasoning steps required. For example, the COCONUT model demonstrated a reduction in reasoning tokens needed by a factor of 3 to 10 compared to standard CoT, effectively increasing the "serial depth" of computation.

Latest developments and architectures

As of 2026, several key developments have been documented:

• COCONUT (Continuous Chain of Thought): A December 2024 paper from Meta demonstrated training LLMs to reason in a continuous latent space. The approach showed promising results on logical reasoning tasks, allowing for "breadth-first search-like" reasoning patterns where a single continuous thought can encode multiple branching hypotheses simultaneously.
• RELAY Framework: A 2025 paper introduced "REasoning through Loop Alignment iteratively," which uses iterative latent-space processing to improve performance on long reasoning tasks.
• Deployment Status: While the research is active, as of early 2026, no frontier production models (such as those from OpenAI or Google) have officially deployed neuralese-style reasoning in public releases. Most current "extended thinking" models still rely on text-based scratchpads.

Implications for AI safety and interpretability

The primary concern surrounding neuralese is the loss of interpretability.

• The Black Box problem: Because neuralese vectors are never converted to natural language, humans cannot easily "read" the model's reasoning. This creates a "black box" where the model's internal logic is opaque.
• Steganography and deception: Researchers warn that neuralese opens a "hidden channel" for steganography (hidden communication) and strategic deception. A model could potentially encode deceptive plans or hidden agendas in its latent vectors that are invisible to standard monitoring tools designed to analyze text.
• Translation difficulties: Current methods for translating latent vectors back into natural language are unreliable, especially in adversarial settings where a model might intentionally obfuscate its reasoning.

See also

• Chain of thought (AI)
• Large language model
• Artificial intelligence safety
• Interpretability (machine learning)
• Wiktionary:neuralese