On Mathematics: Thought to Be a Relic of the Past

Mathematical Rigor and Vision at the Frontiers of AGI Engineering

An Seungwon · Wonbrand CEO · August 21, 2026

---

There was a time when I viewed mathematics merely as a tool for entrance exams—a dusty relic of a bygone era. In an age dominated by programming languages and flashy frameworks, clinging to ancient formulas felt hopelessly outdated. Yet, upon diving into the absolute frontier of technology—engineering Artificial General Intelligence (AGI)—the reality I confronted was entirely different. Stripping away the sleek facade of artificial intelligence, what quietly sustains the system underneath is nothing other than pure mathematical logic and binary operations.

Deconstructing the Transformer decoder architecture—the foundation of modern Large Language Models (LLMs)—reveals an extraordinarily precise sequence of mathematical pipelines:

• Embeddings & Positional Encoding (Linear Algebra): Ambiguous human words are mapped into high-dimensional vector spaces (\(d_{model}\)). To compensate for the loss of sequence order during parallel processing, positional encoding formulas inject deterministic coordinates.
• Self-Attention Mechanism (Matrix Operations & Probability): Query, Key, and Value matrices (\(Q, K, V\)) are extracted from input vectors. The dot product of Query and Key is scaled and passed through a Softmax function to compute attention weights. Causal masking (\(-\infty\)) prevents peaking at future tokens, constructing a conditional probability distribution over subsequent words based strictly on prior context.
• Feed-Forward Networks & Backpropagation (Calculus & Optimization): While attention captures token relationships, dense layers (FFN) with non-linear activation functions refine the model’s internal representations. Gradient descent and partial derivatives continuously update model weights to minimize prediction error.

Upon this Transformer framework, researchers have historically leveraged several mathematical foundations to realize early LLMs.

Five Mathematical Frameworks of Early LLMs

• 1. High-Dimensional Representation: Linear Algebra & Dimensionality Reduction
  · Concept: The foundation of linear combinations, dot products, matrix operations, and space transformations.
  · Application: Maps human text into continuous vector spaces and computes projection matrices across various representation heads via Multi-Head Attention.
  · Significance: Enables computers to compute unstructured text within precise, high-dimensional spaces.
• 2. Conditional Generation: Probability & Gibbs Distributions
  · Concept: Probability theory for modeling uncertainty, combined with Boltzmann/Gibbs statistical mechanics formulation.
  · Application: Applies Softmax functions to context vector inner products to yield probability distributions over next tokens, modulated by a temperature parameter.
  · Significance: Drives the selection of the most plausible subsequent words within ambiguous linguistic contexts.
• 3. Continuous Space Optimization: Calculus & Differential Geometry Basics
  · Concept: Partial differentiation for tracking rates of change and optimization techniques across high-dimensional parameter manifolds.
  · Application: Minimizes loss functions across parameter spaces using Backpropagation and Gradient Descent algorithms.
  · Significance: Allows neural networks to autonomously learn data patterns by iteratively reducing training loss.
• 4. Massive Data Storage: Empirical Information Compression
  · Concept: Information theory concepts for quantifying entropy and optimizing data encodings.
  · Application: Memorizes statistical patterns from global web data into parameter matrices via lossy compression.
  · Significance: Stores vast human knowledge internally within a single neural network for real-time retrieval.
• 5. Alignment with Human Intent: Preference Optimization
  · Concept: Decision-theoretical frameworks designed to optimize agent reward signals.
  · Application: Uses techniques like RLHF and DPO to adjust weights according to human preference feedback.
  · Significance: Aligns model outputs with human safety guidelines and behavioral expectations.

The ground truth for these language models stems from the vast collective intelligence recorded in global web pages, articles, and literature. Because human language is fundamentally a domain of social consensus, probabilistic convergence toward crowd consensus was the most viable approach. For a conversational tool, the average of collective intelligence serves as an acceptable proxy for correctness.

However, as we look toward AGI, a fundamental question emerges: Is brute-force scaling of parameters and data truly the sole path to AGI? And is collective consensus always synonymous with truth?

In building AI systems, the most dangerous trap is the circular fallacy of "AI evaluating AI." Models sharing similar inductive biases often uncritically reinforce each other's hallucinations, stamping false claims as truth. An AI trapped within a purely probabilistic generation paradigm cannot self-verify. While social consensus may define language, physical reality operates on exact logic: if a mathematical calculation fails, the bridge collapses, regardless of consensus. For an AGI acting in the physical world, probabilistic hallucinations are not merely linguistic errors—they are catastrophic system failures.

This premise forms the bedrock of my approach to AGI engineering. Rather than relying on brute-force scaling, I advocate for AGI by engineering, not scale. I define the true essence of AGI through four core capabilities:

1. Transfer of thought across disparate fields
2. Resistance to catastrophic forgetting during continuous learning
3. A unique, persistent persona beyond mere superficial style matching
4. Active self-learning that dynamically adapts to external environments

To realize this vision, the primary pillar of my engineering architecture is embedding an uncompromising, independent formal verification layer at the system's core.^[1]

Rather than letting the AI uncritically accept its own outputs, I construct dedicated execution environments designed to rigorously stress-test generated hypotheses. This setup blocks superficial workarounds and integrates Symbolic Logic and SMT solvers for formal verification. Instead of prematurely declaring empirical success as absolute truth after millions of simulations, the system conservatively reports truth only within proved bounds (verified-to-bound).

Beyond formal verification, researchers are exploring various mathematical frameworks to transcend probabilistic limits toward structural control and deterministic verification. Comparing the conventional paradigm with these emerging frontiers highlights this paradigm shift:

These mathematical frontiers represent active domains of exploration aimed at bridging the gap between statistical pattern matching and true general intelligence.

Seven Mathematical Frameworks for Next-Gen AGI Engineering

• 1. Order in Complexity: Category Theory ^[2]
  · Concept: A framework that abstracts and connects structural relationships across distinct mathematical domains.
  · AGI Application: Provides the algebraic foundation for cross-domain thought transfer by mapping shared structural invariants (Functors/Monads) rather than retraining from scratch.
  · Significance: Embeds high-level structural abstraction into systems, mimicking how humans map analogous problems across disciplines.
• 2. Energy-Based Models & Gibbs Distributions (EBM) ^[3]
  · Concept: Formulates system dynamics by assigning scalar energy values to configurations, modeling physical convergence to low-energy states.
  · AGI Application: Replaces pure Softmax probabilities by casting system errors as high-energy states, guiding inference toward low-energy logical configurations.
  · Significance: Serves as a robust control framework that constrains AI reasoning away from unsafe or irrational outputs.
• 3. Active Inference & Free Energy Principle ^[4]
  · Concept: An information-theoretic principle asserting that self-organizing systems act to minimize variational free energy (prediction error).
  · AGI Application: Transitions agents from passive text generators to active entities that interact with environments to refine internal world models (active self-learning).
  · Significance: Offers a rigorous theoretical candidate for unifying perception, learning, and autonomous decision-making.
• 4. Topological Data Analysis (TDA) ^[5]
  · Concept: Utilizes algebraic topology to study the geometric shape and persistent topological features of high-dimensional data.
  · AGI Application: Extracts global structural invariants from high-dimensional representations that remain robust against local noise.
  · Significance: Provides a mathematical foundation for catastrophic forgetting resistance by preserving core structural features during continuous learning.
• 5. Information-Theoretic Optimization: Kolmogorov Complexity ^[6]
  · Concept: Defines the algorithmic information content of an object as the length of its shortest generating program.
  · AGI Application: Shifts the learning objective from empirical memorization toward discovering the most concise generative rules governing the data.
  · Significance: Guides systems toward minimal, causal explanations, embodying an algorithmic implementation of Occam's Razor.
• 6. Equilibrium of Autonomous Agents: Game Theory & Mechanism Design ^[7]
  · Concept: The mathematical study of strategic interactions and equilibrium states (e.g., Nash Equilibrium) among rational decision-makers.
  · AGI Application: Coordinates internal sub-agents to resolve conflicting sub-goals and maintain optimal collaborative stability.
  · Significance: Prevents system divergence and supports the maintenance of a stable, coherent unique persona.
• 7. Navigation in Curved Spaces: Differential Geometry ^[8]
  · Concept: The study of geometry on smooth manifolds, Riemannian metrics, and geodesic trajectories.
  · AGI Application: Replaces standard Euclidean gradient optimization with natural gradient updates along curved parameter manifolds.
  · Significance: Mitigates optimization distortions in high-dimensional non-Euclidean loss landscapes.

Ultimately, these advanced mathematical tools convergence upon a single principle: Mathematics in the AGI era is not merely a backend optimization tool for training neural networks; it is the rigorous arbiter overlaying intuitive architectures and the foundation of robust World Models.

Achieving cross-domain transfer, resistance to forgetting, persona persistence, and active self-learning through engineering rigor represents my core vision for AGI. Building systems that resist hallucinations requires unifying probabilistic intuition with formal mathematical logic—a frontier we continue to actively engineer.

In a noisy world, mathematical proof remains one of the few avenues to uncompromised truth. As an AGI engineer, my daily work extends beyond scaling compute; it is an effort to anchor intelligent systems upon the unwavering foundation of mathematical rigor—the profound clarity found in 1s and 0s.

---

[References & Footnotes]

1. De Moura, L., & Bjørner, N. (2008). Z3: An efficient SMT solver. TACAS, 337-340. (Formal verification via SMT solvers)
2. Fong, B., & Spivak, D. I. (2019). An Invitation to Applied Category Theory: Seven Sketches in Compositionality. Cambridge University Press. (Category theory for compositional transfer learning)
3. LeCun, Y., et al. (2006). A tutorial on energy-based learning. Predicting Structured Data. (Energy-Based Models)
4. Friston, K. (2010). The free-energy principle: a unified brain theory?. Nature Reviews Neuroscience, 11(2), 127-138. (Active Inference and Free Energy)
5. Carlsson, G. (2009). Topology and data. Bulletin of the AMS, 46(2), 255-308. (Topological Data Analysis)
6. Li, M., & Vitányi, P. (2008). An Introduction to Kolmogorov Complexity and Its Applications. Springer. (Algorithmic Information Theory)
7. Shoham, Y., & Leyton-Brown, K. (2008). Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press. (Multi-agent Game Theory)
8. Bronstein, M. M., et al. (2017). Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4), 18-42. (Geometric Deep Learning and Manifolds)

An Seungwon / Wonbrand / https://wonbrand.co.kr