On Mathematics: Thought to Be a Relic of the Past
Mathematical Rigor and Vision at the Frontiers of AGI Engineering
The Relic That Turned Out to Be the Foundation
There was a time when the author saw mathematics as nothing more than a tool for entrance exams — a dusty relic of a bygone era, hopelessly outdated in an age of programming languages and flashy frameworks. Diving into the absolute frontier of technology, the engineering of Artificial General Intelligence, produced the opposite conclusion. Strip away the sleek facade of artificial intelligence and what quietly sustains the system underneath is nothing other than pure mathematical logic and binary operations.
The Transformer Decoder as a Precise Mathematical Pipeline
At the base sit embeddings and positional encoding, drawn from linear algebra: ambiguous human words are mapped into a high-dimensional vector space of the model's dimension, and positional encoding formulas inject deterministic coordinates to restore the sequence order lost to parallel processing. Above that, self-attention works in matrix operations and probability — Query, Key and Value matrices are extracted from the input vectors, the dot product of Query and Key is scaled and passed through a softmax function to yield attention weights, while causal masking pushes future positions to negative infinity so the model builds a conditional probability distribution over the next word from prior context alone. At the top, feed-forward layers with non-linear activation functions refine the internal representations, and gradient descent with partial derivatives continuously updates the weights to minimize prediction error.
Five Frameworks of Early LLMs: Algebra, Probability, Calculus, Compression, Preference
The list names, in order: high-dimensional representation through linear algebra and dimensionality reduction, which maps text into continuous vector spaces and computes projection matrices across multi-head attention; conditional generation through probability and Gibbs distributions, applying softmax to context-vector inner products with a temperature parameter to pick the most plausible next word; continuous-space optimization through calculus and the basics of differential geometry, minimizing loss across parameter spaces by backpropagation and gradient descent; massive data storage as empirical information compression, memorizing statistical patterns from global web data into parameter matrices by lossy compression; and alignment with human intent through preference optimization, using techniques such as RLHF and DPO to fit weights to human feedback. The author adds that the ground truth for these models is the collective intelligence recorded in the world's web pages, articles and literature — and since language is a domain of social consensus, probabilistic convergence toward crowd consensus was the most viable approach. For a conversational tool, the average of collective intelligence serves as an acceptable proxy for correctness.
Seven Tool-Groups for Next-Generation AGI, from Category Theory to Curved Spaces
The seven are: category theory, which abstracts shared structural invariants through functors and monads to give an algebraic basis for cross-domain thought transfer; energy-based models with Gibbs distributions, which recast system errors as high-energy states and steer inference toward low-energy logical configurations; active inference and the free energy principle, which turn passive text generators into agents that act on their environment to minimize prediction error; topological data analysis, which extracts global structural invariants robust to local noise and thereby underwrites resistance to catastrophic forgetting; Kolmogorov complexity, which shifts the learning objective from memorization toward the shortest generating program, an algorithmic Occam's Razor; game theory and mechanism design, which coordinate internal sub-agents around equilibrium states such as Nash equilibrium to hold a stable persona; and differential geometry, which replaces flat Euclidean gradient steps with natural gradient updates along curved Riemannian manifolds. A comparison table sets each against its counterpart in the probabilistic generation paradigm — lossy compression against rule induction, vector coordinates against topological invariants, softmax conditional probability against energy levels, passive response against active sampling, fine-tuning against functorial transfer, standard gradient descent against curvature-aware optimization, and preference fitting against multi-agent equilibrium design. The author frames these as active domains of exploration aimed at bridging statistical pattern matching and true general intelligence.
AGI by Engineering, Not Scale — and the Four Elements of True AGI
The highlighted box states the premise that forms the bedrock of the author's approach: rather than brute-force scaling of parameters and data, he advocates AGI by engineering, not scale. He defines the true essence of AGI through four core capabilities — transfer of thought across disparate fields, resistance to catastrophic forgetting during continuous learning, a unique and persistent persona beyond superficial style matching, and active self-learning that dynamically adapts to the external environment. The primary pillar of the architecture that follows is an uncompromising, independent formal verification layer at the system's core: dedicated execution environments stress-test generated hypotheses, symbolic logic and SMT solvers block superficial workarounds, and the system reports truth only within proved bounds rather than declaring empirical success after millions of simulations. Behind this stands his warning about the circular fallacy of AI evaluating AI, where models with similar inductive biases stamp each other's hallucinations as truth — tolerable in language, catastrophic for an AGI acting in a physical world where a failed calculation collapses the bridge regardless of consensus.
Proof as the Last Uncompromised Avenue to Truth
All these advanced tools converge on a single principle: mathematics in the AGI era is not merely a backend optimization utility for training neural networks, but the rigorous arbiter overlaying intuitive architectures and the foundation of robust world models. Achieving cross-domain transfer, resistance to forgetting, persona persistence and active self-learning through engineering rigor is the author's core vision, and building systems that resist hallucination means unifying probabilistic intuition with formal mathematical logic. In a noisy world, he writes, mathematical proof remains one of the few avenues to uncompromised truth — so his daily work as an AGI engineer extends beyond scaling compute to anchoring intelligent systems on that unwavering foundation, the profound clarity found in ones and zeros.
Insights
Learn more >
