The AI Pendulum

For seventy years, AI has swung between two ideas of the mind: reasoning as rules over symbols, and learning as tuning connections from data. A third, quieter camp has asked a different question all along: what is a mind from the point of view of physics and biology? This essay traces the swings and offers our view of where the next one may lead.

· Kirill Kazakov, Founder & CEO, HiHolo · 11 min read

Early computing machines (vacuum tubes, a tape drive and a mainframe console) drawn in grey line art, linked by thin blue traces to a glowing network sphere.
From vacuum tubes to networks: the idea of a thinking machine is as old as the computer itself. Illustration: HiHolo.

Beyond language models

Since ChatGPT arrived in 2022, many people have come to treat "AI" and "large language models" as synonyms, and to bet that transformers will open new horizons for every human problem. We don't think that's right. Artificial intelligence is a much broader idea. Its roots, in Turing's work and in cybernetics, reach back to the first half of the twentieth century, although AI only took shape as a discipline at the Dartmouth workshop of 1956 [5].

Origins

The story starts in the 1940s. In 1943 Warren McCulloch and Walter Pitts proposed a model of the formal neuron and showed that networks of such simple elements can compute logical functions [1], the foundation of the whole neural-network tradition. At the same time Norbert Wiener was building cybernetics, the science of control and communication in animals and machines [2]. In 1950 Alan Turing's "Computing Machinery and Intelligence" in Mind replaced the vague question "Can machines think?" with a concrete test, the "imitation game", in which a judge must tell from conversation alone whether they face a person or a machine [3]. The same year Claude Shannon published a paper on programming a computer to play chess, laying the groundwork for game-tree search [4].

Two camps

Two rival traditions grew quickly from these roots. Wiener's cybernetics, with its focus on feedback and homeostasis, framed brains and machines as self-regulating systems and suggested that mind emerges from many simple, connected elements. That line, from McCulloch and Pitts through Frank Rosenblatt's perceptron [7], became connectionism.

The other tradition was shaped in the 1950s by Allen Newell, Herbert Simon and John McCarthy, the people behind the Logic Theorist and the Dartmouth project [5][6]. It rested on Turing's idea of a universal computer: if thinking is symbol manipulation by rules, it is enough to write down the rules. This became symbolic AI, built on logic, search trees and formal knowledge representation. In 1985 the philosopher John Haugeland named it GOFAI, "good old-fashioned AI", not without irony [11].

"Mind as connections and learning" versus "mind as computation over symbols" became the axis of the field's history. At first the symbolists led, and in 1969 Marvin Minsky and Seymour Papert's book Perceptrons dealt neural networks a famous blow [8]. Connectionism spent most of the next two decades in the shadows.

A glowing network of nodes on the left and a stack of logic-marked gears on the right, with paths from both meeting at one central node.
Two answers to one question: mind as learned connections, or mind as rules over symbols. Illustration: HiHolo.

Expert systems and the second winter

Meanwhile symbolic AI had its golden age, and expert systems were its peak. If a specialist's knowledge can be written as "if–then" rules, the idea went, a machine holding those rules should reason like a doctor, a geologist or a chemist. MYCIN, which diagnosed bacterial infections, used a few hundred rules [9]; XCON, which configured computer orders for Digital Equipment Corporation, had grown to around ten thousand by the late 1980s [10]. They showed for the first time that AI could make real money, and corporations, then governments, rushed in.

The weakness soon showed: knowledge had to be extracted by hand from experts and kept current, and every new rule raised the risk of contradictions. This was the knowledge-acquisition bottleneck. By the end of the decade expert systems had hit a ceiling. The Lisp-machine market collapsed, and large state programmes such as Japan's Fifth Generation project fell short of their goals (it formally closed in 1992). The second "AI winter" set in, and with it came doubt that mind could be captured by symbolic rules alone.

The connectionist comeback

Neural networks never fully disappeared. Bernard Widrow and Ted Hoff's adaptive linear elements (ADALINE, later MADALINE) led to adaptive filters that cancel echoes on telephone lines, an early practical use of trainable networks [12]. In 1986 David Rumelhart and James McClelland published Parallel Distributed Processing [14], and a paper by Rumelhart, Geoffrey Hinton and Ronald Williams popularised backpropagation [15]. The method had been described earlier, notably in Paul Werbos's 1974 doctoral thesis [13], but this work showed it was effective in practice. Multilayer networks, whose prospects Minsky and Papert had judged bleak, could now be trained.

Rules no longer had to be written by hand: show a network examples and it tunes its own weights. Knowledge came from data rather than from experts' heads, so the bottleneck never arose, and "rules versus connections" became a real question about where intelligent behaviour comes from: formal logic, or the statistics of experience.

The decisive turn came in 2012. AlexNet, the convolutional network of Alex Krizhevsky, Ilya Sutskever and Hinton, was trained on GPUs on over a million labelled images and cut the ImageNet top-5 error from about 26% to about 15% [16]. In 2017 "Attention Is All You Need" introduced the transformer [17], the architecture behind BERT and GPT [18][19], and in 2022 ChatGPT brought large language models to a mass audience [20]. Connectionism went from outsider to mainstream, and the pendulum seemed to rest on the side of connections. But the story always had a third participant.

The third camp: life and physics

From the mid-twentieth century, researchers such as Heinz von Foerster, and later Humberto Maturana and Francisco Varela, asked how living systems process information and maintain their internal state [21][22]. In the 2000s and 2010s the neuroscientist Karl Friston developed this line into a physical theory of the brain.

Three streams (a rule diagram, a neural network and cell-like organic shapes) braiding into a single glowing knot.
Rules, networks and living matter: the third camp asks what a mind is from the standpoint of physics and biology. Illustration: HiHolo.

This view is not a branch of connectionism but a third position. The symbolists asked which rules describe thinking; the connectionists asked how to tune connections from examples. This camp asks what a mind is from the standpoint of the physics of living things.

Friston, who formulated the free-energy principle in the mid-2000s [23], sees the brain neither as a calculator nor as a trainable map of weights, but as a system that continually minimises free energy, a quantity that bounds "surprise", the mismatch between its predictions and the signals from its environment [24]. Perception and action serve the same end: predict the inputs and keep prediction error low. In later work Friston and colleagues put this in geometric terms: a system's internal states form a space of beliefs about the outside world with its own measure of distance, an information geometry, and inference becomes movement through that space [25]. Intelligence becomes a property of any self-sustaining system that resists disorder, from a bacterium to a human, silicon or not. Instead of chasing benchmarks, the camp tries to derive the principles a mind must obey because of physical law, as thermodynamics describes steam engines without knowing their designs. Critics charge that the theory is unfalsifiable, closer for now to a philosophical framework than to a testable theory with predictive power.

Khrennikov's quantum-like models

A more radical step is to borrow the formalism of quantum theory. In the 2000s the Russian-Swedish mathematician Andrei Khrennikov of Linnaeus University developed what he calls quantum-like modelling: the mathematics of quantum mechanics (probability amplitudes, superposition, non-commuting observables) applied to decision-making, perception and cognition, wherever classical probability breaks down. He set out the programme in books including Contextual Approach to Quantum Formalism (2009) [26] and Ubiquitous Quantum Structure (2010) [27].

Cognitive psychology does record violations of classical probabilistic reasoning. In a well-known Gallup poll, the share of respondents who called Bill Clinton honest and trustworthy depended on whether the same question about Al Gore came first [28]. Quantum-like models describe such question-order effects naturally. Khrennikov stresses that he is not claiming the brain runs on quantum processes; quantum logic serves as a mathematical language for systems whose states cannot be split into a classical sum of alternatives. If mind reduces neither to rules nor to weights, perhaps the logic used to describe it has to change.

Levin and basal cognition

Michael Levin of Tufts University questions the boundary between mind and life itself. He studies bioelectric signalling: tissues "talk" through differences in electrical potential, and these signals help determine how an organism builds its shape. By changing tissues' bioelectric patterns, his lab has made planarian flatworms grow a second head [30], and in another study a 24-hour treatment with a five-drug cocktail triggered regrowth of a lost leg in adult frogs [31].

Hence his thesis of "basal cognition" [32]: cells and groups of cells solve problems, store information about a target shape and act to reach it. Mind is not a privilege of brains but a property distributed, in varying degrees, across living matter. Through this lens "Can machines think?" loses its edge: a single cell already solves some problems, at a scale alien to us.

A wireframe landscape of hills and a deep blue basin, with a sphere rolling down a slope towards the minimum.
Learning as relaxation: a system settling into a minimum of an energy landscape. Illustration: HiHolo.

Vanchurin and Katsnelson: learning as physics

Mikhail Katsnelson is a leading theoretical physicist of graphene and condensed matter; Vitaly Vanchurin is known for the provocative paper "The World as a Neural Network" (2020) [33]. Their shared question: can learning be described not as an engineering algorithm but as a physical process?

In "Towards a Theory of Machine Learning" (2020) Vanchurin described neural-network training in the language of statistical mechanics, with analogues of entropy, temperature and the laws of thermodynamics [34]. In "Emergent Quantumness in Neural Networks" (2021), he and Katsnelson argued that an ensemble of learning networks in learning equilibrium is described by the equations of quantum mechanics [35]. A 2022 PNAS paper with the biologists Yuri Wolf and Eugene Koonin extended the approach to evolution, treated as multilevel learning [36]. Here a network is a physical object that reaches a solution the way a crystal relaxes to an energy minimum: mind as a property of matter, applied to artificial systems.

Two routes into physics

Khrennikov and the Vanchurin–Katsnelson line both refuse the language of rules or weights and both turn to fundamental physics, but to different branches. One uses statistical mechanics, where learning is the evolution of a material system; the other uses quantum formalism, with its non-commutative, contextual logic, deliberately kept apart from physically quantum-brain hypotheses such as those of Roger Penrose and Stuart Hameroff [29]. The first says mind is how matter organises itself against disorder; the second, how a system handles incompatible alternatives.

A 2024 paper showed that large language models doing in-context learning violate the martingale property that classical Bayesian inference requires [37], so which probabilistic logic describes such systems becomes a practical question. The two approaches may be two slices of one problem: in Vanchurin and Katsnelson's work, the quantum description itself emerges from the statistics of learning. Both aim to show that intelligence can be derived from first principles rather than assembled from engineering tricks. Look closely and all these lines, from Friston to Khrennikov, speak one language: that of spaces, their shape and connectivity. We will come back to this.

The pendulum swings back

Inside the mainstream, the pendulum is ready to swing back, this time because of the winner's own contradictions. Large language models brought new ailments: hallucinations, where a model confidently produces plausible but false information; fragile multi-step reasoning; and unreliability with formal structures such as mathematics, program verification and strict logical inference. This echoes the symbolists' old complaint: statistics capture form well but guarantee correctness poorly.

So neurosymbolic hybrids are back: a neural network for perception and generation, formal logic for checking and reasoning. The most striking example is Google DeepMind's AlphaProof and AlphaGeometry 2: in 2024, neural networks paired with formal and symbolic engines solved four of six International Mathematical Olympiad problems, at silver-medal level [38][39]. AI's history moves not in a straight line from rules to connections but along a pendulum's arc, where each extreme produces its counterweight.

Where the third camp points

The bio-physical tradition, from von Foerster, Maturana and Varela to Friston, Levin, Vanchurin and Katsnelson, doesn't fit "rules versus connections" at all. For it, weights and rules are just two descriptions of a system whose behaviour really comes from physics and biology.

From this camp's perspective, LLM hallucinations are not a defect to patch with an external verifier but a symptom. Living cognitive systems, from cells to brains, constantly check their predictions against the world and minimise the mismatch; a language model generating text without such a check simply isn't solving that task. That gives the pendulum a third pivot, one neither main camp noticed, and we suspect it will be the main challenge of the next cycle.

Friston became chief scientist at VERSES, which builds AI on active-inference principles as an alternative to training by backpropagation [40]. Levin directs the Allen Discovery Center at Tufts, where steering cells' bioelectric signals turns "intelligence without a brain" into a research programme. Even the mainstream is looking beyond backpropagation: in 2022 Geoffrey Hinton proposed the Forward-Forward algorithm, which trains a network without it [41]. In our view, the next swing may go not to rules or connections but toward that third pivot.

Turing's mirror

Turing thought "Can machines think?" too vague to discuss, which is why he replaced it with the imitation game. Each camp answered it anyway. For the symbolists, a machine thinks if it derives correct conclusions by rules; for the connectionists, if it has learned from enough examples. The bio-physical camp says the question is badly posed: in some sense any system that keeps itself in agreement with the world thinks, whether a cell, a brain or a physical network.

The Turing test is not a criterion but a mirror. It says nothing about how a mind works inside; it only records that, more than seventy years on, we still argue about mind at the level of outward behaviour. A language model that passes it sharpens the question rather than settling it: learned statistics for one camp, unfinished symbolic inference for another, a physical system that doesn't check itself against the world for the third. Perhaps "Can machines think?", like any good test, examines the person asking. The next swing will come not when another approach wins, but when we ask what exactly we mean by thinking.

A circular dial divided into logic circuitry, a node network and organic roots, with a blue pendulum swinging across its centre.
Rules, connections, life and physics: the pendulum keeps swinging between the three. Illustration: HiHolo.

Shape and connectivity

The pendulum keeps swinging, but behind the succession of rules, weights, free energy and quantum logic something deeper shows through. Physics has been here before: gravity turned out to be the geometry of spacetime [42], and some of the most robust properties of matter turned out to be topological, independent of small details of how a system is built [43]. We believe the basis of mind is the same: topology and geometry, the shape and connectivity of the space in which a system builds and maintains its model of the world. Friston's information geometry, and the quantumness that Vanchurin and Katsnelson see emerging from a learning network, are already steps in this direction. In our view, research will sooner or later arrive here, whichever side of the pendulum it starts from. This is the direction we work in at HiHolo: HoloEngine's reasoning is built on geometric (p-adic) logic, with data organised on a hyperbolic tree.

Where we stand

HiHolo builds HoloEngine, deterministic and explainable AI: learning, reasoning, optimization and verification happen in one engine, with data, logic and proof in one shared state, so decisions can be reproduced and traced back to their reasons. Next to neural models its role is specific: LLMs and VLA models propose; HoloEngine checks and executes. HiHolo publicly describes HoloEngine as an energy-based model [44] built on geometric (p-adic) logic, set out in Kirill Kazakov's paper "Geometric Logic on the Invariant p-adic Twin" (2026) [45]. Our robotics results so far come from simulation experiments, and we are in the process of a real-robot pilot: hiholo.ai/robotics.

References

Origins: 1940s–1950s

  1. McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5(4), 115–133. https://doi.org/10.1007/BF02478259
  2. Wiener, N. (1948). Cybernetics: Or Control and Communication in the Animal and the Machine. The Technology Press (MIT) / John Wiley & Sons; Hermann, Paris.
  3. Turing, A. M. (1950). Computing Machinery and Intelligence. Mind, 59(236), 433–460. https://doi.org/10.1093/mind/LIX.236.433
  4. Shannon, C. E. (1950). Programming a computer for playing chess. Philosophical Magazine, 41(314), 256–275. https://doi.org/10.1080/14786445008521796
  5. McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (1955). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence. Reprinted in AI Magazine, 27(4), 2006. https://doi.org/10.1609/aimag.v27i4.1904
  6. Newell, A., & Simon, H. A. (1956). The logic theory machine: A complex information processing system. IRE Transactions on Information Theory, 2(3), 61–79. https://doi.org/10.1109/TIT.1956.1056797
  7. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408. https://doi.org/10.1037/h0042519

Symbolic AI and expert systems

  1. Minsky, M., & Papert, S. (1969). Perceptrons: An Introduction to Computational Geometry. MIT Press. https://mitpress.mit.edu/9780262130431/perceptrons/
  2. Shortliffe, E. H. (1976). Computer-Based Medical Consultations: MYCIN. Elsevier. https://www.shortliffe.net/Shortliffe-1976/MYCIN%20thesis%20Book.htm
  3. McDermott, J. (1982). R1: A rule-based configurer of computer systems. Artificial Intelligence, 19(1), 39–88. https://doi.org/10.1016/0004-3702(82)90021-2
  4. Haugeland, J. (1985). Artificial Intelligence: The Very Idea. MIT Press. https://mitpress.mit.edu/9780262081535/artificial-intelligence/

The return of neural networks and deep learning

  1. Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. IRE WESCON Convention Record, Part 4, 96–104. https://isl.stanford.edu/~widrow/papers/c1960adaptiveswitching.pdf
  2. Werbos, P. J. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University. Reprinted in The Roots of Backpropagation (Wiley, 1994).
  3. Rumelhart, D. E., McClelland, J. L., & the PDP Research Group (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition (2 vols.). MIT Press. https://doi.org/10.7551/mitpress/5236.001.0001
  4. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. https://doi.org/10.1038/323533a0
  5. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25. https://papers.nips.cc/paper_files/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
  6. Vaswani, A., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems 30. https://arxiv.org/abs/1706.03762
  7. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805. https://arxiv.org/abs/1810.04805
  8. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving Language Understanding by Generative Pre-Training. OpenAI. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
  9. OpenAI (30 November 2022). Introducing ChatGPT. https://openai.com/index/chatgpt/

The third camp: biology and the physics of mind

  1. von Foerster, H. (1981). Observing Systems. Intersystems Publications.
  2. Maturana, H. R., & Varela, F. J. (1980). Autopoiesis and Cognition: The Realization of the Living. Reidel. https://doi.org/10.1007/978-94-009-8947-4
  3. Friston, K., Kilner, J., & Harrison, L. (2006). A free energy principle for the brain. Journal of Physiology – Paris, 100(1–3), 70–87. https://doi.org/10.1016/j.jphysparis.2006.10.001
  4. Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11(2), 127–138. https://doi.org/10.1038/nrn2787
  5. Parr, T., Da Costa, L., & Friston, K. (2020). Markov blankets, information geometry and stochastic thermodynamics. Philosophical Transactions of the Royal Society A, 378(2164), 20190159. https://pmc.ncbi.nlm.nih.gov/articles/PMC6939234/
  6. Khrennikov, A. (2009). Contextual Approach to Quantum Formalism. Springer. https://doi.org/10.1007/978-1-4020-9593-1
  7. Khrennikov, A. (2010). Ubiquitous Quantum Structure: From Psychology to Finance. Springer. https://doi.org/10.1007/978-3-642-05101-2
  8. Moore, D. W. (2002). Measuring New Types of Question-Order Effects: Additive and Subtractive. Public Opinion Quarterly, 66(1), 80–91. https://doi.org/10.1086/338631
  9. Hameroff, S., & Penrose, R. (2014). Consciousness in the universe: A review of the 'Orch OR' theory. Physics of Life Reviews, 11(1), 39–78. https://doi.org/10.1016/j.plrev.2013.08.002
  10. Durant, F., Morokuma, J., Fields, C., Williams, K., Adams, D. S., & Levin, M. (2017). Long-Term, Stochastic Editing of Regenerative Anatomy via Targeting Endogenous Bioelectric Gradients. Biophysical Journal, 112(10), 2231–2243. https://pmc.ncbi.nlm.nih.gov/articles/PMC5443973/
  11. Murugan, N. J., et al. (2022). Acute multidrug delivery via a wearable bioreactor facilitates long-term limb regeneration and functional recovery in adult Xenopus laevis. Science Advances, 8(4), eabj2164. https://doi.org/10.1126/sciadv.abj2164
  12. Levin, M. (2022). Technological Approach to Mind Everywhere: An Experimentally-Grounded Framework for Understanding Diverse Bodies and Minds. Frontiers in Systems Neuroscience, 16, 768201. https://doi.org/10.3389/fnsys.2022.768201
  13. Vanchurin, V. (2020). The World as a Neural Network. Entropy, 22(11), 1210. https://pmc.ncbi.nlm.nih.gov/articles/PMC7712105/
  14. Vanchurin, V. (2020). Towards a theory of machine learning. arXiv:2004.09280. Published as "Toward a theory of machine learning", Machine Learning: Science and Technology, 2(3), 035012 (2021). https://arxiv.org/abs/2004.09280
  15. Katsnelson, M. I., & Vanchurin, V. (2021). Emergent Quantumness in Neural Networks. Foundations of Physics, 51(5). arXiv:2012.05082. https://arxiv.org/abs/2012.05082
  16. Vanchurin, V., Wolf, Y. I., Katsnelson, M. I., & Koonin, E. V. (2022). Toward a theory of evolution as multilevel learning. PNAS, 119(6), e2120037119. https://pmc.ncbi.nlm.nih.gov/articles/PMC8833143/

The limits of large language models, and what comes next

  1. Falck, F., Wang, Z., & Holmes, C. (2024). Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective. Proceedings of the 41st International Conference on Machine Learning, PMLR 235. https://proceedings.mlr.press/v235/falck24a.html
  2. Trinh, T. H., Wu, Y., Le, Q. V., He, H., & Luong, T. (2024). Solving olympiad geometry without human demonstrations. Nature, 625, 476–482. https://doi.org/10.1038/s41586-023-06747-5
  3. Google DeepMind (25 July 2024). AI achieves silver-medal standard solving International Mathematical Olympiad problems. https://deepmind.google/blog/ai-solves-imo-problems-at-silver-medal-level/
  4. VERSES (1 December 2022). Karl Friston joins VERSES as Chief Scientist to Lead New Era in Artificial Intelligence. Press release, GlobeNewswire. https://www.globenewswire.com/news-release/2022/12/01/2565923/0/en/karl-friston-joins-verses-as-chief-scientist-to-lead-new-era-in-artificial-intelligence.html
  5. Hinton, G. (2022). The Forward-Forward Algorithm: Some Preliminary Investigations. arXiv:2212.13345. https://arxiv.org/abs/2212.13345

Physics: geometry and topology

  1. Einstein, A. (1915). Die Feldgleichungen der Gravitation. Sitzungsberichte der Königlich Preußischen Akademie der Wissenschaften, 844–847. https://edoc.bbaw.de/opus4-bbaw/frontdoor/deliver/index/docId/5270/file/BBAW_SB_1915_TB2_S844_847.pdf
  2. The Nobel Prize in Physics 2016: David J. Thouless, F. Duncan M. Haldane and J. Michael Kosterlitz, "for theoretical discoveries of topological phase transitions and topological phases of matter". https://www.nobelprize.org/prizes/physics/2016/summary/

HiHolo

  1. HiHolo (May 2026). Energy-based model explainer. https://hiholo.ai/ebm
  2. Kazakov, K. (2026). Geometric Logic on the Invariant p-adic Twin. Preprint, Zenodo. https://doi.org/10.5281/zenodo.19391085
← Back to HiHolo