The Discipline of Wonder
When I began my career in the early 90s artificial intelligence was emerging from its "winter". The field was shifting from grand ambitions towards the more modest pursuit of practical solutions. Questions about knowledge acquisition, representation, and compute left many wondering whether machines could ever truly understand.
As a PhD student I found myself repelled by the idea that complex intelligence systems should be built bottom up. Instead I found myself drawn to a simple question: could we create the machinery to leverage word associations for intelligent applications?
Inspired by the statistical approaches coming from IBM's machine translation group, and an intuition about how intelligence might be modelled, I looked towards neural networks. These temperamental machines possessed an elegant simplicity. Among them, Hopfield networks fascinated me most. Borrowing from statistical physics, they treated learning as the sculpting of an energy landscape, where understanding emerged from the discovery of stable states. Intelligence appeared not as something engineered, but as something that arose naturally from structure.
My supervisor's interest in machine translation shaped my focus: how could these networks bridge the conceptual gaps between English and Japanese? The results were modest - two papers presented to polite conference audiences, minimal citations. Yet something profound revealed itself in those experiments. I watched the network distinguish between two senses of 'background', the setting in a picture versus someone's history, without being explicitly taught. It had discovered, through pure association, a semantic distinction. Here was meaning emerging from pattern alone, without formal grammars or encoded rules.
That idea stayed with me. Over the years my research shifted toward statistical machine learning, yet the central question remained: how could structure and meaning emerge from data rather than be imposed upon it?
This principle has re-emerged powerfully in today's large language models, which demonstrate at massive scale that human-like understanding can crystallize from patterns we never explicitly program. When Hopfield and Hinton received the Nobel Prize in 2024, it felt like recognition not only of their revolutionary contributions but of a way of thinking about intelligence that values emergence over prescription.
Between my small experiments and today's breakthroughs lies something worth preserving: the willingness to follow curiosity into uncertain territory, to sometimes value insight over impact factor, to recognize that understanding unfolds on its own timeline. Easy to say as unlike today's competitive urgency, early connectionism offered clear blue water for patient exploration. But nevertheless an enduring idea.
Looking back, I see my early work not as a missed opportunity but as participation in something larger. Citations are validating but satisfaction can come from one's own personal journey of understanding.
In our era of instant commentary, wonder persists only when we resist the pressure to react to everything we read. Some of the questions that matter most, how meaning emerges, how understanding forms, how intelligence regulates itself, cannot be rushed. They require what makes science possible in the first place: the discipline to see clearly, even when clarity reveals our own limitations.
