On October 8, U of T Computer Science Professor Emeritus Geoffrey Hinton was sitting in a cheap hotel room in California when he received a call — he had just been named as a co-winner of the 2024 Nobel Prize in physics. Known to many as the ‘Godfather of AI,’ Hinton was notably shocked to receive the award. 

The Nobel Foundation awarded the prize to both Hinton and John J. Hopfield, an American physicist and professor emeritus of Princeton University. According to the official Nobel Prize press release, they were selected “for foundational discoveries and inventions that enable machine learning with artificial neural networks.” 

Neurons, networks, and nodes

A neural network refers to interconnected systems of neurons — or nerve cells — in the brain. Artificial neural networks (ANNs) are machine learning models inspired by our own complex biological neural network. Each ANN is made of layers of interconnected artificial neurons, or nodes.

The strength of these nodes can be trained to select for stronger connections between nodes of similar weight or importance. The more important the node is, the stronger its connections will be. In a way, the strength of each connection is like a weighted average for a course — each assignment or exam is worth a certain amount, but doing better on one assignment will also impact the outcome of the others. ‘Training’ an ANN is what makes the network seem like artificial ‘intelligence’ (AI). If you can teach something to a machine, it can learn and evolve. 

The Hopfield network: A spin on associative memory

In 1982, Hopfield created a neural network using the concept of spin in physics, in which the total energy of a spin system is the sum of the weighted values of each part of the system. 

Spin is the atomic property that makes it so that each atom is like a tiny magnet, and is attached to other atoms. The spin of neighbouring atoms can affect each other, making materials behave differently depending on what they’re made of. Similarly, the total energy in the network is equated to the total energy of a spin model; where each node has a value and the nodes’ connections vary in strength, which affects how the network behaves as a whole. Each node in the Hopfield network is connected to every other node, with different weights for each connection.

The Hopfield network can save and reproduce patterns based on an inputted image made of black or white pixels — assigning values of ‘zero’ for black and ‘one’ for white. The network runs the pattern through each node of its system, adjusting the weights of each node until the connections — or their energies — match that of the training image. In physics terms, it continues processing until the difference between the image and the network’s connections is the smallest.

The Hopfield network can generalize new images, recalling and refining patterns based on its associative memory, or ability to remember the connections between unrelated items. Once trained with multiple images, it continues processing until the pattern is closest to the one saved in its memory. 

Statistical physics and the Boltzmann machine

In 1985, Hinton used statistical physics concepts to build on Hopfield’s work. Statistical physics involves applying statistical methods to large groups of multiple elements or particles that would be difficult to identify independently. It considers the collective system of molecules — groups of connected atoms — as a whole unit, analyzing the various components and the states in which they can occur. Boltzmann’s formula calculates the statistical behaviour of such a system or what the distribution of molecules would be at different energy levels. In Hinton’s machine, this formula determines the probability of each node’s patterns.

The Boltzmann machine has two types of nodes: visible and hidden. The visible nodes receive and read out information, while the hidden nodes’ connections contribute to the network’s energy as a whole. This machine is trained with examples that are likely to occur when the machine is running so that it can recognize familiar traits in whatever pattern is fed to it next. The remarkable part of the Boltzmann machine is that it can create entirely new patterns based on what it was trained with. 

Both Hopfield and Hinton’s physics-based neural networks laid the foundation for the current success and popularity of machine learning. This work made early strides in AI that exploded into the large language models, search engine assistants, and image search algorithms that we know today. Hinton had plenty to say about the Nobel Prize, as well as current and future developments in AI.

The future of AI and other messages from Hinton

The day the Nobel Prize announced the winners, U of T held a press conference with Hinton. Global news outlets asked about the rapidly changing field of AI, his reception to the news, and his advice for future researchers seeking to follow in his footsteps. 

Hinton emphasized the importance of AI safety, as with increasingly powerful models, a comparable effort to keep its use safe and responsible is needed. He stated that this safety would come from the developers, as opposed to the individuals using it, especially since larger companies would have both the resources and responsibility to conduct such research on AI safety.

Speaking of resources, the prize money, which is 11 million Swedish kronor or about $1.4 million, will be split equally between the two winners. Hinton plans to donate his share to multiple charities, including one that provides jobs for neurodiverse young adults. When asked about a message for professors and students persisting with research efforts that not everyone believes is worthwhile, Hinton responded with this:

“So long as you believe in something and you can’t see why that’s wrong — like, the brain has to work somehow so we have to figure out how it learns the connection strengths to make it work — keep working on it and don’t let people tell you it’s nonsense if you can’t see why it’s nonsense.”

When Hinton was just starting out in AI research, many people believed neural networks would never work or that the brain was too complex to create an imitation of. As we now know, AI is very capable of understanding and producing language, with large language models like ChatGPT gaining more popularity. Receiving the Nobel Prize is proof of the merit of the collaborative efforts of multiple researchers who have worked with ANNs for decades.

As U of T President Meric Gertler puts it best, “[Hinton has] literally created new ways of thinking about thinking and learning.”