The past and present of Japan, which was once at the forefront of AI research and development

Did you know that when Geoffrey Hinton and John Hopfield received the Nobel Prize in Physics in 2024, some media outlets were making a fuss saying, "Why not Amari?" The "Amari" in this case refers to Shunichi Amari, professor emeritus at the University of Tokyo.

In 1972, 10 years before Hinton and Hopfield began their research in the 1980s, Amari published a paper with almost the same content as the neural network mechanism they discovered. In fact, Hinton has said in various places that "Amari was the first to discover the content of my research."

There was another Japanese who researched neural networks and deep learning in the 1970s. There was Kunihiko Fukushima. Fukushima came up with the concept of deep learning in AI, and in 1978, he devised and established the theory for its realization.

Japan was undoubtedly a pioneer in neural network research, which is the basis of the concept of AI. Why has it come to be seen as a backward country in AI today?

First, we need to briefly understand what it means for AI to learn. In the nervous system of living organisms, there are cells called neurons. These neurons are connected to each other by organs called axons and dendrites, and transmit information received by nerve cells. The part where the axons and dendrites are connected is called a synapse.

The strength of the connection of these synapses, which transmit information between neurons, changes periodically in response to external stimuli. For example, when a nerve cell first senses the stimulus of touching something hot, a synapse is formed to transmit that stimulus to other neurons.

The first time, the synapse is only thin, but if you touch something hot for the second and third time, the synapse becomes thicker and thicker, and eventually you will react to touching something hot and feel pain or fear. This is how living organisms learn.

If the part that stimulates the synapse is done by turning on/off an electrical signal, and learning by repeated stimuli is replaced by the weight of the number of times the computer turns on/off as a function value, perhaps it would be possible to reproduce human learning in a machine.

This theory had been proposed since the Second World War, but it was considered an armchair theory and did not attract much attention. However, in 1967, the structure of the neural circuit network of the cerebellum was discovered, and it was found that the model that was thought to be an empty theory was not so different from reality. Suddenly, the enthusiasm for machine learning research increased worldwide.

Thus, AI research became a global boom, but it was later proven that the inferences used at the time had a computational limit to learning, and by the 1970s, the boom in machine learning research had subsided. However, Japan was different.

While the world was giving up on machine learning, Japanese researchers such as Amari and Fukushima were desperately researching ways to overcome the barriers that were thought to be computational limits. The researchers themselves may not have expected that the results of discoveries such as "stochastic gradient descent" and "neocognitron" would become the basic technology behind the birth of modern AI.