Nobody Knows

Some technical terms can sound quite intimidating at first—“artificial neural network” is one of those. I remember being awestruck the first time I heard it. People still don’t fully understand the structure of the human brain and here we are with an “artificial” neural network that seems to simulate the brain using computers? Anyone’s first reaction would be that artificial neural networks must be extremely advanced. However, we have to bring up the OG:

Look at their graph:

They provided detailed explanations in the text for each figure. Essentially, it’s just logic gates. You could say that 30 years ago neural networks aren’t really complex; it’s something you could explain to your grandmother.

Getting back to the article, this was the first time that the behavior of biological neural systems was abstracted into a mathematical model, a pioneering step for the later development of artificial intelligence and machine learning. It’s fair to say that without these initial attempts, today’s neural network models would look entirely different.

Building on this foundation, neural networks have developed over decades. From the initial perceptron model to multi-layer neural networks and the rise of deep learning, researchers have continuously sought to emulate the complex behavior of biological neural systems. With increasing computational power and the advent of big data, deep neural networks have achieved remarkable results in areas like image recognition, speech recognition, and natural language processing. Over time, various intricate models emerged: CNNs, RNNs, RCNNs, GANs, VAEs, Diffusion models, ResNet, and Transformers.

As models grow more complex and larger in scale, our understanding of how neural networks work internally has paradoxically become more obscure. They’ve become a black box because we can observe inputs and outputs, but it’s challenging to know precisely how information is processed internally. This has led to a growing interest in model interpretability, with the hope of demystifying neural networks.

To address this issue, many researchers have devoted significant effort to interpreting neural networks’ decision-making processes. For example, they use visualization techniques to display feature maps in convolutional neural networks or leverage attention mechanisms to understand what aspects the model focuses on. It’s a bit like how no one has ever actually seen a cardiac stem cell, yet countless articles in Nature and Science discuss their potential for heart regeneration, until, eventually, prominent experts had to step in and reveal the emperor has no clothes. Now we try to interpretate neural networks.

Here’s my personal take: the effort to explain or study the black box has shifted from a scientific question to a philosophical one. Deep learning has hit a bottleneck and its a good time to criticize interpretability. We aspire for these systems to function like our brains—and, in a sense, they’re on the right path. Just as we struggle to fully comprehend our own minds, it’s no surprise that we grapple with understanding why machine learning works as it does.