This artificial intelligence chip recognizes images in nanoseconds

A new type of artificial eye, conceived by combining electronic light detection devices with a neural network in a single tiny chip, is able to recognize what you are seeing in just a few nanoseconds. It’s a much faster time lapse than the image sensors that exist so far.

Object detection is the part of artificial vision that studies how to detect the presence of objects in an image based on their visual appearance.

A design that imitates nature

The design of this chip, published in the magazine Nature, by researchers at the Institute of Photonics in Vienna, Austria, mimics the way animals' eyes preprocess visual information before passing it to the brain.

That is, in order for image recognition to be much faster and to use much less energy in it, a sensor captures and processes the image at the same time. Usually two parts can be distinguished in the detection process: the extraction of characteristics of the content of an image and the search for objects based on those characteristics. In this new research, however, the data does not have to be read and processed by a computer, but the chip itself provides information about what it is seeing.

The team of researchers built the chip from a tungsten dissent sheet of a few thick atoms, engraved with diodes light sensors. The diodes were then connected to form an artificial neural network, a computational model vaguely inspired by the behavior observed in their biological counterpart.

The material used to make the chip gives it unique electrical properties so that the photosensitivity of the diodes, the nodes of the network, can be modified externally. This means that the network can be trained to classify visual information by adjusting the sensitivity of the diodes until it provides the correct answers.

In the study published in Nature, it is shown how the chip was trained to recognize stylised and pixelated versions of the letters "n", "v" and "z".

This new sensor is another step on the way to advancing the hardware in which an artificial intelligence is installed, making it faster and more efficient. However, there is still a long way to go. For starters, the eye consists of only 27 detectors and cannot control much more than 3x3 block images.

“Our test chip is still small but you can easily expand the technology depending on the task you want to solve," says Thomas Mueller, research leader. “In principle, the chip could also be trained to distinguish apples from bananas, but we see its use more in scientific experiments or other specialized applications."

Even so, however small, the chip can perform several supervised and unsupervised standard machine learning tasks, including, as seen above, the classification and coding of letters. Machine learning techniques aim to automatically differentiate patterns using mathematical algorithms. These techniques are commonly used to classify images.

Reference: Mennel, L., Symonowicz, J., Wachter, S. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020). https://doi.org/10.1038/s41586-020-2038-x