Tokyo researchers developed a novel lensless camera

Developments in lensless photography hint at a future of tiny but powerful cameras.
A demonstration of "lens-less" photography.
First row: The subjects. Second row: Patterns captured by the sensor. Remaining rows: The reconstructed images. Xiuxi Pan from Tokyo Tech

This article was originally featured on Popular Photography.

Every photographer knows this truth: When it comes to cameras and gear, things can get huge—and fast. Especially in the lens department. Though mirrorless was first touted as the smaller, lighter alternative to chunky DSLRs (and in some ways, they do still hold to that promise), it’s hard to ignore the fact that these once-promising lightweight systems can quickly become bogged down with glass. And while lenses have been a part of the photographic process since, well, the very beginning, groundbreaking research by a group of scientists at the Tokyo Institute of Technology could lead to new lensless imaging systems and eventually, a lensless future.

Challenges of a lensless camera 

The concept of the lensless camera isn’t entirely new. However, until now, the technology has yet to produce usable results, and the computational time needed to render the images has been too lethargic for practical use. 

The current technology requires the solving of a convex optimization problem, and iterative calculations (meaning multiple results can be returned) make it a drawn-out process. But Tokyo Tech’s proposition has the possibility of making vast improvements to the process.

“Deep learning could help avoid the limitations of model-based decoding, since it can learn the model and decode the image by a non-iterative direct process instead,” the team explained on “Existing deep learning methods for lensless imaging, which utilize a convolutional neural network (CNN), cannot produce high-quality images. They are inefficient because CNN processes the image based on the relationships of neighboring ‘local’ pixels, whereas lensless optics transform local information in the scene into overlapping ‘global’ information on all the pixels of the image sensor, through a property called ‘multiplexing.’”

The lensless camera. Xiuxi Pan from Tokyo Tech

Tokyo Tech researchers propose novel technology

The team at Tokyo Tech has proposed a new image reconstruction method using a mathematical algorithm. The hardware consists of a mask and image sensor, with the mask encoding the incident light (light that falls on the subject), which then casts patterns on the sensor.

Looking at these patterns, the human eye would be unable to decipher any usable information—everything is rendered as incomprehensible blobs, similar to the mark left when you throw spaghetti on the wall or go Jackson Pollock on a canvas. However, the optical algorithm is able to decode the information, yielding fairly accurate results. 

“Without the limitations of a lens, the lensless camera could be ultra-miniature, which could allow new applications that are beyond our imagination,” says Prof. Masahiro Yamaguchi of Tokyo Tech.

The Vision Transformer (ViT). Xiuxi Pan from Tokyo Tech

How it works 

The team has pioneered a technology they call Vision Transformer (ViT), which can learn image features in what they describe as a “hierarchical” fashion, thereby avoiding the traditional CNN processing. This means direct reconstruction, with no iterative calculations needed, and a reduction in approximation errors.

Where CNN relied on local pixel information, ViT can use global features in an image. According to experiments, the process is viable and the proposed camera creates high-quality images with a processing time that would enable real-time capture. 

“The ultimate goal of a lensless camera is being miniature-yet-mighty. We are excited to be leading in this new direction for next-generation imaging and sensing solutions,” says the lead author of the study, Mr. Xiuxi Pan of Tokyo Tech.