A new kind of thermal imaging sees the world in striking colors

A team of researchers has designed a completely new camera imaging system based on AI interpretations of heat signatures. Once refined, “heat-assisted detection and ranging,” aka HADAR, could one day revolutionize the way autonomous vehicles and robots perceive the world around them.

The image of a robot visualizing its surroundings solely using heat signature cameras remains in the realm of sci-fi for a reason—basic physics. Although objects are constantly emitting thermal radiation, those particles subsequently diffuse into their nearby environments, resulting in heat vision’s trademark murky, textureless imagery, an issue understandably referred to as “ghosting.”

[Related: Stanford researchers want to give digital cameras better depth perception.]

Researchers at Purdue University and Michigan State University have remarkably solved this persistent problem using machine learning algorithms, according to their paper published in Nature on July 26. Employing AI trained specifically for the task, the team was able to derive the physical properties of objects and surroundings from information captured by commercial infrared cameras. HADAR cuts through the optical clutter to detect temperature, material composition, and thermal radiation patterns—regardless of visual obstructions like fog, smoke, and darkness. HADAR’s depth and texture renderings thus create incredibly detailed, clear images no matter the time of day or environment.

“Active modalities like sonar, radar and LiDAR send out signals and detect the reflection to infer the presence/absence of any object and its distance. This gives extra information of the scene in addition to the camera vision, especially when the ambient illumination is poor,” Zubin Jacob, a professor of electrical and computer engineering at Purdue and article co-author, tells PopSci. “HADAR is fundamentally different, it uses invisible infrared radiation to reconstruct a night-time scene with clarity like daytime.”

One look at HADAR’s visual renderings makes it clear (so to speak) that the technology could soon become a vital part of AI systems within self-driving vehicles, autonomous robots, and even touchless security screenings at public events. That said, a few hurdles remain before cars can navigate 24/7 thanks to heat sensors—HADAR is currently expensive, requires real-time calibration, and is still susceptible to environmental barriers that detract from its accuracy. Still, researchers are confident these barriers can be overcome in the near future, allowing HADAR to find its way into everyday systems. Still, HADAR is already proving beneficial to at least one of its creators.

“To be honest, I am afraid of the dark. Who isn’t?” writes Jacob. “It is great to know that thermal photons carry vibrant information in the night similar to daytime. Someday we will have machine perception using HADAR which is so accurate that it does not distinguish between night and day.”