Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns

Abstract

We introduce Deep Thermal Imaging, a new approach for
close-range automatic recognition of materials to enhance
the understanding of people and ubiquitous technologies of
their proximal environment. Our approach uses a low-cost
mobile thermal camera integrated into a smartphone to
capture thermal textures. A deep neural network classifies
these textures into material types. This approach works
effectively without the need for ambient light sources or
direct contact with materials. Furthermore, the use of a deep
learning network removes the need to handcraft the set of
features for different materials. We evaluated the
performance of the system by training it to recognize 32
material types in both indoor and outdoor environments. Our
approach produced recognition accuracies above 98% in
14,860 images of 15 indoor materials and above 89% in
26,584 images of 17 outdoor materials. We conclude by
discussing its potentials for real-time use in HCI applications
and future directions.