Researchers at the University of Arizona have developed a new approach based on a highly accurate object detection algorithm.

The new high-precision YOLO algorithm finds certain objects in the camera stream. An association method is also used to denote a set of radar points.

A deep learning algorithm that uses a radar needs to be trained on a large amount of labeled data – this is a non-trivial, extremely time-consuming process that is usually done manually. Our idea here was that if the camera and radar are looking at the same object, then instead of manually reviewing the images, we can use a frame-based object detection system (in our case, YOLO) to automatically label the radar data. .

Arindam Sengupta, Ph.D. and student at the University of Arizona
The authors identified three salient features of the approach:

  • the possibility of joint calibration,
  • clustering,
  • associations.

This approach jointly calibrates the radar and its camera. This is necessary to determine how the location of an object detected by the radar will be displayed on the camera.

We used a density-based clustering scheme (DBSCAN) to eliminate interference and stray radar signals, and to separate the radar signals into clusters. The latter is necessary to distinguish between individual objects. And for associations, the intraframe and interframe Hungarian algorithm (HA) is used.

Arindam Sengupta, Ph.D. and student at the University of Arizona
The developers believe that their approach will help automate the creation of datasets for radar cameras and radars. In addition, in their paper, the team explored both classification schemes based on combining radar camera sensors and data collected by radars alone.