Listen to the rattling
Unusual noises while driving often cause uncertainty and concern. Artificial intelligence can now be used to quickly determine the cause of technical discord. “SonicSeek” is the name of the tool developed by IAV.
Robin Kittelmann, April 2022
A system that can distinguish between normal and questionable operating noises would not only help prevent unnecessary consequential damage, but also reduce the number of avoidable workshop visits. That is why IAV has joined forces with the Technical University of Braunschweig to investigate the advances that artificial intelligence (AI) can make in detecting and classifying such noises.
Noise can provide valuable information on the general condition of individual components or the vehicle as a whole. Various optimizations of the acoustics during operation are already carried out during vehicle design and development to ensure that as few disturbing noises as possible penetrate into the interior. After production and delivery of a vehicle, operating noises become interesting for service and quality assurance. Possible wear of individual components can be identified at an early stage based on noises, meaning the affected parts can be replaced in good time and any potential consequential damage can be avoided. The challenge, however, lies in reliably detecting and correctly classifying the noises.
Complex noise world
The high diversity of conventional operating noises turns the search for anomalous sounds into the proverbial search for the needle in a haystack. Normal noises include the engine and transmission, but also rolling noise, which can vary depending on the surface, as well as weather-dependent noises (e.g. rain or hail) or noise from other road users or construction sites. In addition, the vehicle’s own noise varies depending on the model, age and mileage. “The classic methods of noise detection cannot do justice to the wide variability of normal and abnormal acoustic sources,” says Dr. Alexander Roy, Senior Technical Consultant for Advanced Diagnostics at IAV. “With the help of neural networks, we can track down critical sounds with surprisingly good detection rates.”
Neural networks listen in closely
After preprocessing the data to be used for acoustic diagnosis, the signals are processed by a mel-filterbank and converted into a format that can be used by the neural networks. During the learning process, different layers of the neural network record different information. The folded layer primarily captures information about the frequency structure of the sound, while the recurrent layer detects temporal relationships.
For this analysis process, a large data set with driving recordings from different vehicles in diverse driving situations and with a wide variety of environmental and vehicle-specific noises is used. To obtain a measurable indication of the accuracy and reliability of the process, synthetically anomalous noises at different volumes are mixed in. “We divide the data set into three sections. The first section is used to train the neural network, the second section is used to check the function, and we then use the other material to measure the performance and accuracy with which the network detects anomalous sounds,” says Roy.
To increase the precision of the diagnostic system, anomaly detection is useful in parallel with the neural network. The anomaly detection system is only aware of normal operating noises and filters out anomalous noises from its perspective, without, however, being able to categorize them more precisely. If both the neural network and the anomaly detection system sound an alarm, the probability of a need for action on the vehicle is quite high.
Acoustic diagnosis interesting for many use cases
It is possible to customize how proactively the AI sends alerts for possible anomalous sounds based on specific customer requirements. “However, one hundred percent precision can never be achieved with AI; there can always be cases where anomalous noises are not detected, or are predicted but none are present.” says Roy, summarizing the fundamental characteristic of all AI approaches.
Integration of the system into modern vehicles is possible, for example, via the infotainment system. The AI can record live acoustic data via the on-board microphones, for example, and transmit it via cloud computing services for further analysis. In this way, automatic acoustic diagnosis can be implemented for many end devices without installing additional hardware.
In addition, the intelligent system is also of interest outside the automotive world and could be used, for example, in wind turbines, machines in production or aircraft. Wherever machines cause noise, the system can add value in early fault detection and relieve human ears.
More about “SonicSeek” for acoustic diagnostics:
Our expert on the topic
Alexander Roy
alexander.roy@iav.de