Typically, predictive analytics for maintenance makes predictions about the performance of a machine based on data, statistical methods and machine learning. It’s not very usual to come across a predictive maintenance programme that applies deep learning.
However, there is a considerable amount of research being done and significant progress made with deep learning applications for maintenance, particularly diagnostics and prognostics with a focus on corrosion detection.
Matias Ferrero, RAMS Engineer at Norwegian University of Science and Technology (NTNU) brought to us some state-of-the-art deep learning approaches at the Maintenance Analytics Summit 2019. Although most of them are in the research and academics domain, they hold enormous potential and promise to be applied in the industry.
Maintenance problems that ML can solve
Before diving into the novel DL approaches, Matias gives an introduction with the typical maintenance problems that can be solved using machine learning in general:
- Anomaly detection – machine learning notifies when there’s an anomaly or outlier in a machine or component so the issue can be further explored and solved.
- Fault detection and classification – the ML system notifies when there’s a fault and classifies it so technicians know which maintenance action to perform.
- Failure prediction (prognostics) – the ML system tries to predict when a component will fail, which is used to plan maintenance activities.
Deep learning in maintenance
As deep learning is used to analyse images or sequential data (such as time series), it can be used for visual inspection such as corrosion, defects on the surface, or sensor data, as a type of sequential data, states Matias.
Combined with computer vision for image detection and analysis, deep learning can do
- Object classification and localisation (is there a sheep and where is it?)
- Object detection (what are the instances of the class – sheep 1, sheep 2, sheep 3)
- Semantic segmentation (which pixel on the image belongs to which class – sheep, road or grass)
- Instance segmentation (which pixel on the images belongs to which instance of the class).
State of the art in DL applications for corrosion detection
Having established what deep learning can do with images and objects, Matias presented several deep learning applications in maintenance based on research papers on detecting corrosion.
The first one is a research paper on “Using deep learning and computer vision techniques to improve facility corrosion risk management systems”, which proposes a deep learning model for detecting corrosion. The researchers in this study (C. C. Ejimuda and C. M. Ejimuda) collected corrosion images from the web, labelled them according to the type of corrosion and drew boxes around the corrosion manually. However, because they only had 36 images, which is a really small data set, they resorted to data augmentation – they rotated the images, made copies of them, or zoomed in and zoomed out versions to enrich their data set to 336 samples for training.
In terms of model, they used RESNET pre-trained on the COCO data set, and on top of that, they put Faster R-CNN for classification and detection.
The results they got were great. The prediction results corresponded with the boxes drawn on the test data images.
The second research paper Matias describes is again on deep learning-based object detection. It’s on “CNN-based automatic coating inspection system” for protecting marine and offshore structures. The approach is similar as in the first paper, with small differences in the models. The researchers used a pre-trained model VGG19 trained on ImageNet, and they put Faster R-CNN on top for classification and detection. But they also included Fully Convolutional Neural Network on top for the segmentation task. Also, instead of collecting images from the web, they used drone images of offshore facilities taken with a visual and infrared camera which can reveal corrosion beneath the surface. They recognised that the area where there’s corrosion underneath heats faster.
The issue with quantity and quality of corrosion data sets
These are some significant innovations in detecting corrosion with deep learning. However, Matias draws attention to a study on “A review of deep learning in the study of material degradation” which underlines the need for a publicly available large data sets to drive innovation in corrosion detection with deep learning, same as with the self-driving cars, object detection or tumour detection domains that rely on large public data sets. The need is that much intensified by the social and economic impact caused by corrosion of facilities and equipment.
The issue with the size of the dataset in corrosion detection, or the lack thereof, is also emphasised in another study Matiad refers to precisely covering the topic of “Quantity beats quality for semantic segmentation of corrosion in images”. The researchers here used a data set of 250 images which they asked high school students to label them. Additionally, they used a smaller data set of ten images to ask experts to do the same. They trained a Fully Convolutional Neural Network model on semantic segmentation on both the large and the small data set. The results showed that the model with the large data set labelled by “non-expert’ high school students outperformed the model with the smaller data set, because a model with a smaller data set tends to overfit. While the model with the larger poorly labelled data set has some noise, but still provides a good basis for algorithms to learn.
The lack of available data set inspired a great side project initiative by two of the authors of the above study, Wil Nash and Nick Birbilis. They created a website where people can upload their images, and the website detects if there’s corrosion in it for free. As they stated themselves, the purpose of the corrosion detection website is to pool data to create a trained model for automated detection of corrosion. Hopefully, this website would be a great start to tackle the lack of freely available machine learning type model for corrosion detection.
Takeaways
Matias concludes that deep learning methods can successfully be implemented for diagnostics and prognostics, as we’ve seen with corrosion above. However, some challenges remain to be solved, the main ones being limited labelled data, no freely available ML models for corrosion detection and limited computational resources.
Add comment