According to the United States Department of Labor, more than 5000 workers died on the job in the United States in 2017. This has clearly brought into further focus the need to enhance industrial site safety. Forward-looking organizations are trying to improve safety protocols and training methods using the latest technologies. This is where AI-based solutions have come into play with the possibility of using machine learning to detect and flag violations.
On a typical construction site or industrial shop floor, a large number of people work in shifts. They need to use personal protective equipment (PPE) for safety. Round the clock safety monitoring on the entire site is essential to avoid accidents. The incumbent process involves a combination of direct human supervision and human monitoring of CCTV footage and alerting when safety protocols are breached. A new-age approach would be to build an AI solution that scans through the CCTV footage in real-time and alerts any safety protocol breach.
Such a solution must have a high degree of accuracy to not miss out on violations and must be fast enough to detect violations in real-time. A typical solution for such a problem uses an object detection algorithm to detect people and personal protective equipment in the video footage and relates it to safety compliance.
Implementing such a solution will be challenging on multiple fronts:
– Pre-trained networks for object detection are trained on common objects like animals, automobiles, and humans. No such trained model exists for detecting personal protective equipment.
– Size of objects to be detected in the same frame are very different. The model must be able to detect a person and the industrial safety component he is wearing at the same time.
– CCTV cameras are usually fixed high above the ground and construction equipment on the site obstruct views at times. Pretrained networks are useful for detecting people at short distances, but fail when used in detection from high mast cameras. Also, dust, rain, etc. degrade the image quality and performance of an object detection model.
– The video feed resolution is limited by existing infrastructure. It is not feasible to zoom into parts of the video stream for making detections.
Despite these challenges, it is feasible to build a deep learning solution which can detect personnel without PPEs in a construction site/shop floor. When we build solutions for such problems, we train deep nets to detect two classes – person class and PPE class. We prepare training data by pooling images from various sources like ImageNet, CoCo dataset and publicly available images of personnel with safety gear. The PPE class, which has comparatively limited images available, can be augmented with additional synthetic images generated from 3D geometry files. Then, we use an open-source image to annotate the images and mark the two classes.
We build an initial solution using a basic neural net architecture – VGG19 with Faster-RCNN algorithm – that offers speed of training. We start adding new images to the training data set to improve the accuracy of the two classes. Here we face a trade-off between a more complex deep learning architecture vs. accuracy. If the performance of the algorithm is not satisfactory e.g. accuracy of 70%, a more complex approach is likely required. This will involve shifting to deeper neural networks and creating an architecture that extracts more features. Usually, this should boost accuracy to 90% and more. In real-life scenarios that we solved, we achieved an accuracy of 100% for the person class and more than 98% for the PPE class.
In addition to the PPEs, sites have other protocols as well, e.g. permissions to enter specific zones, special PPEs for different tasks, etc. We would need to implement zoning to mark areas that are barred from entry, from available video frames. Personnel entering a barred zone or walking in without PPE will be marked as a violation of safety procedure. The violations can typically be flagged by highlighting the person and logging the coordinates.
The advent of AI and machine learning has thus enabled the creation of new ways to solve traditional business problems that can significantly enhance the quality of work life. Organizations should be closely watching these developments to understand how they can be used to benefit their industries.Tags: AI Data Science Image Processing Industrial Safety Video Analytics