FatigueMonitor uses proven computer vision technology to monitor operators for fatigue and distractions at the wheel. Combined with our Collision Avoidance System, FatigueMonitor is an integral part of Hexagon Mining’s SAFEmine safety suite, which addresses two major causes of mine accidents – blind spots and fatigue.
The following article explains the origins of the technology and how it works. In a related video, Product Manager Barbara Hirtz discusses FatigueMonitor and mine safety.
Bringing innovation down to earth
The origins of Hexagon Mining’s SAFEmine Safety Suite lie in avionics, in the Swiss Alps. A few skillful engineers and passionate glider pilots designed a system to avoid collisions between glider planes. The system had to work in all three dimensions of space. With little sight around the cockpit, glider pilots tend to fly in close proximity, chasing thermal upwinds. Collisions are rare but can end tragically. The first collision avoidance system was built and became a de facto standard for light aircrafts in Swiss airspace and beyond.
When a South African mine manager called inquiring about a collision-avoidance solution for his mine, the entrepreneurs responded by building what has become the world’s leading collision avoidance system for the open-pit mining industry: Hexagon Mining’s SAFEmine CAS.
Bridging the gap
As an innovative technology provider, Hexagon Mining seizes growth possibilities. CAS tackles one of the major causes of accidents in a mine – blind spots. The natural progression is to tackle the other major cause of accidents – operator fatigue.
If you have driven for long distances or late at night you’re probably familiar with the feeling of being tired at the wheel. Anyone who has experienced a microsleep is also familiar with the disturbing feeling of not knowing what might just have happened.
Now imagine the same scenario but in the context of driving a million-dollar-vehicle loaded with 300 tons of rock on a steep dust road in the middle of the night.
We can measure alcohol in the human body with handheld devices and set legal limits. Measuring human fatigue scientifically and in real-world conditions is not so straightforward.
Different approaches
The market tries to solve this dilemma with many different approaches. One way of looking at it is to segment them into predictive and reactive methods.
Predictive: The system evaluates operator tiredness and issues notifications of potential fatigue risks. Currently, all known predictive methods are slow to measure variations of a person’s fatigue status. They update information in the order of several minutes, hours, or even shifts.
Reactive: The system reacts when the operator falls asleep. A wake-up alarm is issued in the case of a microsleep to avoid immediate danger. To do so, reactive systems must identify and alert within seconds or fraction of seconds.
The ideal fatigue system is as predictive as it is reactive, and combines the best of both worlds.
One predictive approach is to conduct the challenging transfer from lab-based EEG (electroencephalogram) measurements to scalable real-world conditions on working humans. Lab-based EEG measurements typically use 20-plus tightly attached electrodes on a patient’s head. This method allows scientists to retrospectively identify sleep by specific patterns in the EEG signals.
Another approach is based on a person’s sleep history over past nights. Modern smart watches do this job usually by recording movements of a person while sleeping and therefore counting sleep hours and possibly sleep quality. Based on these measurements, the fitness-to-drive is deduced. Quality data is generated if the watch is worn 24/7 tightly around the wrist. The fit-for-duty check can only be done once per day or once before the shift, respectively. A new status is updated after the next sleep.
The eyes have it!
Humans naturally close their eyes when they fall asleep. It is a body reaction that happens immediately when microsleeps occur. It is also a body reaction that is measurable. Consequently, all reactive approaches are using eye-lid closure as the indicator of choice for fatigue events, either as wearable device integrated in the operator’s glasses, or as camera systems mounted on the truck’s dashboard. Hexagon Mining is conducting a research project with the leading Sleep-Wake Centre for fatigue at the wheel at the University Hospital in Bern, Switzerland. Microsleep EEG measurements are referenced with varying eye apertures while at the wheel.
Hexagon Mining’s choice
Computer vision technology is the core of FatigueMonitor. It has developed quickly in recent years and is integral to many high-tech industries, including self-driving cars. Its potential is vast and not just for fatigue and distraction, but also for other applications in the mining industry.
Modern processors can handle the massive computational effort of image processing algorithms, allowing for reaction time for microsleep alarms within fractions of a second.
Computer vision technology is highly versatile. While all preventive systems and wearables are limited to fatigue, computer vision also identifies operator distraction and can therefore help to keep attention on the road.
Camera systems are truck-based and need not be customized for each operator, which is a major advantage considering spare part management and administration.
All computer vision-based systems work in the near-infrared range, regardless of lighting conditions (day/night). This wavelength penetrates many standard sun glasses that are otherwise completely dark in the visual range. Once an incident is identified, the system issues a unique in-cabin alert. Simultaneously, the corresponding image sequence is sent to the control room for review. Video review not only helps to trigger the mine’s fatigue procedures but it can also be used for training and prevention by reviewing the videos with the operator after the shift.
Just as modern cars are equipped with seat belts and airbags, FatigueMonitor’s integration with CAS is crucial for increased safety, providing two layers of safety.
Safety first
Surveillance cameras are everywhere today but at close quarters, people are naturally more uncomfortable with being watched continuously. FatigueMonitor ensures that an operator is not watched when there is no danger. High-precision CAS data determines when a vehicle is moving above a certain speed. Computer-vision algorithms are only activated when the operator is actually driving and thus, when there is a potential risk for an accident. In stand-still, the operator is not watched and privacy is guaranteed. FatigueMonitor is the only fatigue monitoring system with a display on the dashboard, allowing behavior to be shown transparently to the operator via an icon.
Multifactorial fatigue risk estimation
FatigueMonitor provides a rolling fatigue risk estimation for each operator. While there are human influences beyond anyone’s control, FatigueMonitor weighs all available data for a continuously updated fatigue risk index, including the following:
FatigueMonitor takes these factors and computes a fatigue risk, displaying it on the dashboard as low, medium, or high. It’s also displayed on the control room application, showing the operators with the highest fatigue risks at the top of the list. Operators and supervisors are continuously informed.
Your eyes on the road
Hexagon Mining is serious about safety. CAS and FatigueMonitor address two of the major causes of mining accidents and represent a significant building block in any safety campaign.
Keep your operators safe and your business running.
For more information, email [email protected]