MONITORING AFFECTIVE STATES IN SMART MANUFACTURING OPERATORS WITH MULTIMODAL TOOLS

Abstract

As the smart manufacturing domain evolves, human operators remain relevant but often not considered with respect to emotional and cognitive well-being. While efficient processes and automation have component technology, there has yet to see affective state monitoring as part of an operator's safety and performance system. We examined a psychological method to monitoring affective states in smart manufacturing environments using multimodal sensing tools. We examined the real-time screening of emotional and cognitive states of human operators through electroencephalography (EEG), eye-tracking, galvanic skin response (GSR), and posture sensors. With these tools, we can reveal stress, cognitive overload, fatigue, and disengagement; conditions that can invoke a direct impact on decision-making, response speed, and productivity.Built around the theories of emotional regulation and cognitive load, our research focused on how physiological responses can be inferred to describe the psychological states causing the responses. We put forth a theorized framework that combines multimodal data to constantly assess affective states of operators, which then provides the opportunity to make adaptively responsive and allow for human-centered system design. Our end goal is operator safety and performance improved through psychological awareness embedded into smart systems. This not only helps move the industrial world forward in its consideration for affective computing considerations, but advances sustainable and empathetic workplace practices in the age ofIndustry 4.0.