Dear SMRP Community,
I am excited to share with you our recently published paper from the 49th Annual Conference of the IEEE Industrial Electronics Society (IES) titled: "Performance of ChatGPT on CMRP: Potential for Assisting Maintenance and Reliability Professionals Using Large Language Models."
Abstract:
As Industry 4.0 and digital transformation progress, there's an increasing emphasis on Smart Maintenance and AI-assisted maintenance solutions. While most of the focus has been on the application of advanced AI algorithms in maintenance optimization, our study delves into how AI can directly support maintenance and reliability (M&R) professionals in a comprehensible way.
Using the Certified Maintenance & Reliability Professional (CMRP) exam as a benchmark, we compared the performance and reasoning capabilities of OpenAI's GPT-3.5 and GPT-4 models against human experts. Our findings revealed that GPT-4 showed significant improvements over its predecessor in terms of self-consistent explanations. However, both models faced challenges, especially with arithmetic operations and domain-specific terminologies.
While there are areas of improvement, our research highlights the vast potential of AI models in supporting various M&R tasks, setting the stage for a more hands-on and comprehensive AI-assisted approach in smart maintenance.
Acknowledgment:
I would like to express my profound gratitude to this community. Many of you generously devoted your time to answer our mock CMRP exam, which greatly enriched our research. Your contributions and the insights we derived from this study will, we hope, benefit M&R practitioners worldwide.
Warm regards,
Xingheng Liu
PhD, CMRP
Norwegian University of Science and Technology
Trondheim
------------------------------
Xingheng Liu
Norwegian University of Science and Technology
Trondheim
------------------------------