An experimental hybrid customized AI and generative AI chatbot human machine interface to improve a factory troubleshooting downtime in the context of Industry 5.0

For over a decade, the industrial sector has experienced a tremendous revolution with the advent of the Industry 4.0 (I4.0) paradigm [1, 2, 3]. This concept, also known as smart factory, focuses on implementing advanced and intelligent technological concepts such as artificial intelligence (AI) and machine learning (ML) [4] to improve production processes’ efficiency and automation. Since 2021, a new industrial revolution, Industry 5.0 [5], has slowly been pathing its way into the industry. It builds on top of the existing I4.0 technological principles while shifting the attention toward smart manufacturing [6] systems that are more operator-centric (human-centred), sustainable, and resilient [7].

AI is the driving force behind the deployment of Industry 4.0. It plays a critical role in its implementation, bringing forth benefits such as the creation of automated tasks to improve system productivity and reduce human error, increased efficiency in decision-making, and prevention of system failure with data insights analysis (predictive maintenance); and the design of more resilient production systems reducing overall production costs, growing competitiveness, and increasing customer satisfaction [8]. Being built on similar technological principles, in Industry 5.0, AI carries the same promises and potential, focusing on strategically re-inserting the human factor within production processes to create more sustainable outcomes for factories and the planet. With its emphasis on human well-being, I5.0 also responds to the long pending concern of job losses in industries due to the introduction of AI-automated technologies and solutions in production processes. In addition to the advantages mentioned above, I5.0 aims to offer human-centered AI solutions that upskill operators to efficiently interact with machines and reduce waste and energy consumption, which is beneficial to the environment [9].

While looking at the great benefits of AI and Industry 5.0 in the manufacturing sector, monitoring the challenges emanating from their implementation is crucial to mitigate the risks of substantial failures and losses within factories during the execution phase. Some of these challenges are a call for skilled workers to understand the high level of new technological concepts, a high initial capital required with often unclear return on investment in the initial phase, a need for more data collection and security, and an ethical and legal responsibility in the implementation of these new technologies to avoid more job losses and outcome bias [11].

With the advent of I4.0 [11], manufacturing plants deal with more devices to monitor and a high amount of data to process [12]. In the context of I5.0, factories need to find the right balance between effective system monitoring, reducing equipment downtime and operators’ growth and well-being. Techniques like predictive maintenance [13, 14], empowered by several ML algorithms such as convolutional neural network (CNN) support vector machine (SVM) classification [15], successfully mitigate system downtime by detecting potential equipment threats before they become fatalities. Beyond the predictive maintenance technological ability to keep systems running, I5.0 calls for an additional societal layer that personalized human-to-machine interaction employing natural language.

Chatbots are automated conversational tools currently utilized in various sectors [17, 17] to create low-cost, efficient, customized, intelligent interactive systems between humans and machines [18]. In recent years, generative AI [19], an AI branch that generates new data based on a specific input, has been the driving force behind chatbot creations. ChatGPT [20] is a powerful, ready-to-use generative AI large language model (LLM) tool enabling users to enter almost any kind of query and receive back insights in the requested format (texts, videos, or images). While ChatGPT and generative AI conversational models produce remarkable results in several academic and business areas [21], their integration still needs to catch up in the operational manufacturing sector [22].

We propose the design of an experimental hybrid customized chatbot HMI that combines AI and generative AI functions to: We structure the remaining sections of the paper as follows: In Section 2, we present a background and literature overview on some key concepts utilized in the research, such as the shift from I4.0 to I5.0, and summarized benefits and challenges of AI and I5.0, generative AI technologies, LLMs and ChatGPT, equipment failure downtime algorithms in factories, and the use of chatbots in manufacturing plants. In Section 3, we describe the theoretical design, the architectures, and the flowchart of the customized Chatbot HMI. Section 4 presents the experimental results of the hybrid chatbot, and we conclude the research in Section 5 with valuable insights on design considerations, limitations, and guidelines for future works. We present in Fig. 1 a process flow diagram of the manuscript.

Generative AI is a branch of AI that utilizes generative algorithms and models to create new content, such as texts, videos, sound, images, code, and other types of data based on a user’s input [34]. One of the most recent considerable successes of generative AI was the ability to train its Generative Pretrained Transformer 3 (GPT-3) model on over 500 GB of data with over 100 billion parameters, achieving outstanding results for universal language description and interpretation. As per Brown, T.B. et al. [35], GPT-3 outperforms most pre-trained state-of-the-art language models. In November 2022, generative AI became more popular with the official launch of the generative pretrained transformer (GPT) chatbot application called ChatGPT by the company OpenAI. OpenAI utilized reinforcement learning with human feedback (RLHF) to fine-tune the GPT-3 model, which made the creation of ChatGPT possible [36]. ChatGPT was attractive to users because of its ability to answer queries in a human-like conversation format [37]. ChatGPT’s ease of use and openness to the public have drastically impacted people’s perception of the utilization of AI and ML, which were seen as technologies reserved for high-skilled developers and engineers. It was reported that in the last year, ChatGPT became a reference for various research and interactive requests on the internet. In fact, upon its release, ChatGPT reached over 100 million users worldwide [38].

Large language models (LLMs) are technologies used to generate texts exclusively in the generative AI field. For the past year, LLMs have also become a center of interest for several researchers for their ability to produce very precise human-like chats and conversations. For this reason, various customized applications opt to integrate pretrained LLMs in their back-end rather than trained models that are even time-consuming to build. Despite the explosion of ChatGPT’s experimentation, research on the practical implementation of ChatGPT and generative AI techniques within the manufacturing environment remains scarce [19].

Over the years, strategies such as predictive maintenance [39, 41] have helped factories improve equipment breakdown time by performing early machine failure detection before they become production-threatening. However, the method still depends on the operator’s capacity to interpret the fault, understand the action required, and actively implement it without wasting more time.

There are various components and variables impacting factories’ equipment downtime estimation. They are not fixed and can vary depending on a plant type or size. The following are samples of basic elements practical to approximate a small factory equipment downtime. Let us assume the following variables:The total number of equipment that failed in the plant, \(\Delta \), can be estimated by Eq. 1 as follows:where \( x \epsilon N ^{*} \) and represents the factory equipment count increment and y is the equipment health status. We define y by the following:From the above variables, the overall factory’s equipment downtime \(\theta \) can be estimated by Eq. 2 as follows:By substituting Eqs. 1 in 2, \(\theta \) can further be expanded in Eq. 3 as:From Eq. 3, we can understand that the more maintenance operators are involved in troubleshooting, the less equipment downtime is. Unfortunately, increasing the number of maintenance operators can be costly for the factory. In real factory operations scenarios, not all operators can be allocated to troubleshooting during production, as several other plant parts require attention.

A more cost-effective way to control \(\theta \) would be to reduce the value of Z, the troubleshooting buffer time, which depends on the operators’ or supervisors’ ability to analyze and detect equipment issues. Our customized chatbot HMI aims to support operators in reducing Z by providing precise factory equipment data analytics necessary for troubleshooting.

The manufacturing industry has demonstrated its readiness to implement various AI applications within its production processes. However, chatbot applications have yet to become popular within the field. Wang, S. et al. [41] highlight three primary use cases for implementing chatbots in the manufacturing industry. Our customized chatbot HMI combines a monitoring and maintenance use case (without visual aid) created from factory historical data to provide valuable insights on equipment health statuses for troubleshooting purposes.

The customized AI and generative AI chatbot has two essential components: a front-end, which serves as the interface between the operator’s requests and the factory data and a back-end, containing the system’s brain power, the OpenAI GPT 3.5 language model (LM) to understand the operator’s inquiries and advise on a suitable solution.

We design the factory’s front-end with Streamlit [44], an open-source tool to develop data science and machine learning (ML) web applications. The factory’s customized application has a security layer to it. It allows to safely process data in a separate platform without exposing it to open fields such as the web. Loading factory data straight in an open platform like ChatGPT would have this security risk. We use Python programming to create the front-end application in Streamlit and connect it to the back-end unit (GPT 3.5 LM model) using OpenAI APIs with an exclusive API key, ensuring more data security.

We store the factory data in a CSV file. It is a convenient format for most controllers, such as PLCs that can automatically save information in this format. Unfortunately, by default, LMs do not understand CSV file format. They understand texts. To resolve this issue, we incorporate a Langchain agent via Langchain libraries, which understand CSV, into our back-end unit. The Langchain agent becomes the tunnel linking our factory data to the LM. Therefore, we can train GPT 3.5 LM on the factory data, which becomes the primary source of information to answer users’ prompts.

The chatbot HMI’s goal is to improve the equipment downtime in Eq. 3 by decreasing “Z,” the buffer time required for a plant maintenance supervisor to analyze overall data and retrieve faulty devices’ information. Based on historical insights, the factory’s general requirements would be to timeously:

In Fig. 3, we present the customized chatbot HMI process operation flow diagram. The flowchart has two sections: the front-end web app and the back-end LM (GPT 3.5) brain. Through the first section, the operator loads the factory data in CSV format to train the LM in the back-end and sends queries related to the loaded factory data. The web app receives operators’ prompts and links them to the back-end to extract the corresponding information. It also displays the final solution obtained by the back-end at the end of the process in a natural language format. We illustrate the second section of the flow chart (the back-end thinking process) in Fig. 4. It is the system’s brain that performs all the intelligent “thinking routines” to provide the most accurate answer to the operator.

We display in Figs. 5 and 6, the customized chatbot HMI web application front-end before and after loading the factory csv data respectively. Figure 5 shows the initial look of our customized chatbot HMI web application front-end before loading the factory data for training. It is the welcome page of the chatbot, the first page the user sees when launching the chatbot HMI application. It contains the following primary information:

Note: The information mentioned above about the web application front-end has the same meaning in all other figures of the manuscript displaying our customized chatbot HMI front-end (Figs. 6, 8, 10, 11, 12, and 13).

Figure 6 presents the chatbot HMI web application front-end page after successfully loading a CSV file for training. In addition to the above information, it displays:Figure 4 is a simplified model of the customized chatbot HMI displaying the main system stakeholders: the factory data (in CSV format), the operator (loading data and sending prompts), the front-end web app receiving loaded data and prompts to transmit to the back-end, the Langchain agent playing the role of an interpreter between the loaded CSV data and the GPT 3.5 LM, and the GPT 3.5 LM itself.

Figure 7 shows the query/answer process architecture. It portrays how the Langchain agent, boosted by the GPT 3.5 LM, initiates a “thinking process” when receiving a prompt from the user. It goes through the CSV file and other tools made available by the LM until it finds the best answer for the operator.

Algorithm 1 is a translation of Fig. 7 describing the essential steps integrated into the customized chatbot HMI design. As mentioned in the previous section, we designed our customized chatbot (front-end and back-end) with Python programming language.

Note: We assume that a factory expert prepared the data used to train our system in an understandable format (example of data format in Table 1) after accurately learning the equipment’s behavior and that its content is correct and accurate. Because of the token limitation a new dataset is used for testing the generative AI functions with five existing initial conveyors (from prompts 9 to 11 of Table 2).

In this research, we proposed a hybrid customized AI and generative AI chatbot HMI design for an experimental factory. Using the GPT 3.5 LM trained on factory data in its back-end, the customized chatbot provides operators, upon request, with the factory’s equipment details helpful in troubleshooting in case of failure. Our chatbot can also generate new data with characteristics similar to the loaded data thanks to the generative AI feature of the GPT 3.5 LM in the system’s brain. We design the chatbot front end with Streamlit, a web application that ensures security by separating the loaded factory data from the unsafe open web. We use additional LM frameworks like Langchain to integrate into the GPT 3.5 LM the ability to read CSV files. The experimental results we achieved by running our chatbot to extract equipment statuses and predictive maintenance data analysis show the system’s accuracy in extracting information using natural language. We obtained an accuracy of 90.9% by computing 13 queries on the data retrieval for system troubleshooting and 100% for predictive maintenance analysis. The system processing time for troubleshooting prompts and answers is insignificant (less than 10 s on average), which eliminates the need for a waiting period for a maintenance supervisor to perform data analysis before troubleshooting, therefore improving the factory breakdown time. Our customized chatbot introduces the implementation of I5.0 HMIs to an I4.0 environment. It promotes interactive, personalized human-to-machine communication with natural language (with the operator at the system’s center) over graphical and static interactions on traditional HMIs. Our research also contributes to boosting manufacturing SMEs’ confidence in complying with current state-of-the-art technologies and industrial revolutions to remain competitive in the fast-growing industrial market.

In future chatbot app versions, we plan to improve the integration of generative AI chatbot HMIs in Industry 5.0 by creating feedback loops between users/operators and the system. This feature can empower the decision-making experience in factories by generating more accurate information based on user prompts. We would achieve this function by notifying the system of wrong or less accurate answers and rewarding better ones. We intend to make the system more autonomous by automating the data training process, saving the factory information in a server, and connecting it directly to the chatbot app. Having the factory data in a secure server would allow us to manipulate the generated outcome (emailing them automatically to supervisors or saving them in separate folders). We also plan on integrating visual aids like graphs and charts to improve the data analytics experience when troubleshooting factory equipment.