Despite aiming to boost work efficiency, AI can be rife with issues when performing real-world operations
The rapid progress of AI in recent years has solidified the expectation that if leveraged, AI will automatically boost work efficiency and productivity. However, a factor rarely considered is that AI models built with a business’ training data can become obsolete.
This is usually due to changing trends in input data which can be triggered by shifts in society, markets or the environment. AI models progressively become less accurate after these shifts, and this problem is becoming more commonplace.
When trying to boost work efficiency through the implementation of AI into operations, it’s crucial to periodically check its accuracy and retrain models if they’re becoming outdated. Otherwise, the AI model will lose its high level of predictive accuracy.
The process of checking an AI model’s accuracy requires users to have latest version of the input data and its correct label as a set. Accessing and preparing correct labels of the input data obtained during operation can be expensive and tedious. This is because it sometimes requires an expert to annotate correct labels to large amounts of data. In addition, it’s difficult for people to judge when to retrain the models, meaning sometimes large drops in accuracy can go unnoticed (See Figure 1).
Figure 1: Differences in the flow of AI operations. Autonomous annotation and self-model restoration can help cut costs
Our world-first technology automatically estimates accuracy from AI model and restores its model without retraining
In light of this, Fujitsu Laboratories Ltd. has developed a solution called ‘High Durability Learning’ for when these AI issues arise in real-world operations. This world-first technology maintains the accuracy of AI models while achieving stable operations.
High Durability Learning continuously estimates the accuracy of AI models, and restores its model automatically whenever its accuracy drops. As its name indicates, the technology aims to create AI that can offer advanced levels of durability for a wide range of environments and conditions, and consistently produce good results.
This technology has two main advantages: automatically estimating AI model’s accuracy during operation, and being able to restore its AI model without retraining.
Automatically estimating AI model’s accuracy during operation
First, the technology captures the shape of the distribution of the training data being used to train AI models, as well as that of the distribution of input data obtained during operation. From there, it can ascertain differences between the data used in training against those obtained during operation, making comparisons with the estimation results from the original AI model. This is how our technology quantitatively estimates the AI model’s accuracy.
Automatically restoring AI model without retraining
The technology can also restore AI models in accordance with estimated results and trends in input data, meaning it can adapt AI models to fit new input data. By doing this, our technology can account for drops in the accuracy of the AI models and remedy it without costly model retraining.
High Durability Learning: Estimating and preserving the accuracy of AI operations in finance, retail, and logistics
High Durability Learning technology is incredibly versatile and can be utilized with any type of input data or AI model. It can be combined with either preexisting or newly installed AI systems, allowing for a wide range of opportunities to demonstrate its benefits.
For example, we verified that High Durability Learning could estimate the accuracy from the AI model within a 3% margin of error that assesses credit risk in the financial sector by using financial data from 3,800 companies. Conventional AI operations can sometimes experience up to a 69% drop in accuracy, while our technology consistently kept accuracy levels above 89%.
We also confirmed its ability to classify product images in the retail sector and recognize character on slips in the distribution and logistics sector. Again, we saw impressive results regarding its error rate when predicting accuracy from AI models and its ability to maintain accuracy through automatic model restoration (See Figure 2).
Figure 2: Examples of technical verification in different industries
Going forward, Fujitsu Laboratories Ltd.’ aim is to verify as many different business use cases for High Durability Learning as possible. By 2020, we want to incorporate it into our "Design the Trusted Future by Data × AI", Fujitsu's goal-oriented process and framework.
In addition, we also hope to leverage the technology practically in the form of a new machine learning tool to support FUJITSU Human Centric AI Zinrai, Fujitsu's proprietary AI technology.
