When AI Keeps Running but Stops Being Trustworthy
Most factories know how to react when a system goes down. Alarms appear, production teams open incident procedures, maintenance is called and operators move toward a known fallback state. The more difficult problem in an AI-enabled factory begins when nothing appears to be down at all.
An artificial intelligence system can remain online, return answers quickly and still become operationally unreliable. That possibility deserves more attention as AI moves deeper into manufacturing.
The United Arab Emirates has just launched Factory Forward UAE, a national initiative designed to accelerate the adoption of AI and Fourth Industrial Revolution technologies in manufacturing. It consolidates programmes that have already supported more than 700 factories, including more than 620 assessed through the Industrial Technology Transformation Index. The platform is intended to help manufacturers move from assessment to implementation by connecting them with experts, technology providers, training and financing.
This is part of a global transition. AI is no longer confined to dashboards and retrospective analytics. It is increasingly being used to recommend maintenance priorities, identify defects, optimise energy consumption, sequence production, forecast demand and support workforce decisions. As these systems move closer to operational control, trust becomes as important as accuracy.
The reason is simple: AI models do not fail only in binary ways. A conventional component often appears healthy or failed. A model can degrade gradually. Input data can drift. A sensor can remain functional while its calibration changes. A new supplier can alter material characteristics. A software update can shift a process variable. A seasonal pattern can emerge that was not present in the training data. A cyber attacker may manipulate a small number of inputs without creating an obvious outage.
The result can be a system that is technically available but operationally misleading.
Predictive maintenance shows the problem clearly. Suppose a model ranks ten machines by probability of failure. If the model is unavailable, the team knows it needs another method. If the model is available but its assumptions are no longer valid, the team may confidently direct resources toward the wrong assets. The failure is not in uptime. It is in trust.
Machine vision creates a similar risk. A quality model may have been validated with specific camera positions, lighting conditions and product variants. Over time, apparently small changes can accumulate. The model may still classify parts, but its false-negative rate can rise. Unless the system is designed to detect distribution shift or declining confidence, the factory may not recognise the problem until defective products appear downstream.
This is why industrial AI needs a concept of a trustworthy operating envelope. Engineers already define safe operating limits for temperature, pressure, speed and load. AI-enabled processes need comparable boundaries around data quality, model confidence, environmental assumptions and decision authority.
The objective is not to create a single universal confidence score. Different applications will need different evidence. A low-consequence optimisation model can tolerate more uncertainty than a system influencing product release or safety-related maintenance. The important step is to make the limits explicit enough that operations teams know when the system should continue, when it should request human review and when it should be suspended.
This also changes alarm design. Traditional industrial alarms tell an operator that a physical parameter has crossed a threshold. An AI-aware control environment may need to show that the decision system itself is moving outside its validated conditions. Data freshness, sensor disagreement, model drift, missing dependencies and unusual recommendation patterns can become operational indicators rather than purely technical diagnostics.
The control room is the natural place to bring these signals together. Its job is to build a common operating picture. In a smart factory, that picture increasingly includes the state of the intelligence layer. Operators should be able to see not only what the AI recommends but how much confidence the organisation should place in that recommendation.
Fallback matters just as much. If an AI quality system is suspended, what replaces it? If predictive maintenance is unavailable, what conservative rule set takes over? If an optimisation engine is isolated after a security concern, which schedule or operating envelope is used? A resilient plant decides these questions before the incident.
Human oversight is often presented as the answer, but it can become superficial if people are no longer prepared to operate without the system. A supervisor cannot meaningfully challenge an AI recommendation if the underlying process knowledge has been allowed to disappear. Human-in-the-loop architecture therefore depends on competence-in-the-loop. Skills, procedures and access to raw operational information must remain available even as automation improves.
The UAE’s wider adoption of Agentic AI makes this issue especially relevant. Federal programmes are moving toward systems capable of supporting services, operations and tasks, while Dubai Police and Digital Dubai have already launched an AI-assisted system that recommends personnel deployment based on operational needs and workforce skills. The general trend is clear: AI is progressing from analysis toward action.
Manufacturing will experience the same shift, and it will expose a new category of operational risk. The question will not only be whether an AI system was cybersecure, accurate during testing or available when needed. It will be whether the factory could recognise the moment when confidence should have been withdrawn.
The smartest factory is not the one that trusts AI the most. It is the one that knows precisely when not to trust it — and can keep operating safely after making that decision.
Abdulla Saeed Alhebsi (A.S. Alhebsi) — Author and Researcher specializing in Security, Heritage, Risk Management, and Control Room Operations.
Featured Product
