By Stamatis Samaras, Researcher at CERTH.
Artificial Intelligence (AI) is quietly making decisions around us every day, recommending what we watch, filtering spam from our inboxes, and even helping doctors spot early signs of disease. Yet for most people, how an AI arrives at a specific conclusion remains a complete mystery.
In the tech world, this is known as the “black box” problem: data goes in, an answer comes out, and the reasoning in between is hidden behind millions of complex calculations.
When AI is simply suggesting a movie, a little mystery doesn’t hurt. But when AI is used in high-stakes situations, like approving a bank loan, diagnosing a medical condition, or keeping drivers safe on the road, blindly trusting a machine isn’t enough.
What Is Explainable AI?
This is where Explainable AI (XAI) comes in. XAI is a set of tools and techniques designed to peel back the curtain on AI models. It translates their complex internal math into insights that everyday people can actually understand.
Instead of just saying “Here is the answer,” an XAI system can say “Here is the answer, and here is why I chose it.” By making AI more transparent, XAI helps developers catch hidden biases, ensures systems follow ethical and legal standards, and most importantly builds genuine human trust.
Automating Trust: How the ALFIE Project Uses XAI
Building trustworthy AI shouldn’t require a PhD in computer science. That is the core mission behind the European-funded ALFIE project (Assessment of Learning technologies and Frameworks for Intelligent and Ethical AI). ALFIE aims to bridge the gap between ethical guidelines and real-world technology by making AI development accessible, transparent, and people-centered.
At the heart of ALFIE is an AutoML (Automated Machine Learning) platform, a smart system that automates the heavy lifting of building, optimizing, and testing AI models. Rather than leaving users with an opaque, newly generated model, ALFIE weaves XAI directly into its workflow. Every time AutoML creates a model, it can generate a comprehensive XAI Report. These reports give users clear visibility into both the underlying dataset (helping spot imbalances or anomalies) and the model’s exact decision-making logic, complete with visual charts and plain-language summaries.
Real-World Example: Keeping Drowsy Drivers Awake
To see how XAI works in practice, let’s look at a life-saving application: driver drowsiness detection. Imagine a smart car system designed to alert a driver before they fall asleep at the wheel. With the help of cameras installed in the vehicle, an AI model can detect fatigue in two very different ways and XAI explains both.
1. Looking Through the Camera (Visual Data)
In the first setup, a dashboard camera monitors the driver’s face and feeds images into a computer vision model. If the model flags a driver as “drowsy,” how do we know it isn’t just getting confused by the glare on their sunglasses or the angle of the sun?
In the generated XAI report, a technique called GradCAM (Gradient-weighted Class Activation Mapping) solves this by creating a color-coded “heatmap” directly over the driver’s photo.
- Red zones highlight the exact areas the AI focused on most to make its decision.
- Blue or cooler zones show areas the AI largely ignored.
By glancing at the report, engineers and safety regulators can immediately verify that the red hotspots are centered right over the driver’s drooping eyelids and yawning mouth, proving the AI is looking at genuine signs of fatigue rather than irrelevant background details.
2. Reading the Numbers
A second approach doesn’t feed raw video into the model. Instead, it calculates specific facial measurements, such as how wide the eyes are open (eye aspect ratios) and the relationship between the eyes and mouth (tracking yawns), and stores them as tabular data for the model to analyze.
Here, the XAI report uses tools like SHAP values and feature importance charts to break down the math behind every alert. Instead of a heatmap, the report ranks which measurements mattered most. A key insight from the SHAP analysis is that the model has access to many details in the dataset; in our example this includes sensitive attributes such as the driver’s ethnicity and biological sex. In a well-trained system, however, these factors should not influence the decision. The XAI report can confirm this: ethnicity and gender rank near the bottom of the importance list, while the left and right eye openness ratios sit firmly at the top. This is a strong positive signal for developers. It offers measurable evidence that the model is focusing on genuine signs of fatigue rather than personal characteristics, exactly the kind of transparent, human-aligned justification that builds real trust in the system. The report also checks the training dataset itself, ensuring the AI learned from a balanced mix of alert and tired examples so it doesn’t over-predict fatigue.
