Explainable AI is a subfield of artificial intelligence focused on making the decisions and inner workings of AI models understandable to humans.Explainable Artificial Intelligence (XAI) is a field that makes the operation of machine learning models transparent to humans. Unlike traditional neural networks — 'black boxes', XAI provides an interpretation: why the model came to a particular conclusion, which features were important, and how reliable the prediction is. Approaches are divided into interpretable models (decision trees, linear models) and post-hoc explanations for already trained complex models (LIME, SHAP, gradient maps). This is critical in medicine, finance, and law, where an error can cost lives or freedom.
The black box problem intensified with the deep learning renaissance in 2012. Early work on model interpretation includes weight analysis in the 1990s, but XAI as a separate discipline took shape after 2016, when DARPA launched the Explainable AI program. The formal right to explanation of algorithmic decisions was established by the General Data Protection Regulation (GDPR) in the EU.
XAI is based on methods that approximate a complex decision function with a simple interpretable model in the local neighborhood of each example (LIME). SHAP uses game theory: the contribution of each feature is computed as its marginal contribution over all possible subsets of features. Gradient methods (Grad-CAM) visualize which parts of the input image most strongly activated the neurons of the last convolutional layer, using gradients of the target class. For text models (transformers), the attention mechanism allows highlighting important words.
The doctor sees not only the AI's diagnosis but also the image with the tumor highlighted to verify for themselves. The bank explains to the client why the loan was denied.
One of the XAI methods, SHAP, is named after Shapley from game theory — it fairly distributes the 'contribution' of each feature to the decision, like a bonus among team players.





















