Deep learning has a transparency problem.
Modern neural networks achieve impressive results across vision, language, and decision-making tasks, yet they often fail at answering a basic question: why was this decision made? This opacity limits trust, auditability, and adoption in real-world, high-stakes systems.
Over the years, the community has explored two main directions. On one side, highly accurate black-box models with post-hoc explanations. On the other, fully interpretable models that attempt to force reasoning into human-defined concepts often at the cost of performance.
Both approaches have limitations.
This tension motivated a question that guided my recent work:
Is it possible to constrain deep learning models for interpretability without destroying their expressive power?
The Problem With Forcing Interpretability Everywhere
Concept-based approaches, such as Concept Bottleneck Models, enforce predictions through human-interpretable concepts. When successful, they provide clear and intuitive explanations. However, they also assume that all relevant information can be expressed using predefined concepts.
In practice, this assumption rarely holds.
Many discriminative features textures, subtle shapes, spatial correlations do not map cleanly to human language. Forcing all information through concepts often leads to excessive compression and, consequently, a significant drop in predictive accuracy.
The issue is not interpretability itself, but where and how it is enforced.
A Hybrid Perspective
Instead of trying to eliminate the black box, I explored a different approach:
make it explicit, bounded, and structurally separated from what we can explain.
This idea led to HGC-Net (Hybrid Guided Concept Network).
The core design principle is simple:
- Part of the model is trained to represent human-defined, interpretable concepts
- Another part remains unconstrained, capturing residual information necessary for performance
- The final prediction uses both components
- Interpretability is enforced selectively, not globally.
What This Enables
This architectural separation leads to an important behavioral shift:
- When a human-defined concept applies, the model exposes it clearly
- When no known concept applies, the model does not fabricate explanations
- Performance remains close to a standard convolutional baseline
In qualitative analyses, this distinction becomes explicit. Some predictions are fully explainable through concepts. Others are correct but rely on latent representations, with the model being honest about the absence of a semantic explanation.
Rather than pretending to be fully interpretable, the system makes the limits of interpretability visible.
Why This Matters
Interpretability should not mean oversimplification.
Forcing explanations where none exist can be just as misleading as offering no explanation at all. A model that acknowledges the boundary between what is explainable and what remains latent is often more trustworthy than one that claims total transparency.
HGC-Net treats interpretability as a structural property, not a post-hoc add-on. By deciding where semantic constraints belong, we can preserve both accountability and performance.
Read the Full Paper
The complete technical details, experiments, and qualitative analyses are available in the full preprint:
Zenodo (DOI): click here
The paper includes:
- the full architectural formulation
- quantitative comparisons with baselines
- qualitative examples of instance-level explanations
- links to executable experiments
Closing Thoughts
Breaking the black box does not require eliminating it.
Sometimes, the most honest solution is to draw clear boundaries, explain what can be explained, and explicitly expose what cannot. Hybrid semantic bottlenecks offer one possible path in that direction, and this work is an ongoing exploration of how far that idea can go.
If you’re interested in interpretability, concept-based learning, or trustworthy AI, I’d be glad to hear your thoughts.
Top comments (0)