HOW TO BUILD RESILIENT AI MODELS?
How to build resilient AI models under the AI Act?
An 8-Step lifecycle and practical defense selection framework
This article is the fourth and final installment in a series dedicated to AI system security and adversarial machine learning. The content is based on an MBA thesis in Cybersecurity completed at the Military University of Technology and presents a practical framework for building resilient AI systems aligned with the requirements of the AI Act and the NIST AI Risk Management Framework (AI RMF).
We now understand how adversarial attacks work, how much effective protection costs, and why the effectiveness of AI defenses depends heavily on data quality. The remaining question for organizations is how to turn these insights into a repeatable, regulation-compliant process.
This article introduces a practical eight-stage lifecycle for secure machine learning models, mapped to relevant provisions of the AI Act and NIST AI RMF functions. It also provides a simple decision framework for selecting appropriate defense mechanisms. Together, these elements complete a four-part series on AI security and model resilience.
Why AI Resilience is a continuous process rather than a one-time test?
One of the most common organizational mistakes is treating AI resilience as a single validation checkpoint prior to deployment.
Article 9 of the AI Act defines risk management as an ongoing, iterative process that must be maintained throughout the entire lifecycle of a high-risk AI system and regularly reviewed and updated.
A model that is resilient at the time of deployment may no longer remain resilient after months of data drift, environmental changes, and the emergence of new attack techniques.
The AI Act and continuous risk management
The AI Act assumes that risk management does not end when a system is deployed. Organizations are required to continuously assess the effectiveness of safeguards, update models where necessary, and respond to emerging threats.
NIST AI RMF and the AI system lifecycle
The same principle is embedded in the NIST AI Risk Management Framework. Its four core functions, Govern, Map, Measure, and Manage, form a continuous feedback loop rather than a linear sequence.
In practice, this means AI security should be designed as an ongoing lifecycle, not as a one-time compliance checklist.
What does the lifecycle of a resilient AI model look like?
The framework presented below consists of eight stages, each linked to specific regulatory requirements.
Particular attention should be given to the stages where lightweight defense mechanisms and adversarial testing are incorporated, as these have a significant impact on overall system resilience.
Stage 1: System Design and Risk Classification
- AI Act, Article 9 (Risk Management System)
- AI Act, Article 6 and Annex III (Risk Classification)
At this stage, organizations define the intended purpose of the AI system, assess its risk profile, and assign responsibility for risk ownership.
Stage 2: Data Preparation and Protection
- AI Act, Article 10
- Data governance requirements
- Detection of data poisoning attacks
Organizations should verify the provenance of training data, document dataset composition, and implement safeguards against manipulation of training datasets.
Stage 3: Model Training and Resilience Engineering
- AI Act, Article 15
- Data augmentation
- Model robustness enhancement
This stage is where lightweight defense mechanisms such as Jitter and RandomZero can be introduced to improve model resilience against adversarial attacks.
Stage 4: Adversarial Testing
- AI Act, Article 15(5)
- NIST AI RMF Measure 2.7
- Robust Accuracy (RA) and Attack Success Rate (ASR) metrics
Model resilience should be actively validated using strong adversarial attack methods such as Projected Gradient Descent (PGD) and Carlini-Wagner (CW) attacks.
Stage 5: Secure Deployment
- AI Act, Article 15(5)
- Rate limiting
- Input sanitization
Following deployment, organizations should minimize interface exposure and control how users interact with the model in production environments.
Stage 6: Operational Monitoring
- AI Act, Article 72 (Post-Market Monitoring)
- AI Act, Article 12 (Logging)
Continuous monitoring enables the detection of data drift, anomalous behavior, and emerging security threats.
Stage 7: Model Retraining and Update
- Iterative process aligned with AI Act Article 9
New data, monitoring results, and identified vulnerabilities should continuously feed subsequent model training iterations.
Stage 8: Audit and Documentation
- AI Act, Article 11
- Annex IV (Technical Documentation)
Every significant decision affecting the model should be traceable, reproducible, and documented to support compliance and auditing requirements.
In Part 4/4, I will translate these findings into a practical secure machine learning lifcycle framework, showing how to build robust AI systems aligned with both AI Act requirements and NIST recommendations. I will also introduce a simple decision tree supporting the selection of appropriate defense mechanisms.
AUTHOR:
Piotr Hawryło is a Software Team Leader at ALTEN with more than 10 years of experience in embedded systems development. He holds an MBA in Cybersecurity from the Military University of Technology and has participated in machine learning projects, including initiatives related to exoplanet detection. His areas of expertise include software engineering, technical team leadership, and emerging technologies.
