AI model resilience in practice

Why does the same defense method produce different results?

Two datasets, two diferent scenarios

To evaluate when a lightweight defense strategy is effective, I deliberately selected two fundamentally different time-series datasets from the UCR 2018 archive: 

Both datasets represent a realistic resource-constrained scenario characterized by limited data availability, a single training run, and no budget for computationally expensive adversarial training. 

Why was PatchTST selected?

I chose PatchTST, a modern transformer-based architecture,and evaluated it using seven lightweight augmentation-based defense techniques introduced in the previous article of this series. 

The research hypothesis was straightforward: Lightweight adversarial training improvesthe robustness of transformer-based models. The results, however, show that the answer is: “it depends.” And that dependency turns out to be the most interesting finding. 

How does lightweight defense affect AI model robustness?

The Carlini-Wagner Attac as a reference point

The Carlini-Wagner (CW) attack was selected as the reference attack because it proved to be the most effective among all evaluated attack methods. 

Jitter versus Carlini-Wagner Attack

Model	Rock (20 samples, irregular)	ECG5000(500 samples, periodic)
Baseline – clean data	72.0%	93.6%
Baseline + CW	70.0% (-2.0 pp)	67.1% (-26.5 pp)
Jitter – clean data	70.0%	93.5%
Jitter + CW	70.0% (0 pp)	92.6% (-0.9 pp)

Figure 1. Effectiveness of the Jitter defense against the CW attack. Source: author’s analysis (2026). 

On the ECG5000 dataset, the undefended model loses as much as 26.5 percentage points of acuracy under the CW attack. The same Jitter defense reduces this degradation to only 0.9 percentage points while preserving accuracy on clean data, significantly improving the model’s robustness against CW attacks. 

Why did data augmentation work only partially?

The importance of training data volume

On the Rock dataset, the Jitter method did not improve robustness and even reduced baseline accuracy from 72% to 70%.  Moreover, more aggressive augmentation techniques such as GaussianNoise and SmoothTS degrded model performance to 48-52% accuracy.  The first and most obvious explanation is dataset size. With only 20 training samples, agmentation noise disrupts representation learning rather than enriching it. The model lacks sufficient examples to generalize beyond the introduced distortions. With 500 training samples, the model effectively learns to “see through the noise,” demontrating how strongly defense effectiveness depends on both the quality and quantity of training data. 

How does signal structure influence defense effectiveness?

The second factor is the nture of the data itself. ECG signals are periodic and relatively smooth. Random amplitude perturbations resemble ntural biological variability and do not destroy the signal’s key characteristics. Rock spectra, by contrast, are significantly more irregular. For such data, the same augmntation approach may distort the frequency-domain features on which the model relies for classification. 

The natural robustness of the PatchTST transformer

One of the most surprising finding involved the Projected Gradient Descent (PGD) attack.  With a perturbation budget of ε = 0.1, PGD achieved a 0% success rate against PatchTST onthe Rock dataset. The model proved naturally robust to local perturbations.  The explanation lies in the transformer’s self-attention mechanism. A perturbation applie to a single signal patch is distributed across all tokens, reducing the effectiveness of attacks targeting small portions of the input. This serves as an important warning: conclusions drawn from convolutional neural networksshould not be automatically transferred to transformer-based architectures. 

How should AI robustness test results be interpreted?

The most important conclusion is simple: the choice of a defense mechanism is inseparably linked to the size and characteristics of the training data. The same method may prove marginally effective in one scenario and almost perfect in anoter. Without experiments conducted on your own data, reliable prediction is impossible. 

AI Act and training data quality

These findings are also closely aligned with regulatry requirements. Article 10 of the AI Act requires training, validation, and testing datasets to be relevat, representative, and of sufficient quality. My results demonstrate that the quality and quantity of training data affect not only modl accuracy but also determine whether a defense mechanism works at all. Similarly, Article 15(1) requires AI systems to declare their levels of accuracy and robutness. The experiments illustrate how easily these two concepts can be confused. 

Three common mistakes in AI robustness evaluation

Mistaking small sample size for high robustness

On the Rock dataset, the baseline model achieved a seemingly high robust accuracy of 72%. However, with only 50 test samples, a single misclassification changes the result by two percentage points. This reflects statistical noise rather than genuine robustness. 

Generalizing from a single dataset

Had I analyzed only ECG5000, I could have concludd that “Jitter works.” Had I analyzed only Rock, I could have concluded that “augmentation is harmful.” The complete picture emerges only when both datasets are considered together. 

Ignoring model architecture

PatchTST’s robustness against PGD is a characteristic of the transformer architecture itself, not a universal property of all machine learning models. 

The analysis included:

• White-box attacks only
• A single model architecture
• Single-rn experiments without repetition across different random seeds

These results should therefore be viewed as a foundation for further research rather than as a definitive conclusion

What do these results mean for organizations deploying AI?

The experiments produced aclear and measurable conclusion: lightweight defense through data augmentation can significantly improve AI model robustness and reduce the effectiveness of the strongest attacks to nearly zero, but only when the training dataset is sufficiently large and the signal structure aligns with the selected defense strategy. 

For small and irregular datasets, the same technique may be ineffective or even detrimentl. 

This is not a failure of the method itself but rather a precise indication of the conditins under which it can be successfully applied. This is exactly the type of evidence expected by Articles 10 and 15 of the AI Act.