RobustSSL Benchmark

Introduction to Robust Self-Supervised Learning (RobustSSL) Benchmark

The wide-ranging applications of foundation models, espeically in safety-critical areas, necessitates the robust self-supervised learning (RobustSSL) which can yield adversarially robust foundation models. Fine-tuning these robust foundation models can introduce strong adversarial robustness in downstream tasks. Here, we build a leaderboard of the transferability of RobustSSL methods, which aims to provide a comprehensive evaluation of RobustSSL methods and facilitate the research in robust pre-training.

Performance Mesurement

The code for conducting fine-tuning and evaluation as well as the model zoo are provided in GitHub.

Fine-Tuning Modes

• Standard linear fine-tuning (SLF): only standardly fine-tuning the classifier while freezing the feature extractor.
• Adversarial linear fine-tuning (ALF): only adversarially fine-tuning the classifier while freezing the feature extractor.
• Adversarial full fine-tuning (AFF): adversarially fine-tuning both the feature extractor and the classifier.

Fine-Tuning Methods

• Vanilla fine-tuning: You need to specify the hyper-parameters such as the learning rate and the batch size for each pre-trained models.
• AutoLoRa: It is a parameter-free and automated robust fine-tuning framework. You DO NOT need to search for the appropriate hyper-parameters.

Evaluation Metrics

• Robust accuracy (%) refers to the robust test accuracy evaluated on the adversarial test data generated via the standard version of AutoAttack.
• Corruption accuracy (%) refers to the mean test accuracy of the test data under common corruptions with corruption severity ranging {1,2,3,4,5}.
• Standard accuracy (%) refers to the standard test accuracy evaluated on the natural test data.

Leaderboards

• Self-task robustness transferability on CIFAR-10
• Self-task robustness transferability on CIFAR-100
• Self-task robustness transferability on STL-10
• Cross-task robustness transferability from CIFAR-10 to STL-10
• Cross-task robustness transferability from CIFAR-100 to STL-10