… of pre-trained language models. In this paper, we first propose using example forgetting to
find … We consider three sentence pair classification tasks, namely natural language inference, …

… on natural language inference (NLI) debiasing mainly targets at one or few known biases
while not necessarily making the models more robust. In this paper, we focus on the model-…

… to enable pretrained LLMs to follow natural language instructions. While instruction tuning
improves … and leverage transformer capabilities, providing robust and steerable methods for …

… Surprisingly, certain capabilities are robust to forgetting even after fine-tuning on random …
To qualify an example as forgotten, we require the model have a confidence of at most 0.6 in …

… and natural language inference, under different attacks across various pre-trained … the
forgetting problem and achieve more robust models, we propose a novel approach named Robust …

… nor about generic facts for language inference. Therefore, we … forgetting depends on the
input modality (natural language … , a naive fine-tuning approach is robust and does not show a …

… As shown in the top part of Table 1, which we discuss later, we found that several strong
baselines … without optimal aggregate performance are often prone to catastrophic forgetting. …

… Anyone working in the field will know that we are still far away from having models that can
perform NLI in a robust, … trained on later rounds don’t suffer from catastrophic forgetting. …

… at every timestep of the decoder, to avoid forgetting. … Since we collected 3 explanations for
each example in the … , we conclude that this measure is not reliable for our task, and we further …

… , we demonstrate a passive form of privacy amplification: examples used early in model
training may be more robust to privacy attacks. Our findings align with the theoretical findings of …