PEFT

Thus, we, as a com- munity of researchers and engineers, need efficient ways to train on downstream task data.

Parameter-efficient fine-tuning, which we denote as PEFT, aims to resolve this problem by only training a small set of parameters which might be a subset of the existing model parameters or a set of newly added parameters.

https://arxiv.org/pdf/2303.15647.pdf