We give simpler. sparser. and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models. which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. https://www.hindigyanvishv.com/