Thanks again for raising these points Jeremy! I posted an updated example which does not use LoRA but instead uses transfer learning here: https://medium.com/towards-data-science/fine-tuning-bert-for-text-classification-a01f89b179fc