Been experimenting with CRMA — a stability layer for LoRA fine-tuning that handles sequential training without catastrophic forgetting. Tested on TinyLlama and Mistral-7B. Full results dropping soon.
Laimleon
2
Thanks for this post.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| CRMA: Drop-in adapter for fine-tuning + continual learning — zero catastrophic forgetting at 7B scale | 0 | 52 | February 27, 2026 | |
| CRMA: Stable Fine-Tuning + Continual Learning for Small LLMs | 0 | 71 | February 27, 2026 | |
| Catastrophic Forgetting of language models | 2 | 37 | February 28, 2026 | |
| Catastrophic Forgetting by Language models | 0 | 66 | February 27, 2026 | |
| CRMA: A low-rank stability adapter for QLoRA fine-tuning — ablation results on TinyLlama, Gemma, Mistral | 0 | 81 | February 22, 2026 |