Is there no other way than increasing computation power when we get OOMs? Is Lora, qlora the only way.
I’m pretty sure many must have faced this problem, what other ways other than trying qlora/lora, deepspeed, mixed-precision training, are there if we get OOMs during trying for full-finetuning?
The first thing that comes to mind is gradient accumulation…