Hi, I was curious about how the self.loss_function is implemented in the Qwen2.5-VL model to compute the loss during training. Could someone explain how it works or point me to the relevant part of the code? Here’s the link to the line I’m referring to: src/transformers/models/qwen2_5_vl/m…

The loss functions are defined in src/transformers/loss/loss_utils.py. The logic for selecting which loss function to use is implemented in the PreTrainedModel class, located in src/transformers/modeling_utils.py. link: transformers/src/transformers/loss/loss_utils.py at main · huggingface/transfor…

Maybe this? [image] Where to look for a loss definition for a pretrained model? Beginners I am using facebook/opt-350m model: model = AutoModelForCausalLM.from_pretrained("facebook/opt-350m") As far as I understand, its default loss is the crossentropy loss. But how can …

Thank you so much for sharing. However, these issues predated the Transformers version 4.53.0.dev0. What I want to know is where the self.loss_function was implemented for these models so I can modify it correctly. Thank you!

How was self.loss_function implemented

🤗Transformers

John6666 June 9, 2025, 11:13am 2

Maybe this?

Topic		Replies	Views
Where to look for a loss definition for a pretrained model? Beginners	2	2113	November 15, 2022
What is `self.loss_function` in `forward()` of newly released LLM? 🤗Transformers	0	83	January 14, 2025
Having troubel in understanding what loss is currently in use Beginners	1	827	November 24, 2023
How can I know what loss function I am using? Beginners	1	375	May 25, 2025
Custom Training Loss Function for Seq2Seq BART Beginners	1	1790	July 21, 2023

How was self.loss_function implemented

Related topics