How was self.loss_function implemented

[OmarSamir]

Hi, I was curious about how the self.loss_function is implemented in the Qwen2.5-VL model to compute the loss during training.
Could someone explain how it works or point me to the relevant part of the code?

Here’s the link to the line I’m referring to:

src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py

Thanks in advance!

[John6666]

Maybe this?

Where to look for a loss definition for a pretrained model?

I am using facebook/opt-350m model: model = AutoModelForCausalLM.from_pretrained("facebook/opt-350m") As far as I understand, its default loss is the crossentropy loss. But how can I verify it, and where can I see the implementation details? Thank you.

Having troubel in understanding what loss is currently in use

I was going through this hugging face code and I am having trouble understanding what loss the model is currently using. Although I know most seq2seq models uses CrossEntrophy loss but I don’t see the definition anywhere in the code Actually I want to train the model with a new custom loss. I have trained a baseline model and its working fine. Thank You

[OmarSamir]

Thank you so much for sharing. However, these issues predated the Transformers version 4.53.0.dev0. What I want to know is where the self.loss_function was implemented for these models so I can modify it correctly.

Thank you!

[OmarSamir]

The loss functions are defined in src/transformers/loss/loss_utils.py. The logic for selecting which loss function to use is implemented in the PreTrainedModel class, located in src/transformers/modeling_utils.py.

link: transformers/src/transformers/loss/loss_utils.py at main · huggingface/transformers · GitHub
link: transformers/src/transformers/modeling_utils.py at main · huggingface/transformers · GitHub