The latest articles on DEV Community by wree (@_1f0995eba7c81ed78c499). https://dev.to/_1f0995eba7c81ed78c499 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F2569341%2F2c9a7b72-ffb0-4701-ac86-8dbe33b49ae0.jpg https://dev.to/_1f0995eba7c81ed78c499 en wree Sat, 14 Dec 2024 07:56:50 +0000 https://dev.to/_1f0995eba7c81ed78c499/issue-with-mismatched-tensor-sizes-during-training-with-deepspeed-506k https://dev.to/_1f0995eba7c81ed78c499/issue-with-mismatched-tensor-sizes-during-training-with-deepspeed-506k <p>I'm currently training a model using** Hugging Face**'s Trainer with DeepSpeed integration, and I'm encountering an error related to mismatched tensor sizes. Specifically, I am getting the following error:</p> <h2> The size of tensor a (50) must match the size of tensor b (3) at non-singleton dimension2 </h2> <p>I hope someone can fix it and share your version please!😊😊😊</p> <p><a href="https://drive.google.com/file/d/1hjrvCaQFVE-EUk_-i_VEo950XY9F70tW/view?usp=drive_link" rel="noopener noreferrer">my data: </a></p> <p><a href="https://colab.research.google.com/drive/1KUO_g239Ii7kdG_4j8L_jE0XbgrEm9uS?usp=sharing" rel="noopener noreferrer">Here is my setup:</a></p> <p>I’ve checked that the input_ids and labels have the same shape. I've verified the batch size in both the Trainer configuration and the DeepSpeed config. I've also ensured that the model is correctly placed on the device (cuda or cpu).</p> <p>maybe, i'm not sure.</p> ai python discuss development