Multiple choice task

#876
by feministmystique - opened

Hi there!

I am trying to test some model's response to multiple choice questions using a pipeline (I don't need to train it but simply record its answer), and I was wondering what the best way to go about doing this. I know there is a specific task called question-answering; however, I am pretty sure this task isn't meant to be used for answering multiple choice questions, especially as the context part doesn't really make sense when I need receiving a single letter (A-D).

Please let me know what you think; I will be very grateful for your help!

Open LLM Leaderboard org

Hi @feministmystique ,

Thank you for your question!

Unfortunately, for the inquiries related to the use of the Transformers library, especially about handling specific tasks like multiple-choice questions, we would recommend checking the Transformers documentation.

If you don’t find the answers you’re looking for there, you might consider opening an issue on the Transformers GitHub repository to get support from the community and developers. They can provide more detailed guidance tailored to your specific situation.

alozowski changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment