How to use ModernBERT with the AutoModelForQuestionAnswering class?
#15
by
sraj
- opened
I wanted to fine tune the model on the SQUAD dataset. But currently AutoModelForQuestionAnswering does not work with ModernBERT. Is there a workaround for the moment where I can train ModernBERT for QA tasks given a context?
Hey! This is planned, but indeed missing at the moment.
cc @NohTow @bwarner @wgpubs @tomaarsen , we should ensure our January update includes the missing heads (QA, MultipleChoice)
Thank you very much for ModernBERT!
@bclavie
It will be amazing to have TokenClassification also. : )