How did you preprocess the datasets to remove Copyrighted material?
Hey, this line is mentioned in the Training Data section:
"The publicly available datasets were preprocessed to filter out texts with copyright issues, and all datasets were preprocessed to remove sensitive information"
I have a couple of questions.
How did you go about doing this?
- What metric/filter did you use to determine something with a copyright issue?
- Do you have a paper on the training process or the filtering process?
- Do you have documentation on exactly what data was used to train this model? VG, NRK and Schibsted are mentioned, which indicates that articles published through these might have been used to train this LLM. Could you clarify what that means?
Before using models such as NorwAI-Mistral-7B it's essential that the associated Copyright and personal data/sensitive data risks are known. The documentation is very vague on these subjects
(Disclaimer: Not affiliated with the project)
Do you have documentation on exactly what data was used to train this model? VG, NRK and Schibsted are mentioned, which indicates that articles published through these might have been used to train this LLM. Could you clarify what that means?
The launch announcement (6:15) clarifies that the models are mainly based on the Norwegian Colossal Corpus from the National Library. VG, NRK and Schibsted are listed as collaborators, which I would take to mean they've granted NTNU license to use their archives for the purpose of model training, similar to what the National Library is describing for their Mimir Extended dataset (8:00).
Hey jlu, I believe you have done a classic mistake of mixing NorwAI and NORA LLM. (Don't worry, you're not the only one)
Though believe me when I say that NORA LLM has their own Copyright problems