Update README.md
Browse files
README.md
CHANGED
@@ -55,9 +55,9 @@ The pre-processing operations used to produce the final training dataset were as
|
|
55 |
|
56 |
1. Dataset is filtered based on 'medium' value in 'strategy' column (sequence length = 85).
|
57 |
2. For ClimateWatch, all rows are removed as there was assessed to be no taxonomical alignment with the IKITracs labels inherent to the dataset. For IKITracs, labels are assigned based on the presence of certain substrings based on 'parameter' values which correspond to assessments of Net-Zero targets by human annotaters. The specific assignments are as follows:
|
58 |
-
>
|
59 |
-
>
|
60 |
-
>
|
61 |
3. If 'context_translated' is available and the 'language' is not English, 'context' is replaced with 'context_translated'.
|
62 |
4. The dataset is "exploded" - i.e., the text samples in the 'context' column, which are lists, are converted into separate rows - and labels are merged to align with the associated samples.
|
63 |
5. The 'match_onanswer' and 'answerWordcount' are used conditionally to select high quality samples (prefers high % of word matches in 'match_onanswer', but will take lower if there is a high 'answerWordcount')
|
|
|
55 |
|
56 |
1. Dataset is filtered based on 'medium' value in 'strategy' column (sequence length = 85).
|
57 |
2. For ClimateWatch, all rows are removed as there was assessed to be no taxonomical alignment with the IKITracs labels inherent to the dataset. For IKITracs, labels are assigned based on the presence of certain substrings based on 'parameter' values which correspond to assessments of Net-Zero targets by human annotaters. The specific assignments are as follows:
|
58 |
+
> - 'NET-ZERO': target_labels = ['T_Netzero','T_Netzero_C']
|
59 |
+
> - 'NEGATIVE': target_labels_neg = ['T_Economy_C','T_Economy_Unc','T_Adaptation_C','T_Adaptation_Unc','T_Transport_C','T_Transport_O_C','T_Transport_O_Unc','T_Transport_Unc']
|
60 |
+
> - 'TARGET_FREE': random sample of other (non-target) labeled data
|
61 |
3. If 'context_translated' is available and the 'language' is not English, 'context' is replaced with 'context_translated'.
|
62 |
4. The dataset is "exploded" - i.e., the text samples in the 'context' column, which are lists, are converted into separate rows - and labels are merged to align with the associated samples.
|
63 |
5. The 'match_onanswer' and 'answerWordcount' are used conditionally to select high quality samples (prefers high % of word matches in 'match_onanswer', but will take lower if there is a high 'answerWordcount')
|