pfr commited on
Commit
fd7ea9e
1 Parent(s): 958080c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -1
README.md CHANGED
@@ -4,4 +4,38 @@ inference:
4
  function_to_apply: "none"
5
  widget:
6
  - text: "I cuddled with my dog today."
7
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  function_to_apply: "none"
5
  widget:
6
  - text: "I cuddled with my dog today."
7
+ ---
8
+
9
+ # Utilitarian Roberta 01
10
+
11
+ ## Model description
12
+
13
+ This is a [Roberta model](https://huggingface.co/roberta-large) fine-tuned on for computing utility estimates of experiences, represented in first-person sentences. It was trained from human-annotated pairwise utility comparisons, from the [ETHICS dataset](https://arxiv.org/abs/2008.02275).
14
+
15
+ ## Intended use
16
+
17
+ The main use case is the computation of utility estimates of first-person text scenarios.
18
+
19
+ ## Limitations
20
+
21
+ The model was only trained on a limited number of scenarios, and only on first-person sentences. It does not have the capability of interpreting highly complex or unusual scenarios, and it does not have hard guarantees on its domain of accuracy.
22
+
23
+ ## How to use
24
+
25
+ The model receives a sentence describing a scenario in first-person, and outputs a scalar representing a utility estimate.
26
+
27
+ ## Training data
28
+
29
+ The training data is the train split from the Utilitarianism part of the [ETHICS dataset](https://arxiv.org/abs/2008.02275).
30
+
31
+ ## Training procedure
32
+
33
+ Training can be reproduced by executing the training procedure from [`tune.py`](https://github.com/hendrycks/ethics/blob/3e4c09259a1b4022607da093e9452383fc1bb7e3/utilitarianism/tune.py) as follows:
34
+
35
+ ```
36
+ python tune.py --ngpus 1 --model roberta-large --learning_rate 1e-5 --batch_size 16 --nepochs 2
37
+ ```
38
+
39
+ ## Evaluation results
40
+
41
+ The model achieves 90.8% accuracy on [The Moral Uncertainty Research Competition](https://moraluncertainty.mlsafety.org/), which consists of a subset of the ETHICS dataset.