Michelle Lam commited on
Commit
48122fa
1 Parent(s): efceaac

Adds initial About page; updates feedback form link text

Browse files
indie_label_svelte/public/img/bernstein.jpeg ADDED
indie_label_svelte/public/img/gordon.jpeg ADDED
indie_label_svelte/public/img/hancock.png ADDED
indie_label_svelte/public/img/lam.jpg ADDED
indie_label_svelte/public/img/landay.jpeg ADDED
indie_label_svelte/public/img/metaxa.jpg ADDED
indie_label_svelte/public/img/temp.png ADDED
indie_label_svelte/src/About.svelte ADDED
@@ -0,0 +1,134 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <script lang="ts">
2
+ </script>
3
+
4
+ <svelte:head>
5
+ <title>About</title>
6
+ </svelte:head>
7
+
8
+ <div class="panel">
9
+ <div class="panel_contents">
10
+ <div>
11
+ <h3>About IndieLabel</h3>
12
+ <p class="body">
13
+ The IndieLabel deployment on this Hugging Face Space is a collaboration between researchers at the <a href="https://hci.stanford.edu/" target="_blank">Stanford HCI Group</a> and <a href="https://avidml.org/arva/" target="_blank">ARVA (AI Risk and Vulnerability Alliance)</a>. The IndieLabel tool is a research prototype designed to empower everyday users to lead large-scale investigations of harmful algorithmic behavior, and this Space is intended as a public-facing deployment of this prototype.
14
+ </p>
15
+
16
+ <div class="card-container">
17
+ <p class="head_5">IndieLabel & the End-User Audits paper</p>
18
+ <p class="body">
19
+ The <a href="https://github.com/StanfordHCI/indie-label" target="_blank">IndieLabel</a> tool was initially presented as a research prototype system in an academic publication called <b><i>End-User Audits: A System Empowering Communities to Lead Large-Scale Investigations of Harmful Algorithmic Behavior</i></b>, presented at CSCW 2022. The full paper is available on the <a href="https://dl.acm.org/doi/10.1145/3555625" target="_blank">ACM Digital Library</a> or via <a href="https://hci.stanford.edu/publications/2022/Lam_EndUserAudits_CSCW22.pdf" target="_blank">Stanford HCI</a>.
20
+ </p>
21
+ <p class="body">
22
+ The key question underlying the End-User Audits paper is: <i>how do we enable everyday users to lead large-scale algorithm audits</i>? IndieLabel aids users in rapidly identifying where they disagree with a system's behavior with the help of a personalized recommender system model.
23
+ </p>
24
+
25
+ <p class="body">
26
+ Our paper abstract summarizes our core motivation, approach, and findings:
27
+ "Because algorithm audits are conducted by technical experts, audits are necessarily limited to the hypotheses that experts think to test. End users hold the promise to expand this purview, as they inhabit spaces and witness algorithmic impacts that auditors do not. In pursuit of this goal, we propose end-user audits—system-scale audits led by non-technical users—and present an approach that scaffolds end users in hypothesis generation, evidence identification, and results communication. Today, performing a system-scale audit requires substantial user effort to label thousands of system outputs, so we introduce a collaborative filtering technique that leverages the algorithmic system's own disaggregated training data to project from a small number of end user labels onto the full test set. Our end-user auditing tool, IndieLabel, employs these predicted labels so that users can rapidly explore where their opinions diverge from the algorithmic system's outputs. By highlighting topic areas where the system is under-performing for the user and surfacing sets of likely error cases, the tool guides the user in authoring an audit report. In an evaluation of end-user audits on a popular comment toxicity model with 17 non-technical participants, participants both replicated issues that formal audits had previously identified and also raised previously underreported issues such as under-flagging on veiled forms of hate that perpetuate stigma and over-flagging of slurs that have been reclaimed by marginalized communities."
28
+ </p>
29
+
30
+ <p class="body">
31
+ The End-User Audits work was led by Stanford PhD student <a href="http://michelle123lam.github.io/" target="_blank">Michelle Lam</a> along with co-authors <a href="https://mgordon.me/" target="_blank">Mitchell Gordon</a>, <a href="https://metaxa.net/" target="_blank">Danaë Metaxa</a>, <a href="https://sml.stanford.edu/people/jeff-hancock" target="_blank">Jeffrey Hancock</a>, <a href="https://www.landay.org/" target="_blank">James Landay</a>, and <a href="https://hci.stanford.edu/msb/" target="_blank">Michael Bernstein</a>.
32
+ </p>
33
+
34
+ <div class="image-container">
35
+ <div>
36
+ <img src="./img/lam.jpg" alt="Michelle Lam">
37
+ <p><b>Michelle Lam</b><br/>PhD Student, Stanford CS</p>
38
+ </div>
39
+ <div>
40
+ <img src="./img/gordon.jpeg" alt="Mitchell Gordon">
41
+ <p><b>Mitchell Gordon</b><br/>Incoming Asst Professor, MIT EECS</p>
42
+ </div>
43
+ <div>
44
+ <img src="./img/metaxa.jpg" alt="Danaë Metaxa">
45
+ <p><b>Danaë Metaxa</b><br/>Asst Professor, UPenn CIS</p>
46
+ </div>
47
+ <div>
48
+ <img src="./img/hancock.png" alt="Jeffrey Hancock">
49
+ <p><b>Jeffrey Hancock</b><br/>Professor, Stanford Comm</p>
50
+ </div>
51
+ <div>
52
+ <img src="./img/landay.jpeg" alt="James Landay">
53
+ <p><b>James Landay</b><br/>Professor, Stanford CS</p>
54
+ </div>
55
+ <div>
56
+ <img src="./img/bernstein.jpeg" alt="Michael Bernstein">
57
+ <p><b>Michael Bernstein</b><br/>Assoc Professor, Stanford CS</p>
58
+ </div>
59
+ </div>
60
+ </div>
61
+
62
+ <div class="card-container">
63
+ <p class="head_5">ARVA</p>
64
+ <p class="body">
65
+ To fill in: information about ARVA and AVID.
66
+ </p>
67
+ </div>
68
+
69
+ <div class="card-container">
70
+ <p class="head_5">The Team</p>
71
+ <p class="body">
72
+ The IndieLabel Hugging Face deployment was made possible by a wonderful team of volunteers who worked on adapting IndieLabel for use by a general audience, connecting its reports to AVID (AI Vulnerability Database), and deploying it on Hugging Face Spaces. The team includes:
73
+ </p>
74
+ <div>
75
+ <div class="image-container">
76
+ <div>
77
+ <img src="./img/lam.jpg" alt="Michelle Lam">
78
+ <p><b>Michelle Lam</b></p>
79
+ </div>
80
+ <div>
81
+ <img src="./img/temp.png" alt="Carol Anderson">
82
+ <p><b>Carol Anderson</b></p>
83
+ </div>
84
+ <div>
85
+ <img src="./img/temp.png" alt="Christina Pan">
86
+ <p><b>Christina Pan</b></p>
87
+ </div>
88
+ <div>
89
+ <img src="./img/temp.png" alt="Nathan Butters">
90
+ <p><b>Nathan Butters</b></p>
91
+ </div>
92
+ </div>
93
+ </div>
94
+ </div>
95
+ </div>
96
+ </div>
97
+ </div>
98
+
99
+ <style>
100
+ a {
101
+ color: #333;
102
+ text-decoration: underline;
103
+ }
104
+
105
+ .head_5 {
106
+ margin: 10px 0;
107
+ }
108
+
109
+ .card-container {
110
+ margin: 40px 0;
111
+ }
112
+
113
+ .image-container {
114
+ margin: 20px 0;
115
+ display: flex;
116
+ flex-wrap: wrap;
117
+ justify-content: center;
118
+ gap: 20px; /* Adjust the gap between images */
119
+ }
120
+
121
+ .image-container img {
122
+ max-width: 110px; /* Adjust the maximum width of images */
123
+ height: 110px;
124
+ border-radius: 50%;
125
+ display: block;
126
+ }
127
+
128
+ .image-container p {
129
+ text-align: center;
130
+ margin-top: 5px; /* Adjust the margin between image and name */
131
+ max-width: 110px;
132
+ font-size: 11px;
133
+ }
134
+ </style>
indie_label_svelte/src/HypothesisPanel.svelte CHANGED
@@ -522,7 +522,7 @@
522
 
523
  <div class="feedback_section">
524
  <a href="https://forms.gle/vDXchpbBFjDeKjJA6" target="_blank" class="feedback_link">
525
- Share feedback
526
  </a>
527
  </div>
528
  </div>
 
522
 
523
  <div class="feedback_section">
524
  <a href="https://forms.gle/vDXchpbBFjDeKjJA6" target="_blank" class="feedback_link">
525
+ Share feedback about the tool
526
  </a>
527
  </div>
528
  </div>
indie_label_svelte/src/MainPanel.svelte CHANGED
@@ -1,9 +1,11 @@
1
  <script lang="ts">
2
  import Labeling from "./Labeling.svelte";
3
  import Auditing from "./Auditing.svelte";
 
4
 
5
  import Tab, { Label } from "@smui/tab";
6
  import TabBar from "@smui/tab-bar";
 
7
 
8
  export let model;
9
  export let error_type;
@@ -15,6 +17,8 @@
15
  let tab = searchParams.get("tab");
16
  if (tab == "auditing") {
17
  active = "auditing";
 
 
18
  }
19
 
20
  </script>
@@ -25,22 +29,31 @@
25
 
26
  <div class="auditing_panel">
27
  <div class="tab_header">
28
- <TabBar tabs={["labeling", "auditing"]} let:tab bind:active>
29
- <Tab {tab}>
30
- <Label>{tab}</Label>
 
 
 
 
 
31
  </Tab>
32
  </TabBar>
33
  </div>
34
 
35
  <div class="panel_contents">
36
  <div>
37
- <div id="labeling" hidden={active == "auditing"} >
38
  <Labeling cur_user={cur_user}/>
39
  </div>
40
 
41
- <div id="auditing" hidden={active == "labeling"} >
42
  <Auditing bind:personalized_model={model} bind:cur_error_type={error_type} cur_user={cur_user} on:change/>
43
  </div>
 
 
 
 
44
  </div>
45
 
46
  </div>
 
1
  <script lang="ts">
2
  import Labeling from "./Labeling.svelte";
3
  import Auditing from "./Auditing.svelte";
4
+ import About from "./About.svelte";
5
 
6
  import Tab, { Label } from "@smui/tab";
7
  import TabBar from "@smui/tab-bar";
8
+ import { Icon } from "@smui/common";
9
 
10
  export let model;
11
  export let error_type;
 
17
  let tab = searchParams.get("tab");
18
  if (tab == "auditing") {
19
  active = "auditing";
20
+ } else if (tab == "about") {
21
+ active = "about";
22
  }
23
 
24
  </script>
 
29
 
30
  <div class="auditing_panel">
31
  <div class="tab_header">
32
+ <TabBar tabs={["labeling", "auditing", "about"]} let:tab bind:active>
33
+ <Tab {tab} minWidth={tab == "about"}>
34
+ {#if tab == "about"}
35
+ <Icon class="material-icons">info_outlined</Icon>
36
+ {:else}
37
+ <Label>{tab}</Label>
38
+ {/if}
39
+
40
  </Tab>
41
  </TabBar>
42
  </div>
43
 
44
  <div class="panel_contents">
45
  <div>
46
+ <div id="labeling" hidden={active != "labeling"} >
47
  <Labeling cur_user={cur_user}/>
48
  </div>
49
 
50
+ <div id="auditing" hidden={active != "auditing"} >
51
  <Auditing bind:personalized_model={model} bind:cur_error_type={error_type} cur_user={cur_user} on:change/>
52
  </div>
53
+
54
+ <div id="about" hidden={active != "about"} >
55
+ <About />
56
+ </div>
57
  </div>
58
 
59
  </div>