yujinyujin9393 commited on
Commit
4810abe
·
verified ·
1 Parent(s): 3704b12

Add instruction for how to add a benchmark

Browse files
Files changed (2) hide show
  1. README.md +29 -1
  2. about.md +29 -1
README.md CHANGED
@@ -17,7 +17,35 @@ Tracking AI capabilities in cybersecurity is essential for understanding emergin
17
 
18
  ## Submit your benchmark
19
 
20
- This leaderboard is a collection of cybersecurity-relevant benchmarks. To submit your benchmark, please use this: https://docs.google.com/forms/d/e/1FAIpQLSd0arYQ0xy9FpGbXwu68rAFpCm0HNb-8ZK8Mma3Ru2oa2Astg/viewform. We will regularly update this leaderboard.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  ## Paper & Blog
23
 
 
17
 
18
  ## Submit your benchmark
19
 
20
+ Please follow the steps below to add your benchmark.
21
+
22
+ 1. First you need to add your results in results.json. Under the top-level "results" key, you need to insert an entry that looks like this:
23
+
24
+ ```jsonc
25
+ "Your Benchmark Name": {
26
+ "Metric Name 1": {
27
+ "Model / Agent Name": [value]
28
+ },
29
+ "Metric Name 2": {
30
+ "Model / Agent Name": [value]
31
+ }
32
+ }
33
+ ```
34
+
35
+ Here, if you want, you can add multiple metric scores.
36
+
37
+ 2. Then, add descriptive metadata in meta_data.py
38
+
39
+ ```bash
40
+ LEADERBOARD_MD["Your Benchmark Name"] = """
41
+ Brief description of what the benchmark measures.
42
+
43
+ Paper: <paper URL>
44
+ Code: <repository URL>
45
+ """
46
+ ```
47
+
48
+ 3. Lastly, please open a pull request. You need to commit your changes and open a PR against this repository. We will review and merge submissions. If you have any questions, please contact Yujin Potter at [email protected].
49
 
50
  ## Paper & Blog
51
 
about.md CHANGED
@@ -2,7 +2,35 @@ Tracking AI capabilities in cybersecurity is essential for understanding emergin
2
 
3
  ## Submit your benchmark
4
 
5
- This leaderboard is a collection of cybersecurity-relevant benchmarks. To submit your benchmark, please use this: https://docs.google.com/forms/d/e/1FAIpQLSd0arYQ0xy9FpGbXwu68rAFpCm0HNb-8ZK8Mma3Ru2oa2Astg/viewform. We will regularly update this leaderboard.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
  ## Paper & Blog
8
 
 
2
 
3
  ## Submit your benchmark
4
 
5
+ Please follow the steps below to add your benchmark.
6
+
7
+ 1. First you need to add your results in results.json. Under the top-level "results" key, you need to insert an entry that looks like this:
8
+
9
+ ```jsonc
10
+ "Your Benchmark Name": {
11
+ "Metric Name 1": {
12
+ "Model / Agent Name": [value]
13
+ },
14
+ "Metric Name 2": {
15
+ "Model / Agent Name": [value]
16
+ }
17
+ }
18
+ ```
19
+
20
+ Here, if you want, you can add multiple metric scores.
21
+
22
+ 2. Then, add descriptive metadata in meta_data.py
23
+
24
+ ```bash
25
+ LEADERBOARD_MD["Your Benchmark Name"] = """
26
+ Brief description of what the benchmark measures.
27
+
28
+ Paper: <paper URL>
29
+ Code: <repository URL>
30
+ """
31
+ ```
32
+
33
+ 3. Lastly, please open a pull request. You need to commit your changes and open a PR against this repository. We will review and merge submissions. If you have any questions, please contact Yujin Potter at [email protected].
34
 
35
  ## Paper & Blog
36