File size: 6,023 Bytes
992d47a
29924b3
108d6f8
c2e844a
 
b281d64
 
 
c2e844a
b281d64
c2e844a
4f2d396
 
c2e844a
 
b281d64
 
87bb912
 
c2e844a
87bb912
0078838
87bb912
 
 
 
 
 
 
 
 
71a842d
87bb912
 
 
 
 
 
 
 
 
71a842d
87bb912
 
 
 
 
 
 
b281d64
 
 
 
 
 
a3ec126
 
c2e844a
b281d64
 
 
c2e844a
b281d64
 
 
c2e844a
 
108d6f8
b281d64
 
 
 
 
 
 
 
4b8fcf8
bba381d
4b8fcf8
bba381d
346b9ec
 
015e45c
346b9ec
 
015e45c
 
108d6f8
346b9ec
015e45c
bba381d
 
346b9ec
 
015e45c
346b9ec
015e45c
bba381d
015e45c
 
346b9ec
015e45c
bba381d
346b9ec
015e45c
bba381d
346b9ec
 
 
 
108d6f8
bba381d
108d6f8
 
 
 
015e45c
ad71e5b
8ae1470
ef158e8
8ae1470
 
 
 
a013f34
8ae1470
 
b281d64
 
 
8ae1470
 
 
 
a013f34
8ae1470
 
f28037a
 
8ae1470
e0b70ca
 
 
 
 
 
 
 
3f090e4
e0b70ca
 
 
 
 
 
f28037a
 
b281d64
0291335
 
8ae1470
 
 
992d47a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
import streamlit as st
import pandas as pd
import streamlit.components.v1 as components


def scoring_section():
    # Title
    st.title("Scoring")

    # Intro text
    st.write(
        "Our scoring system rewards performance in all three challenges, with extra emphasis on the Evaluation Challenge. "
        "A team's final rank is determined by the total points they earn across the challenges."
    )
    
    # Points Breakdown in a table
    st.markdown("#### Points Breakdown")
    # Create three columns for a more interesting layout
    col1, col2, col3 = st.columns(3)
    
    with col1:
        st.markdown('<h3 style="margin-left:20px;">Compression</h3>', unsafe_allow_html=True)
        st.markdown(
            """
            - **1st Place**: 10 points  
            - **2nd Place**: 7 points  
            - **3rd Place**: 5 points
            """
        )
        
    with col2:
        st.markdown('<h3 style="margin-left:20px;">Sampling</h3>', unsafe_allow_html=True)
        st.markdown(
            """
            - **1st Place**: 10 points  
            - **2nd Place**: 7 points  
            - **3rd Place**: 5 points
            """
        )
        
    with col3:
        st.markdown('<h3 style="margin-left:20px;">Evaluation</h3>', unsafe_allow_html=True)
        st.markdown(
            """
            - **1st Place**: 20 points  
            - **2nd Place**: 14 points  
            - **3rd Place**: 10 points
            """
        )
    # Tie-Breakers in an expander for a cleaner layout
    with st.expander("Tie-Breakers"):
        st.write(
            "The overall winner will be the team with the highest total points. "
            "In the event of a tie, the following tie-breakers will be applied in order:\n\n"
            "1. Highest Evaluation Challenge score\n"
            "2. Highest Sampling Challenge score\n"
            "3. Highest Compression Challenge score\n\n"
        )

    # Overall Leaderboard Section
    st.markdown("#### Overall Leaderboard")
    st.write(
        "The overall leaderboard, which shows the total points across all challenges, will go live on **March 10th**. "
        "Additionally, each challenge—**Compression**, **Sampling**, and **Evaluation**—will have its own leaderboard on their "
        "respective Hugging Face submission servers. Below is a preview of the overall leaderboard format:"
    )


    data = {
        "Rank": [1, 2, 3],
        "Team Name": ["Team Alpha", "Team Beta", "Team Gamma"],
        "Compression Points": [10, 7, 5],
        "Sampling Points": [10, 7, 5],
        "Evaluation Points": [20, 14, 10],
        "Total Points": [40, 28, 20],
    }
    
    df = pd.DataFrame(data)

    # Custom CSS for a dark-themed table
    table_css = """
    <style>
    .dark-table {
      border-collapse: collapse;
      width: 100%;
      font-size: 1em;
      margin: 10px 0;
      color: #dddddd;
    }
    .dark-table th {
      background-color: #333333;
      color: #ffffff;
      padding: 8px;
      text-align: center;
      font-weight: bold;
    }
    .dark-table td {
      border: 1px solid #444444;
      padding: 8px;
      text-align: center;
    }
    .dark-table tr:nth-child(even) {
      background-color: #2b2b2b;
    }
    .dark-table tr:nth-child(odd) {
      background-color: #1f1f1f;
    }
    </style>
    """

    # Convert DataFrame to HTML with the custom CSS class, hiding the index
    styled_html = df.to_html(index=False, classes="dark-table")
    html_code = table_css + styled_html

    # Use st.components.v1.html to render the table
    components.html(html_code, height=300)


def main():
    st.set_page_config(page_title="World Model Challenge")
    
    st.title("World Model Challenge")
    st.markdown("### Welcome")
    st.write(
        "Welcome to the World Model Challenge server. This platform hosts three challenges "
        "designed to advance research in world models for robotics: Compression, Sampling, and Evaluation."
    )

    st.markdown("---")

    st.markdown("### Motivation")
    st.write(
        "Real-world robotics faces a fundamental challenge: environments are dynamic and change over time, "
        "making consistent evaluation of robot performance difficult. World models offer a solution by "
        "learning to simulate complex real-world interactions from raw sensor data. We believe these learned simulators will enable "
        "robust evaluation and iterative improvement of robot policies without the constraints of a physical testbed."
    )
    st.markdown("---")

    st.markdown("### The Challenges")

    st.markdown("#### Compression Challenge")
    st.write(
        "In the Compression Challenge, your task is to train a model to compress our robots logs effectively while preserving the critical details needed to understand and predict future interactions. Success in this challenge is measured by the loss of your model—the lower the loss, the better your model captures the complexities of real-world robot behavior."
    )
    
    st.markdown("#### Sampling Challenge")
    st.write(
        "In the Sampling Challenge, your task is to predict a future video frame two seconds in the future given a short clip of robot interactions. The goal is to produce a coherent and plausible continuation of the video, which accurately reflects the dynamics of the scene. Your submission will be judged on how closely it matches the actual frame."
    )
    
    st.markdown("#### Evaluation Challenge")
    st.write(
        "The Evaluation Challenge tackles the ultimate question: Can you predict a robot's performance in the real world without physically deploying it? In this challenge, you will be provided with many different policies for a specific task. Your task is to rank these policies according to their expected real-world performance. This ranking will be compared with the actual ranking of the policies."
    )
    st.markdown("---")

    scoring_section()


    
if __name__ == '__main__':
    main()