Spaces:
Running
Running
initial update
Browse files- index.html +59 -51
- static/images/figure1_wma_overview.png +0 -0
index.html
CHANGED
|
@@ -1,26 +1,19 @@
|
|
| 1 |
<!DOCTYPE html>
|
| 2 |
-
<html>
|
| 3 |
<head>
|
| 4 |
-
<meta charset="
|
| 5 |
-
<meta name="description"
|
| 6 |
-
|
| 7 |
-
<meta name="
|
| 8 |
-
<
|
| 9 |
-
<
|
| 10 |
-
</title>
|
| 11 |
-
|
| 12 |
-
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
|
| 13 |
-
rel="stylesheet">
|
| 14 |
-
|
| 15 |
<link rel="stylesheet" href="./static/css/bulma.min.css">
|
| 16 |
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
|
| 17 |
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
|
| 18 |
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
|
| 19 |
-
<link rel="stylesheet"
|
| 20 |
-
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
|
| 21 |
<link rel="stylesheet" href="./static/css/index.css">
|
| 22 |
<link rel="icon" href="./static/images/favicon.svg">
|
| 23 |
-
|
| 24 |
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
|
| 25 |
<script defer src="./static/js/fontawesome.all.min.js"></script>
|
| 26 |
<script src="./static/js/bulma-carousel.min.js"></script>
|
|
@@ -36,59 +29,46 @@
|
|
| 36 |
<div class="column has-text-centered">
|
| 37 |
<h1 class="title is-1 publication-title">Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation</h1>
|
| 38 |
<div class="publication-links">
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
<a href="#">
|
| 43 |
-
|
| 44 |
-
<a href="#">
|
| 45 |
-
|
| 46 |
-
<a href="#">
|
| 47 |
-
|
| 48 |
-
<a href="#">
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
<span class="author-block">
|
| 54 |
-
<a href="#">Sunghwan Kim</a>,</span>
|
| 55 |
-
<span class="author-block">
|
| 56 |
-
<a href="#">Dongha Lee</a>,</span>
|
| 57 |
-
<span class="author-block">
|
| 58 |
-
<a href="#">Jinyoung Yeo</a></span>
|
| 59 |
-
</div>
|
| 60 |
-
<div class="is-size-5 publication-authors"></div>
|
| 61 |
-
<span class="author-block">Yonsei University</span>
|
| 62 |
</div>
|
| 63 |
-
|
| 64 |
<div class="column has-text-centered">
|
| 65 |
<div class="publication-links">
|
| 66 |
<!-- Paper Link -->
|
| 67 |
<span class="link-block">
|
| 68 |
-
<a href="https://arxiv.org/abs/
|
| 69 |
-
class="external-link button is-normal is-rounded is-dark">
|
| 70 |
<span class="icon">
|
| 71 |
-
|
| 72 |
</span>
|
| 73 |
<span>Paper</span>
|
| 74 |
</a>
|
| 75 |
</span>
|
| 76 |
<!-- Code Link -->
|
| 77 |
<span class="link-block">
|
| 78 |
-
<a href="https://
|
| 79 |
-
class="external-link button is-normal is-rounded is-dark">
|
| 80 |
<span class="icon">
|
| 81 |
-
|
| 82 |
</span>
|
| 83 |
<span>Code</span>
|
| 84 |
</a>
|
| 85 |
</span>
|
| 86 |
<!-- Demo Link -->
|
| 87 |
<span class="link-block">
|
| 88 |
-
<a href="https://huggingface.co/spaces/hyungjoochae/WMA-Agent-Demo" target="_blank"
|
| 89 |
-
class="external-link button is-normal is-rounded is-dark">
|
| 90 |
<span class="icon">
|
| 91 |
-
|
| 92 |
</span>
|
| 93 |
<span>Demo</span>
|
| 94 |
</a>
|
|
@@ -101,7 +81,35 @@
|
|
| 101 |
</div>
|
| 102 |
</section>
|
| 103 |
|
| 104 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
|
| 106 |
-
|
| 107 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
<!DOCTYPE html>
|
| 2 |
+
<html lang="en">
|
| 3 |
<head>
|
| 4 |
+
<meta charset="UTF-8">
|
| 5 |
+
<meta name="description" content="World-model-augmented (WMA) web agent that simulates action outcomes for better decision-making in long-horizon web tasks.">
|
| 6 |
+
<meta name="keywords" content="Web Agents, World Models, LLMs, Web Navigation, Autonomous Agents">
|
| 7 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
| 8 |
+
<title>Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation</title>
|
| 9 |
+
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" rel="stylesheet">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
<link rel="stylesheet" href="./static/css/bulma.min.css">
|
| 11 |
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
|
| 12 |
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
|
| 13 |
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
|
| 14 |
+
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
|
|
|
|
| 15 |
<link rel="stylesheet" href="./static/css/index.css">
|
| 16 |
<link rel="icon" href="./static/images/favicon.svg">
|
|
|
|
| 17 |
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
|
| 18 |
<script defer src="./static/js/fontawesome.all.min.js"></script>
|
| 19 |
<script src="./static/js/bulma-carousel.min.js"></script>
|
|
|
|
| 29 |
<div class="column has-text-centered">
|
| 30 |
<h1 class="title is-1 publication-title">Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation</h1>
|
| 31 |
<div class="publication-links">
|
| 32 |
+
<div class="is-size-5 publication-authors">
|
| 33 |
+
<span class="author-block"><a href="#">Hyungjoo Chae</a>,</span>
|
| 34 |
+
<span class="author-block"><a href="#">Namyoung Kim</a>,</span>
|
| 35 |
+
<span class="author-block"><a href="#">Kai Tzu-iunn Ong</a>,</span>
|
| 36 |
+
<span class="author-block"><a href="#">Minju Gwak</a>,</span>
|
| 37 |
+
<span class="author-block"><a href="#">Gwanwoo Song</a>,</span>
|
| 38 |
+
<span class="author-block"><a href="#">Jihoon Kim</a>,</span>
|
| 39 |
+
<span class="author-block"><a href="#">Sunghwan Kim</a>,</span>
|
| 40 |
+
<span class="author-block"><a href="#">Dongha Lee</a>,</span>
|
| 41 |
+
<span class="author-block"><a href="#">Jinyoung Yeo</a></span>
|
| 42 |
+
</div>
|
| 43 |
+
<div class="is-size-5 publication-affiliations">
|
| 44 |
+
<span class="affiliation-block">Yonsei University</span>
|
| 45 |
+
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
</div>
|
|
|
|
| 47 |
<div class="column has-text-centered">
|
| 48 |
<div class="publication-links">
|
| 49 |
<!-- Paper Link -->
|
| 50 |
<span class="link-block">
|
| 51 |
+
<a href="https://arxiv.org/abs/2410.13232" target="_blank" class="external-link button is-normal is-rounded is-dark">
|
|
|
|
| 52 |
<span class="icon">
|
| 53 |
+
<i class="fas fa-file-pdf"></i>
|
| 54 |
</span>
|
| 55 |
<span>Paper</span>
|
| 56 |
</a>
|
| 57 |
</span>
|
| 58 |
<!-- Code Link -->
|
| 59 |
<span class="link-block">
|
| 60 |
+
<a href="https://github.com/kyle8581/WMA-Agents" target="_blank" class="external-link button is-normal is-rounded is-dark">
|
|
|
|
| 61 |
<span class="icon">
|
| 62 |
+
<i class="fab fa-github"></i>
|
| 63 |
</span>
|
| 64 |
<span>Code</span>
|
| 65 |
</a>
|
| 66 |
</span>
|
| 67 |
<!-- Demo Link -->
|
| 68 |
<span class="link-block">
|
| 69 |
+
<a href="https://huggingface.co/spaces/hyungjoochae/WMA-Agent-Demo" target="_blank" class="external-link button is-normal is-rounded is-dark">
|
|
|
|
| 70 |
<span class="icon">
|
| 71 |
+
<i class="fas fa-play"></i>
|
| 72 |
</span>
|
| 73 |
<span>Demo</span>
|
| 74 |
</a>
|
|
|
|
| 81 |
</div>
|
| 82 |
</section>
|
| 83 |
|
| 84 |
+
<section class="hero is-light">
|
| 85 |
+
<div class="hero-body">
|
| 86 |
+
<div class="container is-max-desktop">
|
| 87 |
+
<div class="columns is-centered">
|
| 88 |
+
<div class="column">
|
| 89 |
+
<div class="content has-text-centered">
|
| 90 |
+
<img src="./static/images/figure1_wma_overview.png" alt="WMA Overview">
|
| 91 |
+
</div>
|
| 92 |
+
</div>
|
| 93 |
+
</div>
|
| 94 |
+
</div>
|
| 95 |
+
</div>
|
| 96 |
+
</section>
|
| 97 |
+
|
| 98 |
+
<section class="section">
|
| 99 |
+
<div class="container is-max-desktop">
|
| 100 |
+
<div class="columns is-centered">
|
| 101 |
+
<div class="column">
|
| 102 |
+
<h2 class="title is-2">Abstract</h2>
|
| 103 |
+
<p>Large language models (LLMs) have recently gained much attention in building autonomous agents. However, the performance of current LLM-based web agents in long-horizon tasks is far from optimal, often yielding errors such as repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid such an irreversible mistake, as we have an awareness of the potential outcomes (e.g., losing money) of our actions, also known as the "world model." Motivated by this, our study first starts with preliminary analyses, confirming the absence of world models in current LLMs (e.g., GPT-4, Claude-3.5-Sonnet, etc.). Then, we present a World-Model-Augmented (WMA) web agent, which simulates the outcomes of its actions for better decision-making. To overcome the challenges in training LLMs as world models predicting next observations, such as repeated elements across observations and long HTML inputs, we propose a transition-focused observation abstraction, where the prediction objectives are free-form natural language descriptions exclusively highlighting important state differences between time steps. Experiments on WebArena and Mind2Web show that our world models improve agents' policy selection without training and demonstrate our agents' cost- and time-efficiency compared to recent tree-search-based agents.</p>
|
| 104 |
+
</div>
|
| 105 |
+
</div>
|
| 106 |
+
</div>
|
| 107 |
+
</section>
|
| 108 |
|
| 109 |
+
<section class="section">
|
| 110 |
+
<div class="container is-max-desktop">
|
| 111 |
+
<div class="columns is-centered">
|
| 112 |
+
<div class="column">
|
| 113 |
+
<h2 class="title is-2
|
| 114 |
+
::contentReference[oaicite:2]{index=2}
|
| 115 |
+
|
static/images/figure1_wma_overview.png
ADDED
|