Papers
arxiv:2305.06355

VideoChat: Chat-Centric Video Understanding

Published on May 10, 2023
· Submitted by akhaliq on May 11, 2023
#2 Paper of the day
Authors:
,
,
,
,

Abstract

In this study, we initiate an exploration into video understanding by introducing VideoChat, an end-to-end chat-centric video understanding system. It integrates video foundation models and large language models via a learnable neural interface, excelling in spatiotemporal reasoning, event localization, and causal relationship inference. To instructively tune this system, we propose a video-centric instruction dataset, composed of thousands of videos matched with detailed descriptions and conversations. This dataset emphasizes spatiotemporal reasoning and causal relationships, providing a valuable asset for training chat-centric video understanding systems. Preliminary qualitative experiments reveal our system's potential across a broad spectrum of video applications and set the standard for future research. Access our code and data at https://github.com/OpenGVLab/Ask-Anything

Community

This comment has been hidden

Sign up or log in to comment

Models citing this paper 10

Browse 10 models citing this paper

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2305.06355 in a dataset README.md to link it from this page.

Spaces citing this paper 2

Collections including this paper 1