jadechoghari commited on
Commit
a9b005e
1 Parent(s): b8ac6f6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +42 -3
README.md CHANGED
@@ -1,3 +1,42 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+ # VidToMe: Video Token Merging for Zero-Shot Video Editing
5
+
6
+ Edit videos instantly with just a prompt! 🎥
7
+
8
+ Diffusers Implementation of VidToMe is a diffusion-based pipeline for zero-shot video editing that enhances temporal consistency and reduces memory usage by merging self-attention tokens across video frames.
9
+ This approach allows for a harmonious video generation and editing without needing to fine-tune the model.
10
+ By aligning and compressing redundant tokens across frames, VidToMe ensures smooth transitions and coherent video output, improving over traditional video editing methods.
11
+ It follows by [this paper](https://arxiv.org/abs/2312.10656).
12
+
13
+ ## Usage
14
+
15
+ ```python
16
+ from diffusers import DiffusionPipeline
17
+
18
+ # load the pretrained model
19
+ pipeline = DiffusionPipeline.from_pretrained("jadechoghari/VidToMe", trust_remote_code=True, custom_pipeline="jadechoghari/VidToMe", sd_version="depth", device="cuda", float_precision="fp16")
20
+
21
+ # Edit a video with prompts
22
+ pipeline(
23
+ video_path="path/to/video.mp4",
24
+ video_prompt="A serene beach scene",
25
+ edit_prompt="Make the sunset more vibrant",
26
+ control_type="depth",
27
+ n_timesteps=50
28
+ )
29
+ ```
30
+
31
+ ## Applications:
32
+ - Zero-shot video editing for content creators
33
+ - Video transformation using natural language prompts
34
+ - Memory-optimized video generation for longer or complex sequences
35
+
36
+ **Model Authors:**
37
+ - Xirui Li
38
+ - Chao Ma
39
+ - Xiaokang Yang
40
+ - Ming-Hsuan Yang
41
+
42
+ For more check the [Github Repo](https://github.com/lixirui142/VidToMe).