Model,Type,Size,F/FPS,Dialogue_SA,Dialogue_CC,Dialogue_LC,Dialogue_TU,Dialogue_IC,Dialogue_OS,Streaming_SA,Streaming_CC,Streaming_LC,Streaming_TU,Streaming_IC,Streaming_OS,Average
MovieChat,VideoLLM,7B,2048,20.36,23.74,28.97,22.80,20.51,22.72,18.92,22.38,26.77,20.46,20.98,19.64,21.18
Video-ChatGPT,VideoLLM,7B,100,28.01,34.04,40.89,35.66,29.59,32.24,22.84,28.44,33.93,26.31,26.43,25.02,28.63
Video-LLaVA,VideoLLM,7B,8,31.85,38.38,44.93,41.54,32.80,36.49,26.95,33.68,39.00,31.83,31.53,29.89,33.19
TimeChat,VideoLLM,7B,16,31.09,38.57,45.52,43.37,31.10,36.24,27.14,34.42,39.78,36.80,31.71,31.15,33.70
LLaVA-NeXT-Video,VideoLLM,7B,16,37.71,44.59,52.05,41.80,36.58,41.40,34.29,39.68,47.65,35.33,36.68,36.12,38.76
ShareGPT4Video,VideoLLM,8B,16,36.26,43.68,50.12,47.33,37.25,41.76,33.14,40.48,46.01,38.15,37.81,37.10,39.43
Flash-VStream,VideoLLM,7B,8,37.54,44.74,51.02,47.95,37.94,42.72,35.71,44.24,48.49,38.95,39.00,38.80,40.76
InternVL2,ImageLLM,8B,8,40.53,46.77,52.38,46.97,40.35,44.48,38.92,45.42,50.45,41.53,42.35,41.62,43.05
VILA,ImageLLM,8B,8,43.23,49.30,55.59,52.47,41.27,47.07,38.19,44.27,49.18,41.29,40.55,40.38,43.73
VideoLLaMA2,VideoLLM,7B,8,42.50,49.88,55.96,52.23,41.40,47.10,38.95,46.11,51.77,43.69,42.22,42.77,44.94
StreamingTalking,VideoLLM,7B,1fps,43.16,50.55,59.34,54.53,42.43,48.50,46.10,52.67,58.11,48.32,48.40,44.54,46.52
InternVL2.5,ImageLLM,8B,8,43.73,50.70,56.61,55.03,43.46,48.73,40.44,48.34,52.84,46.93,45.10,45.04,46.89
InternVideo2.5,VideoLLM,8B,128,46.83,53.48,58.22,58.91,47.02,51.73,41.76,49.72,53.25,48.44,47.10,46.58,49.16
InternLM-XC2.5,VideoLLM,7B,32,46.51,53.16,59.84,52.94,45.87,50.71,52.62,58.55,62.89,53.98,54.39,54.39,52.55
MiniCPM-V 2.6,ImageLLM,8B,64,51.70,59.50,65.33,61.72,50.09,56.63,46.44,52.73,58.35,53.48,48.32,49.67,53.15
Qwen2-VL,ImageLLM,7B,8,50.47,57.71,63.46,60.77,49.44,55.29,48.38,55.17,59.91,52.04,51.42,51.39,53.34
Qwen2.5-VL,ImageLLM,7B,8,52.54,59.85,65.52,64.64,51.23,57.57,48.21,56.12,60.31,56.33,52.46,52.84,55.21
LiveStar,VideoLLM,8B,1fps,54.06,61.08,66.43,66.06,52.67,58.95,52.19,59.00,62.85,58.35,54.95,55.87,57.41
Gemini 1.5 Pro,-,-,1fps,49.07,56.15,62.24,58.36,47.72,53.68,49.35,55.77,60.41,52.89,51.11,51.55,52.62
GPT-4V,-,-,10,56.03,62.61,69.09,65.36,53.73,60.30,56.37,61.41,65.80,59.18,57.16,57.93,59.12
GPT-4o,-,-,25,58.26,64.76,70.75,67.68,55.82,62.57,57.99,63.52,67.72,60.18,59.25,59.97,61.27