Video summarization similar to Summe's TextRank

5

1

We have the popular TextRank API which given a text, ranks keywords and can apply summarization given a predefined text length.

I am wondering if there is a similar tool for video summarization. Maybe a library, a deep model or ML-based tool that given a video file and a length, it ranks frames, or video scenes/shots. I'd like to generate a short summary of a video with visual features.

Tina J

Posted 2019-08-13T19:20:23.760

Reputation: 889

as in video -> text or frame ranking? – mshlis – 2019-08-13T22:07:25.240

Frames, or Shots of the video. – Tina J – 2019-08-13T22:17:28.370

Answers

0

Video summarization can be considered as the process of distilling a raw video into a more compact form without losing much information. In a general video summarization system, image features of video frames are extracted, and then the most representative frames are selected through analyzing the visual variations among visual features. This is done either by taking a holistic view of the entire video or by identifying the local differentiation among the adjacent frames. Most of those attempts rely on global features such as colour, texture, motion information, etc. Clustering techniques are also used for summarization. Video summarization can be categorized into two forms:

  • Static video summarization (keyframing) and
  • Dynamic video summarization (video skimming)

Static video summaries are composed of a set of keyframes extracted from the original video, while dynamic video summaries are composed of a set of shots and are produced taking into account the similarity or domain-specific relationships among all video shots.

Following is an attention-based video summarization model - PyTorch implementation of the ACCV 2018-AIU2018 paper Video Summarization with Attention

There is a video summarisation focused models using reinforcement learning - Unsupervised video summarization with deep reinforcement learning (Theano)

There is an LSTM - GAN based approach to video summarization - video summarization lstm-gan pytorch implementation

Microsoft Bing Search has come up with a video summarization technique using thumbnails. Intelligent Search: Video summarization using machine learning

Gokul Alex

Posted 2019-08-13T19:20:23.760

Reputation: 111

Most of the text of your answer was taken exactly from: https://heartbeat.fritz.ai/deep-learning-based-video-summarization-a-detailed-exploration-8b1a25946404 Maybe you should consider citing the source..

– Enrique – 2020-06-17T07:47:17.380