Feat/ Add Video::getDuration

[ej-sanmartin]

Fixes #2050

---

tldr; Video::getDuration implemented, do we want to truncate decimal points and do we want FFmpeg accuracy or "more" accurate?

Description

Was a huge pain but opted for a linear scan of the entire .ogv file, while following roughly what FFmpeg does in its implementation (namely, the linear scan, skip invalid granulepos, track highest, restore position). Difference being that FFmpeg has start_time handling and seeks for the valid video location in the .ogv file format so faster in much larger file sizes, among other internal orchestration logic.

Opted for much simpler implementation and the results were accurate and still pretty fast. A 3 minute video didn't even cause any noticeable slowdowns, and duration is cached per video instance.

---

Discussion

I tested the changes in a local Love2D project and used the :01 second video (sample.ogv) provided by this repo, and also used two .ogv videos from https://filesamples.com/formats/ogv#google_vignette - the 3:03 video (sample_1280x720_surfing_with_audio) and the :13 video (sample_640x360.ogv).

The changes in the PR outputted the following results:

sample.ogv: 0.83333333333333 seconds
sample_640x360.ogv: 13.346666666667 seconds
sample_1280x720_surfing_with_audio.ogv: 182.89104166667 seconds

I thought the slight difference may have been a bug, and comparing it with the ffprobe CLI tool's output from FFmpeg, I'd get the following:

% ffprobe -v error -show_entries format=duration -of csv=p=0 sample.ogv
1.022653

% ffprobe -v error -show_entries format=duration -of csv=p=0 sample_640x360.ogv
13.346667

% ffprobe -v error -show_entries format=duration -of csv=p=0 sample_1280x720_surfing_with_audio.ogv
183.125333

Note: sample_640x360.ogv has the exact same duration calculated between the new Video::getDuration API and FFmpeg's API.

The reason for this is because FFmpeg's API's seem to get full container duration, which is the max between the video duration and the audio's duration if it exists.

Running the following confirms this behavior:

% ffprobe -v error -show_entries stream=duration -of csv=p=0 sample.ogv
0.833333
1.022653

% ffprobe -v error -show_entries stream=duration -of csv=p=0 sample_640x360.ogv
13.346667
# NO SECOND NUMBER, NO AUDIO

% ffprobe -v error -show_entries stream=duration -of csv=p=0 sample_1280x720_surfing_with_audio.ogv
182.891042
183.125333

---

Decisions

1. Do we want to truncate the duration (1.022653 -> 1.02) or have devs manually handle this?
2. Do we want to calculate how FFmpeg calculates the duration of the .ogv container or keep what we have and get the actual video duration. I'm leaning towards how FFmpeg handles this. For example, sample.ogv is 1.022653 seconds long however if we went with this current implementation, a dev would get 0.833333 seconds when calling this API which may be unexpected and cause some sync issues in their games.

Lmk how you feel and I'll update accordingly.

[MikuAuahDark]

I wonder if there's a benchmark on how long video:getDuration() take, or if there's additional overhead with love.graphics.newVideo without and with this patch?

[ej-sanmartin]

I can check with the 3 videos I had how long they took.

or if there's additional overhead with love.graphics.newVideo without and with this patch?

What do you mean?

Duration is calculated during init for a new instance of video, so I imagine there's at least some overhead. Though with these 3 videos calculations seemed instantaneous.

I used the following to load and get the duration of these videos btw for this small "game":

local videoStream = love.video.newVideoStream(filename)
local duration = videoStream:getDuration()

So don't know for sure about love.graphics.newVideo performance.

---

Edit: I'll check love.graphics.newVideo performance also with and without these changes, so give me a day or so to get back on that.

[ej-sanmartin]

with Video::getDuration()

--- sample.ogv ---
  VideoStream:
    Load time:     2.914 ms
    Duration call: 0.001 ms
  Video (love.graphics.newVideo):
    Load time:     11.925 ms
    Duration call: 0.001 ms

--- sample_640x360.ogv ---
  VideoStream:
    Load time:     5.908 ms
    Duration call: 0.001 ms
  Video (love.graphics.newVideo):
    Load time:     17.813 ms
    Duration call: 0.001 ms

--- sample_1280x720_surfing_with_audio.ogv ---
  VideoStream:
    Load time:     75.472 ms
    Duration call: 0.001 ms
  Video (love.graphics.newVideo):
    Load time:     222.381 ms
    Duration call: 0.001 ms

without Video::getDuration() (love.graphics.newVideo Only)

--- sample.ogv ---
  Load time: 11.297 ms

--- sample_640x360.ogv ---
  Load time: 15.859 ms

--- sample_1280x720_surfing_with_audio.ogv ---
  Load time: 157.557 ms

Note: actual getDuration() call takes .001 ms since actual calculation occurs on init.

The overhead seems relatively tiny, though growing linearly depending on the size of the video.

Using .222 ms to load a 3 minute .ogv video, extrapolating linearly, it would take approximately 1 second to load a 15 minute video.

Without the duration calculation, similarly extrapolating linearly, it would take about .78 ms to load a 15 minute video.

[MikuAuahDark]

Alright, code lgtm.

[slime73]

If the extra time for longer videos is unavoidable, is it possible to only pay that cost when using getDuration, so people who don't use it don't have the longer load times?

I'm also thinking about situations where a hard drive is slower, which would multiply the extra costs if they're related to disk access.

[ej-sanmartin]

It's possible to move when this is calculated, it's been a bit but I think I can update when this is calculated to the first time getDuration is called, though there would be some delay at that first call. Should we have the method be explicitly async then?

There are optimizations FFmpeg does that i haven't implemented yet, since it's more complicated given the .ogv format. That can be explored down the line if the delay for longer vids is too much, but ~1 second delay on a 15 min seems like it be more than okay

[slime73]

~1 second delay on a 15 min seems like it be more than okay

It'd depend on the system, but videos are supposed to stream their data in general so I wouldn't expect newVideo to take significantly longer for longer videos compared to shorter ones, at least not by default.

[ej-sanmartin]

The delay is because video duration has to be calculated, it's not part of the metadata explicitly for this file format.

[slime73]

I understand that, I'm just hoping there's a way to avoid the load-time regression for people who are not interested in the video duration.