Video Framing when no speaker is detected

[austinwmille]

Issue #1 brings two separate, and rather large issues into the fray. They are tough to spend time on right now, but here's a branch issue that's more approachable and might help get a step or two towards the video framing problem from Issue #1.

I've been running the script on Dota2 tournament clips recently. Very roughly, about 25% of the frames are panel discussion. The script works incredibly well on these segments. But when the video cuts to Dota content, there are no faces for it to find and the crop choice it makes is ALWAYS and without fail a section of the screen which has absolutely no game action happening. It's almost like it sees the movement of players on the screen and chooses to crop the 'jungle' spaces.

The script seems to notice that there are no faces here (I think), so I am working on a way to have it crop to faces when it detects them, but if there are segments inbetween which DON'T have faces then it should crop maybe just the center. Possibly it could find other movement and crop to that instead.

Anyway, this is something that should be fixable with some hard work and AI help, as opposed to issue #1 which I don't see a good path to begin tackling at the moment.

Here are some examples of this issue:

• https://youtube.com/shorts/s0gk5sK8ZyA shows how well the crop works during a panel shot, but there are segments of the game screen when it just doesn't know what part of the frame to crop on

• https://www.youtube.com/shorts/ZkFo-B1S3BQ clip shows the script cropping the screen during gameplay to show the most unattractive part of the screen.. just straight empty ground

• there's more examples that I might add later