Video synchronization (split from another thread)

As stated in the video color-correction thread, I am running into difficulty synchronizing multiple video streams.

I was told to start a new thread, so here it is. To quote the previous thread:

I’m trying to synchronize videos on adjacent faces with each other. I am not sure how best to do this.

Playing them all at the same time - the time the node is loaded - gets me close, but not quite there. I am hoping there is some way to keep them in sync.

Would it make any difference to modify the bitrates of the video files? The pixel dimensions?

I see that on one node - the only node that is totally functional - the multiple videos sync perfectly, and they all have the same dimensions and same bitrate.

The video clips on that node were also initialized in the code, in clockwise order, all timed to play when the node is first loaded.

On some of the other nodes that don’t work, the videos all are timed to start when the node is loaded, but initialized in a random order. I don’t know if that makes any difference.

Any insights you can offer would be appreciated.

The original video color correction thread.

OK, let’s see… Are you using the ‘autoplay’ parameter? If you set this to ‘true’, then videos will be instantly played whenever you enter a node, and they should be effectively played at once.

If you can’t rely on autoplay because, for example, you must decide whether to play the videos or not, then you can use a custom function and immediately play the videos after doing the switch. Something like this:

function gotoMyNode() switch(myNode) myVideo1:play() myVideo2:play() myVideo3:play() end

If this doesn’t work, then I’m afraid that we’ll have to fine-tune the engine even more… I haven’t run into these sort of problems yet.

I am using the ‘auto’ parameter with the videos, which I thought meant autoplay.
That - the auto parameter -doesn’t result in accurately synced videos.

But maybe that isn’t what it means. I’ll try setting autoplay.

Update: I added the autoplay parameter and a lot of these nodes look noticeably better.

I still have a few ambient video elements which are ‘glitchy’ - they loop off-sync with everything else, and they have what appear to be slight intermittent ‘jumps’, but I am starting to think that is an entirely different problem.

My guess is that those particular video files are somehow missing 1 or 2 frames, which would probably explain why they have slight glitchy ‘jumps’ and why they steadily drift off-sync with everything else around them.

I can check this in my video editing software, if that is what is going on it should be a fairly easy thing to fix.

I’ll let you know how this goes.

Update #2: I figured out that there weren’t missing frames, there was a duplicate frame on these clips, and I also figured out how that happened. I’m going through the affected video clips and fixing them all, it seems to be working perfectly so far on all of them.

Great to hear this then!! Admittedly, playback of spots is a bit finicky still, but dare I say it’s sufficiently efficient to maintain all videos in syncs. Hope it works well!

Sent from my iPad using Tapatalk

UPDATE: I’m looking at this project, and the reality is that I’m still having some subtle synchronization issues. I am running out of things to try by this point… bottom line is that the engine is, as you said Agustin, finicky, and I suppose I’m asking a lot of it, with up to 5 video spots playing at once.

I don’t think DAGON can keep up with that smoothly. Maybe in a later release some of these issues will be sorted out a bit, but I think now would be a good time for me to work on something else for a little while and come back when DAGON is closer to version 1.0.

Matt, we’re sharing some important news about Dagon this weekend. You might want to hang on in there and see what we have in store :slight_smile:

Sent from my iPhone using Tapatalk

Intriguing. I’m looking forward to whatever you’re planning. :o :smiley:

Agustin doesn’t plan, he plots ;D

It occurs to me that where I still wind up with a subtle glitch, a lag or time offset, is always at the end of an animation loop, where the 5-6 video streams all have to start over at the same time. This is a problem I’ve now successfully resolved save for a short subset of problematic nodes with 5 or 6 sizable video hotspots synchronized.

When, however, only 3-4 video clips have to start over at the end of a loop, there is no glitch, and no loss of synchronization. This gives me an idea:

It occurs to me that if the loops were staggered, with some starting/ending at a different time than the rest, this might fix the issue. I imagine 3 out of 6 could start at the moment the node is opened, in a 4-second loop. There’d also be - for the other three areas - three two-second clips that play only once, no loop, followed by another three in a four-second loop offset by two seconds. So in effect the only time there’d be 6 clips starting would be the time the node is initially loaded, every time there’s a loop point after that it’d only affect 3 video clips.

I can understand how to do this conceptually but am finding it difficult to actually write working code for it, using the available DAGON command reference and documentation.

I wrote what I thought would work, then spent a couple of hours debugging it trying to make this crazy idea actually run. It’s still not working. Can anyone give me guidance on how to code this?

[EDIT: After some 30-odd attempts to rewrite the code I’m now finally making progress, I hope. It’s partially working at this point but a few adjustments still need to be made. Still not quite right though… and I’m running out of ideas at this point.]

[EDIT #2 - I’m stumped. I will probably just wait until Dagonity is released and try that next, if it really is as user-friendly as Agustin says it is.]

[EDIT #3 - I just tried something on a hunch - which was taking one of the more problematic nodes and recording the screen with a 60 fps camera, then playing it back at 10 fps in the hopes of getting a better look at the glitchy out-of-sync behavior.
I realized two things:
-there was actually an extra lag, at the time when the videos looped, not much but 1-4 extra lagging frames, out of 60, every time the video hit a loop point.
-the extent of the problem seemed to correlate strongly with video surface area. The largest video hotspot size, a 1024x1024 pixel video clip, was clearly causing more lag than any of the others.
A hypothesis: If every video clip has the exact same, smallish surface area, the video clips will all have the exact same miniscule amount of lag and thus will not only not go out of sync gradually over time, but will not have discernible momentary visual glitches.
I am now testing this hypothesis.]

I believe the problem lies in the poor timing code I did for Dagon. Video synchronization is a very complex issue… especially when you must also sync audio. I’m sorry to say that I never spend much time into this, and the biggest improvement was using the functions provided by SDL 2.0. Still, there’s a lot of room for improvement.

My take is that, due to the high CPU power demanded by 5-6 video streams (remember, Theora decoding is slow!), each individual video starts lagging regardless of their shared attributes. Right now there’s no frame skipping, for example – if a video lags, then the engine should toss away frames it can’t draw. I tried to implement a half-assed solution that tends to work on cutscenes, not so much smaller videos that demand better precision.

So I’m afraid we’re stumped here. By all means, feel free to have a look at Video.cpp, which is where the decoding algorithm is. I can’t commit to look at this, not with so much pending work. I’m keeping my fingers crossed that Dagonity will fix this issue, though. If you want to, you can send me your project just to make sure we’re talking about the same problem. But again, video decoding has always been flaky since the SCream era :frowning: