The archiving API allows developers to record video streams in an OpenTok session. These archives can be played back, or you can download the individual streams.
It turns out that downloading the individual streams does not make sharing very easy. If you recorded a conversation between two people for example you will end up with two FLV files. It would be much nicer if the two video files could be combined into one, so the resulting single video can easily be shared and played back.
This is where the stitching API comes in to play. Stitching allows you to combine an archive with up to four individual streams into a single MP4 file that can be played back in the HTML5 video player of your choice. Stitching individual videos is hard, but the stitching engine we built will time align all the individual streams, and mix the audio properly.
How we built the stitching engine
The algorithm to stitch a set of streams is straightforward. First we download the streams locally. Next we align all the streams on our timeline. For each stream we know exactly at what time it starts playback and at what time the playback ends.
For each stream that starts later than time 0 we prepend silence and a black screen. Similarly we append silence and black screen until the end of the archive for any stream that stopped publishing before the archive was closed. Finally we combine the streams side by side. The image below gives you an idea of how all of this works.
This algorithm can be implemented in a script that calls FFMPEG. Unfortunately the default FFMPEG version does not make the execution of this algorithm straightforward. Very quickly you end up doing multiple runs over the video sources, dealing individually with audio and video – a very computationally expensive process
To learn about how we solved the FFMPEG challenges, built the stitching API, and how you can stitch your archives, check out our next blog post…