OpenTok on WebRTC for iOS raises the bar

A new standard is making its way into web browsers and other clients around the world over the next few months that will likely change the way that we communicate with each other. WebRTC (Real-Time Communication) is a set of protocols and technologies that have been proposed to allow modern web browsers (currently Chrome 23 has support) to embed live audio/video communications without a plugin like flash.

Over the last few months we’ve been hard at work on a new variant to our iOS Video SDK, which we’re dubbing the OpenTok WebRTC for iOS SDK.

In the world of video WebRTC is a really big deal. The quality increase we’ve seen in WebRTC video versus our current Flash SDK is pretty phenomenal. For instance, video latency is typically less than 250ms under most network conditions. This is important to maintain a flowing conversation and avoid talking over other people on the call. Video quality is also noticeably better. The framerate and resolution are higher and adjusted dynamically over time to take advantage of the bandwidth and device capabilities that’s available between the clients.

We’ve seen some pretty exciting results under typical network conditions (broadband WiFi or 4G). The latency (or lack thereof) is instantly noticeable. The days of talking over people due to delays in video transmission are going away.

In addition to these improved stats we’re seeing with our WebRTC implementation, we’re getting a couple additional heavily requested featured implemented in our new iOS SDK. This new SDK introduces support for Peer-to-Peer calls and orientation support with mobile clients. This new Framework will allow you to make video calls between iOS devices and with our new (v2.0) JavaScript SDK in the browser; and it’s fully WebRTC compliant!

When we started looking into implementing peer-to-peer video on our iOS SDK, we were faced with some challenges. One of the big features that sets our platform aside from other video products today is the ability to interoperate between web browsers and mobile clients. We wanted to keep this functionality, but having it work with our current JS/Flash offering wasn’t a viable solution. We started working on a WebRTC implementation of our API back in July for Chrome and this began to open some doors for us.

Google has been one of the driving forces behind the WebRTC standard. They’ve made several acquisitions in order to leverage some key technologies in the standard. In doing this, Google has also open sourced the WebRTC source code. We decided this may be the best place to start when adding peer-to-peer video to iOS. Not only was it already implemented in Chrome – it was also being kept up to date for us by Google and the open source community.

That being said, this wasn’t exactly easy. We faced many challenges when we decided to get WebRTC running, let alone compiled, on iOS devices. The challenge was getting that very large codebase, which was not intended to run on mobile devices (at least yet) to compile for the iOS platform. This required modifying the WebRTC codebase fairly extensively to enable specific code paths that didn’t expect the iOS platform to be executing. Parts of the codebase, including an iOS implementation of the capture pipeline (getting frames from the camera down to the VP8 encoder) had to be built from scratch. Once we got it running, we faced some problems keeping it compatible with the version in Chrome (it’s a pretty rapidly changing codebase). For instance, the implementation of PeerConnection has changed (in a breaking fashion) at least a handful of times since we began this journey back in July.

Once we got it stable and started doing testing, we also realized that this SDK was not going to be able to run on older devices. The key reason is because the current iteration of WebRTC uses a different video codec than we use in our original iOS SDK (VP8 vs H.264). What this meant for us is that we couldn’t rely on hardware accelerated video encoding for this new SDK out of the box. All video encoding and decoding is now taking place in software running on the CPU. This unfortunately is out of our control at the browser implementation. However, with our iOS SDK, we have the ability to dig into the codebase and modify it to our liking.

Adding H.264 support between iOS devices is an effort we have already begun. Once completed, we’ll be able to rely on hardware H.264 video encoding between iOS clients which means the CPU consumption on those devices will go down significantly. At this point in time, we’re limited to newer, dual-core devices for our WebRTC stack (such as the iPhone 4S, 5, iPad 2, the new iPad and 5th Gen iPod Touch). This is something we’re very aware of as a pain-point for developers that want to add live video to their iOS apps. We hope that support for older devices (iPhone 3GS and iPhone 4) will come in the future, though.

We’re excited to release this new SDK (currently a new project on our GitHub page). It’s yet another option we’re offering to enable developers around the world to add live video to their iOS applications. This new Framework raises the bar for acceptable video quality and we’re looking forward to seeing even more use-cases take advantage of what it has to offer. Dive right into our docs to get started. You can be up and running your own version of FaceTime in less 30 minutes.

Related Posts:

OpenTok on WebRTC: Offering the technology of tomorrow, today

  • Anonymous

    When is Android support coming?

    • http://songz.me/ Song Zheng

      We don’t have an exact timeline but we are working on this.

  • Pingback: OpenTok WebRTC for iOS raises the bar | Tokbox Blog | Cloud communications in Japan and elsewhere | Scoop.it

  • Pingback: How Will WebRTC Manifest Itself on Mobile Devices? : BlogGeek.me

  • Pingback: Jive Acquires Meetings.io: Has the Era of WebRTC Begun?

  • http://twitter.com/n1ght_coder Yury Kovalev

    Thank you very much, guys.

  • Mark Pemburn

    Has this been realized yet? I have tried your WebRTC demo on my iPad using the Chrome browser and it reports that WebRTC is not supported on mobile devices. If you have a working demo of _any_ kind for iOS, you need to shout it from the rooftops ‘cuz it could be _the_ deciding factor for someone looking for an RTC solution.

    • http://aoberoi.me Ankur Oberoi

      Hey Mark,

      This isn’t available in any web browser on iOS just yet. Our SDK offering lets you build WebRTC into native Apps. While this isn’t ideal, it does interoperate with WebRTC in the browser on Desktop, so it definitely embraces the Web in WebRTC.

      • Mark Pemburn

        Thanks Ankur,

        I was hoping to find a way to use Tokbox on iOS and other mobile platforms for a project that I’m involved with. It would be great if there was already an app available—especially one for iPad. I’ve asked on your forum but haven’t received a definitive answer.

        • http://aoberoi.me Ankur Oberoi

          Mark, there are a few Apps in the wild that use OpenTok, but the ones I’m aware of aren’t in the AppStore – they were made for enterprise or ad-hoc distribution.

          Besides that, we have our HelloWorld and BasicTutorial available on GitHub: https://github.com/opentok/OpenTok-iOS-Basic-Tutorial, https://github.com/opentok/OpenTok-iOS-Hello-World

          • Mark Pemburn

            I’ve played around to with the sample apps. Nice, but it would require a great deal of work to create a full-featured client of the type that WebEx or GoToMeeting offer. Wish I could do that but time doesn’t permit. I’ll keep OpenTok in mind for future applications.

          • http://aoberoi.me Ankur Oberoi

            I guess a great deal of work is relative. Let us know if you find a better fit or have specific features you would have like to seen for your use case. Thanks!