• Make sure that you are using the OpenTok on WebRTC JS library. You can find the library here, and find the reference documentation here. If you are using the v1.1 JS library, you will need to update your application to the v2.0 library.

• Use a browser with WebRTC support – Chrome 26, FireFox Beta, or IE9 with the Google Chrome Frame plugin.

• When you generate a session, make sure that the p2p.preference flag is set to disabled. If you’re generating your sessions from the Developer Dashboard, then you will need to download one of our server-side SDKs and generate sessions yourself.

• If you haven’t already asked to participate in the Mantis beta, please contact us at mantis@tokbox.com. Then make sure that you are using the correct API key for the Mantis beta. If you are not sure which API key you sent us, then please email us, and we will let you know. Mantis requires that your API key be enabled to access the infrastructure.

It really is that quick, and if you’re finding that you need some more help, then let us know. To make sure that your question gets answered as quickly as possible, please send an email to support@tokbox.com using the following template:

Email subject: [Mantis bug report] [Description of issue]
Email body:

API key:
Session ID:
Browser version:
Time of issue:

and then feel free to describe the issue further after filling out those fields.

We’re excited to see what our partners can build with Mantis, and we’re also ready and available whenever you need help. Happy coding!

This is another big step forward for the TokBox team as we continue to pursue our goal of providing application developers with simple yet powerful APIs. APIs that not only leverage the latest standards to deliver the best possible experience, but that are backed by a scalable, smart cloud which supports interoperability across a variety of end-points.

It was just over six months ago that we launched the OpenTok on WebRTC platform. Since that time we’ve been hard at work, constantly pushing the boundaries of OpenTok on WebRTC’s functionality and performance. We launched the first iOS SDK for WebRTC, introduced cross-platform and device support, improved connectivity with cross-platform TURN support and more.

Mantis for OpenTok on WebRTC acts as a central switching station for all the WebRTC streams in the OpenTok cloud.  Mantis enables:

• Reduced upload bandwidth consumption, with the ability to scale out a single WebRTC stream to many endpoints
• High-quality multi-party video calls
• Cross-browser compatibility for Chrome, Firefox and Internet Explorer (through Google Chrome Frame)
• Cross-device compatibility for iOS native apps and Chrome on Android
• SIP interop: the ability to dial in to a WebRTC video session from any cell or landline phone

Why should you care about Mantis?

1) It’s scalable…

WebRTC inherently uses a peer-to-peer based communication model.

In reality, while this is useful, several real-world applications naturally lend themselves to a multi-party use-case:  online classrooms, collaboration tools, the list goes on. The only way to enable a group video call using off-the-shelf WebRTC would be to create a mesh of PeerConnections between all participants in a call.

This mechanism doesn’t scale the number of connections well (it scales quadratically with the number of participants), and it is bandwidth inefficient. Every publisher needs to pipe out one video stream per call participant, leading to a linear increase in the amount of upload bandwidth required.

Mantis is a cloud-based media routing server, which enables us to get around this mesh problem by providing a centralized relay authority.

Every browser/device end-point thinks it’s connecting to a peer, but instead connects to one of our Mantis servers. Mantis can mediate multiple connections and then route streams from each publisher to any relevant end-point subscribing to that stream. Now OpenTok on WebRTC can scale subscribers effectively without constraints on publisher upload bandwidth, all while still leveraging standards-compliant WebRTC implementations under the covers.

2) It’s smart…

Scale is just one piece of the puzzle.

Mantis also provides a call-control/signaling mechanism that works in lockstep with media routing. This means OpenTok on WebRTC uses some pretty sophisticated algorithms to dynamically adapt to various network conditions that the client endpoints may experience.

Let’s say you were on your iPhone in a coffee shop with Wi-Fi, and then you step out into the street and switch to a much slower cellular connection. Mantis is aware of the change in network conditions and can adapt the bitrate of the encoded stream in order to adapt.

3) It interoperates…

Supporting different devices endpoints is one of the core tenets we strive towards at TokBox.  Mantis helps us work towards interoperability between WebRTC and other communications ecosystems.

One such use-case we are demonstrating with Mantis is interoperability with telephony endpoints. Leveraging Telefónica Digital’s telecom infrastructure, OpenTok on WebRTC can now connect PSTN or VOIP endpoints with WebRTC-enabled group video calls, using SIP to interoperate. This enables a user who is not video-enabled to dial-in to a multi-party OpenTok video session, and have the audio streams appropriately mixed and piped in each direction.

We believe Mantis lays the foundation for not only a powerful routing technology, but also for other functionality that can be layered on top of it: stream mixing, recording of media, signaling/event-notification APIs, and more.

We’re excited to show off our initial implementation of Mantis at the The Next Web ’13 in Amsterdam.  If you are going to be around, swing by our demo booth to see bandwidth-efficient multi-party WebRTC video in action, as well as a demo of dialing in to a WebRTC video call from your everyday mobile phone.

Mantis is now in external beta.  If you are interested in taking it for a spin, please contact us at mantis@tokbox.com.

Mantis: Cloud-scaling infrastructure for OpenTok on WebRTC from TokBox on Vimeo.

Highlights

• A lot of audio bugs in WebRTC were fixed dealing with crashes and non-standard audio bitrates
• Chrome on Android can now be WebRTC-enabled by enabling a flag
• Improvements to the connectivity stack in WebRTC
• Ability to set media constraints for audio

Full list

• Avoids crash in WebRTC audio clients for unsupported capture sample rates.
• Avoids crash in WebRTC audio clients for 96kHz render rate on Mac OSX.
• Enable webrtc build on android.
• Set WebMediaPlayerMS network state to loading instead of loaded
  • This indirectly fixes the problem where WebRTC audio is muted upon refresh. The HTMLMediaElement will try to cache fully Loaded videos when the element is destructed. This will signal to the HTMLMediaElement that the player was destroyed when loading, so it needs to recreate WebMediaPlayerMS upon destruction of the media tag.
• Allowing multiple MediaPlayers to connect to WebRtcAudioDeviceImpl by sharing one WebRtcAudioRenderer.
  •  The audio is gone when new PeerConnection is connecting to a media stream. What is happening is that the stream will pause the existing MediaPlayer and create new MediaPlayers to associated to it. But since we only allow one WebRtcAudioRenderer to connect to WebRtcAudioDeviceImpl, the new MediaPlayers audio won’t be able to associate to stream.

    This patch fixes the problem by allowing multiple MediaPlayers to connect to WebRtcAudioDeviceImpl by sharing one WebRtcAudioRenderer

•  Add webrtc audio-mirroring flag for automated testing
• Set MediaConstraints for audio to WebRTC
• When both stun and turn server are provided, use the turn server as the stun server.
•  This patch fixes two audio issues for webrtc hangout.
  • #1 Audio is muted when switching between the local stream to remote stream.
  • #2 It has glitches when switching between views.
    The first issue is fixed by storing the volume for local stream before muting, and recover the volume when it goes away. The second issue is fixed by re-using the |audio_renderer_| between different webmediaplayer.

We’ve also been working hard on making OpenTok on WebRTC work with both Firefox and Chrome so you too can enjoy all this cross-browser goodness!

Off to the races

The first thing that you need is version 21 or higher of Firefox, currently available through the Aurora FTP site and Nightly site.

If you’re a Firefox user already you may want to set up a separate profile to test with. This will keep your default profile clean, and your test profile will likely also start up a bit faster than your default one. You can find instructions on how to do this here and here.

If you’re testing with Chrome, you will want to make sure that you have updated to the latest version (version 25 at the time of this article), which has all of the fixes required for interoperability with Firefox. All of the relevant WebRTC bits are enabled by default in Firefox 21+ and in Chrome 25, so you should be ready to roll. Now go have a play with TokBox’s OpenTok on WebRTC demo. Then come back here afterwards.

When can I have this in the production release of Firefox?

Parts of WebRTC are working in the current production release version of Firefox (Firefox 19). However, Firefox 21 is the version that interoperates successfully with Chrome, and it also has all of the WebRTC preferences turned on by default. The gang at Mozilla is also pushing hard to fix lots of critical WebRTC bugs before production release of Firefox 21 and hopefully turning on WebRTC for everyone. According to the Mozilla release calendar, Firefox 21 is targeted to be released the week of May 13th.

If this isn’t in the production release of Firefox, why is this news?

What we’re doing today is giving you a head start.

The great thing about OpenTok on WebRTC is that we make face-to-face video work in your users’ browsers when those browsers are ready to work correctly.  So our announcement today lets you know that we are ready for Firefox WebRTC support to go production.  Right now, you can install Firefox Aurora and use it to test your app using OpenTok on WebRTC.  And you can watch as Firefox 21 makes it from Aurora to Beta and then to release and know when a big new chunk of your users can get access to WebRTC through OpenTok on WebRTC, all without your having to write an extra line of code.

What doesn’t work yet?

Mozilla isn’t quite done yet, so there are still some holes. The biggest one involves anything to do with disabling audio and video tracks. Namely:

• Muting audio
• subscribeToAudio and subscribeToVideo on Subscribers
• publishAudio and publishVideo on Publishers

You may also see some connectivity issues in particularly restrictive networks. We’re working on these issues now, and we’ll update you as we have good news to share in the not too distant future.

What were some of the challenges of getting Firefox and Chrome to play nicely?

Some of things we had to do were pretty straight-forward – normalizing the cosmetic differences between the APIs, for example. This just involved making sure that we were using the appropriate vendor prefixed object and function names, say webkitGetUserMedia in Chrome and mozGetUserMedia in Mozilla.

The more complex bits involved working around browser specific issues and incompatibilities in the messaging layer (this is the part that uses text blobs of SDP). One notable example was that earlier implementations in Chrome and Firefox didn’t support compatible implementations of cryptography, with Firefox supporting DTLS and Chrome SDES. This was eventually resolved, but a new related issue popped up in Chrome as a result.

Simply, DTLS adds an extra “a=fingerprint” line to the SDP payload which makes Chrome 25 incorrectly assume that the payload is SDES when it sees it. If SDES was being used there should also be an “a=crypto” line, but in this case there won’t be. So Chrome complains and won’t accept the SDP message.

As Chrome doesn’t actually care about the value of the crypto line, only that it’s present and well formed, the solution is to add a dummy crypto line when sending offers from Firefox to Chrome. Google is landing a fix for this in Chrome 26, but leaving the extra line in causes no harm and allows compatibility with Chrome 25. The dummy line we add in OpenTok looks like this:

a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:FakeFakeFakeFakeFakeFakeFakeFakeFakeFake\\r\\n

On the Mozilla side, Firefox doesn’t support FQDNs when specifying ICE servers for the PeerConnection. There’s a Bug open for this on Bugzilla, but in the meantime, ICE servers must to be specified as raw IPs for Firefox.

Another issue is caused by Firefox sending Data Channel information as part of its JSEP Offers – something that Firefox is doing temporarily until its Data Channels implementation is completed. Unfortunately this upsets Chrome and requires another workaround.

The good news is that we are doing all this hard work for you – making OpenTok on WebRTC interoperate smoothly – so you can spend your time building great applications, not worrying about browser compatibility between different implementations of the WebRTC standard.

Wrapping it up

At TokBox we’re super excited to see another great browser jump on board with the WebRTC standard, and we’re really impressed with how quickly the Firefox and Chrome teams have come together to make this happen. We really believe in the power of WebRTC – in particular, in the quality of the video and audio it delivers while maintaining low latency. We’re constantly keeping an eye out for all of the new developments in WebRTC, making sure we’re staying ahead of the curve to keep our API supporting the latest and greatest. Now we’re off to take a hard look at Microsoft’s first implementation of CU-RTC-Web.

In today’s struggling economy, the founders of LiveNinja saw a new work trend emerging. Friends, family and acquaintances were setting up consultancy shops online, using siloed video chat services like Skype to connect face-to-face with clients. While this worked for some consultants, they also saw many people struggling to bring in to new clients. Just google “Spanish lessons” and imagine how hard it would be to stand out amongst the 59 million + results. ¡Ay caramba!

On why his team decided to develop LiveNinja, CEO and Co-founder Will Weinraub said:

The number of skillful individuals across the world that are unemployed and unable to produce income is exorbitant. However, we feel if we can modernize the traditional concepts of employment by providing the proper tools, we can eliminate the barriers to both job and income creation. LiveNinja effectively allows people to create and accelerate these jobs by empowering any individual with all the tools they need for an online service-based business.

In response, LiveNinja created a full service marketplace that enables experts to set up their own web page and customize the look and feel. Certified Ninjas can set their own rates and schedules, and collect income in the LiveNinja one stop shop.

What is the experience like for a student? Let’s say you’re dying to improve the height of your round-house kick (who isn’t?) Simply head over to LiveNinja and search for “martial arts”. If they can’t find exactly what you’re looking for, LiveNinja’s intelligent search will make recommendations for other areas you may be interested in. Once you find the expert you’re looking for, simply book an appointment and let them welcome you to their virtual dojo.

Have a skill that you’d like to share with the world? Become a Certified Ninja today. Prefer to learn something new? Start browsing expert profiles, reading user reviews, and booking sessions. Welcome to the dojo.

Walking away from the panel, I found myself quite convinced of a few things that beforehand I had only held as loose opinions, and I wanted to share those with the TokBox blog audience.

1) Ultimately a common language set will prevail… but not for the reasons you’re thinking
Today, as we all know, it takes a developer a long time, and real effort, to be available on all platforms (namely web, iOS, and Android) from day one. The reason is simple – HTML5 based applications don’t perform as well as their native counterparts. If anyone learned this lesson the hard way, it was Facebook, but other app developers have similar horror stories.

Further, it takes first-world incomes to be able to buy the top of the line smartphones from the likes of Apple, Samsung, and Motorola. Most of these devices are extreme luxury items outside of the US and Western Europe. As a result, an already saturated app store for existing platforms is a daunting marketplace for new and emerging apps.

Given all of that, we asked ourselves… but what if the native API was HTML5?

The upcoming launch of Firefox OS is exciting exactly because developers targeting untapped markets (Latin America, South America, Africa) using skills (HTML5, JS, and CSS) that they already have is a large ocean of opportunity.

Firefox OS phones, a device our Telefonica Digital family have dedicated significant time to developing, may not compete on hardware specs with the iPhones we have in the US and Western Europe, but, as a platform for the masses, their reach could be much greater, open much larger markets, and open many more interesting opportunities in the years to come. Simultaneously, as mobile browsers continue to invest in speed and quality, a second wave of HTML5 apps wrapped in native application bindings could make a comeback.

The sum of these parts is that developers who have built HTML5-based apps will be well positioned to take advantage of the next generation of smartphone devices, but the panel agreed that today the burden lies on the developer to build multiple native applications.

2) Frameworks are only as powerful as the developer who wields them
Ted Drake spoke a lot about how accessibility has been pushed to new, and powerful, places with the emergence of smartphones and tablet devices. However, he decried frameworks as a programming paradigm where, “shit goes in, and shit comes out.”

This, I thought, was one of the better topics we debated.

It started with the thesis that frameworks are used by developers who don’t know what they’re doing as a shortcut or to quickly prototype their work. However, Jonathan Smiley spoke up for framework users by giving the example of his agency who had built a framework that they used to quickly, and robustly, build responsive applications for their clients. Zurb  knew what they were doing, and knew how to use the tool effectively.

Ultimately, I think the entire panel agreed that the issue with frameworks is speed more than complexity. If people are moving too fast, while not understanding the programming paradigm, and producing second-class applications as a result, then of course, “shit goes in, and shit comes out.” But a framework in the right hands used to solve the right problems can really help development, especially when it has to work across both mobile + web at launch.

I’m personally a huge fan of frameworks as they abstract the mess away from developers. Our OpenTok thesis is that we worry about the details so that our partners don’t have to. Instead they build really great applications that take advantage of a live-video platform without signing up for the hassle that usually comes with one.

3) Regardless of the target audience – mobile + web will both be part of the solution
One of the questions from the audience was about whether our discussions over the course of the hour applied equally to enterprise as well as consumer. At TokBox we’ve learned over the last two years of the OpenTok platform that solutions need to exist wherever the users are.

Doing a customer support app? People may want to call in from a website, a native app, or from the phone.

Heading to the doctor’s office remotely? Requiring patients to buy an iPad would make the solution impossible for most doctors, but having a solution that works as long as the patient has a web connection opens this door.

Mobile + Web is not a consumer only phenomenon, though the headlines are dominated by these types of products. The truth is that mobile + web is the new paradigm in which all products are being built.

I wanted to quickly thank Lisa Oshima who was the moderator of our panel, and just say again how much fun I had. I’d love to hear more about where you think mobile + web development is going so please comment or email me at melih@tokbox.com.

The Microsoft draft outlines a low-level API that allows developers more direct access to the underlying network and media delivery components. It exposes objects representing network sockets and gives explicit application control over the media transport[3]. In contrast, the WebRTC API abstracts these details with a text-based interface that passes encoded strings between the two participants in the call. With the WebRTC draft, developers are responsible for passing the strings between communicating browsers, but not explicitly configuring media transport for a video chat.

In terms of functionality and interoperability, these two approaches are equivalent. The text strings used by the WebRTC API are formatted according to the Session Description Protocol (SDP) [4]. SDP was initially developed for setting up SIP and VoIP phone calls, and contains all the configuration needed for two parties to initiate a call. CU-RTC-Web argues that SDP is not appropriate for use in a Web API, and that the limitation of endpoints that understand SDP is too prohibitive. As a result, the bulk of the difference between the two drafts is that CU-RTC-Web re-imagines the functionality of SDP in JavaScript.

While it is fairly straightforward to translate between the WebRTC 1.0 use of SDP and the equivalent CU-RTC-Web JavaScript interface definitions, there is an additional hurdle that would prevent interoperability. There is an ongoing discussion about required video codecs, focused on the choice between competing video codec specifications, H.264 and VP8. H.264 is used by default for YouTube and iTunes video content. Many modern mobile devices also offer native H.264 processing. However, many in the W3C Working Group have raised concern that the patents on H.264 are too restrictive to be used as a core Web technology. As an alternative, Google offers the VP8 codec, which was purchased in 2010 and relicensed as free-to-use under Creative Commons Attribution 3.0. Right now, Firefox and Chrome are using VP8 for WebRTC support. Microsoft has not yet committed its browser to a video format. Until browser vendors converge on this decision, the cost of interoperability will remain prohibitive for most web application developers.

Shortly after Microsoft announced CU-RTC-Web, the W3C Working Group voted on whether to incorporate the alternative ideas from the Microsoft proposal. The Working Group decided to continue using SDP to configure media transport[5]. Microsoft is continuing forward with CU-RTC-Web, and recently released a demo using the proposed API[6].

As far as web developers are concerned, this is both good news and bad. While it appears that face-to-face communication will be supported in all browsers, they may not work together seamlessly. For OpenTok users, there’s no need to worry. We will continue to offer a consistent API that bridges the technical gaps left behind by the standards development process. OpenTok on WebRTC will work across all browsers and will tackle interoperability issues of today, and tomorrow.

TL;DR: Upload the code from this repo into your Parse Cloud Code application. Include it into your own script (here’s an example). Now you can create Sessions, generate Tokens, and more right from your own iOS (or any Parse powered) App by calling your cloud functions right from your client code (another example).

A few words about Security

OpenTok uses an API Key/Secret mechanism to authenticate a developer’s server-side requests when performing actions like making a new Session. The Secret is also used to sign data that you want to store in a Token. What you don’t want to do is include that Secret in your client side App, whether it be Javascript for a web App or an iOS App. This is important because there are real costs that come from a stream being published in your sessions with your API Key, you wouldn’t want to be picking up a malicious person’s minutes tab.

Parse has built-in authentication and authorization mechanisms that really come in handy here. As a developer, you get to decide how to handle login (maybe you want to use Twitter or Facebook or just plain email/password) to authenticate your users. Then you can use ACLs (Access Control Lists) to define what users are allowed to do (read public data, write private data, identify as part of a group, etc); this is authorization. Or maybe your application isn’t as intense and you are cool with simple anonymous users with all of the data being publicly readable and writable. The choice is yours, you know your needs the best, and thats the important part. We can piggyback off of these mechanisms to help you describe exactly who can make OpenTok Sessions, and to choose what data (like the Role) goes into their Tokens. Poof! You now have a solid security story behind your App.

Our Example App: Broadcasts

I’m going to take you through building an example App called Broadcasts to help illustrate how to set up an integration for yourself. The goal for Broadcasts is pretty simple, we want an App that lets any user start a Broadcast and lets any of the other users of the App watch. When a certain user creates a Broadcast, only that user can Publish video to it, and the rest can subscribe (this adds a little bit of authorization logic to our example).

There still are two parts to Broadcasts, the iOS code and the server-side code. The main difference here is that the server code is easy to write and deploy because its just a few functions using the existing Parse objects in the App, and I’ll show you how to write them.

You need to have an OpenTok Account for an API Key and Secret, and a Parse Application (call it Broadcasts) for an Application ID and Client Key. Let’s get started.

Setting Up: Let’s first set up Cloud Code for your App by installing the command line tool and creating a new directory to hold the project (more detailed instructions):

Now parse/cloud/main.js is the file we will write our functions within. But before we do that we need to pull in the OpenTok library that I’ve provided. Copy the contents of this repository into a new directory called opentok so that you now have the directory layout as seen below:

Writing our Cloud Code: Now we’re going to create a few functions inside main.js. This code is customized to how we want Broadcasts to work. I’ve decided beforehand that the App contains objects of the class Broadcast that have two important properties: sessionId and owner. Note that I’m not calling it a Session; I’ve assigned it a name that is meaningful for the logic of this App (Broadcast) and then referenced an OpenTok Session with its sessionId property. The owner property is a Parse.User object and helps us figure out who is authorized to publish to the Broadcast. Use the code below and substitute your OpenTok API Key and Secret:

The comments in the code above should help you understand what’s going on. You are not limited to follow the exact same logic, all that I’ve done is use the Parse JavaScript API to describe how I want to handle creating my Sessions (before a Broadcast object is saved) and generating my Tokens (when I call the function). Lets deploy it:

Server-side, done.

Setting iOS Up: Now we just hook the backend up to the iOS App. I’ve already provided a shell of the Broadcasts App for you. Download it into the iOS directory so your directory structure is as below:

The first thing you need to do is to add your OpenTok API Key, Parse Application ID, and Parse Client Key to the iOS/Broadcasts/Constants-sample.h file and then rename it to Constants.h. Now open Broadcasts.xcodeproj in Xcode. If you run the App now, you will see that you can create a Broadcast by tapping the ‘+’ in the top right, naming it, and hitting done. If you then select that row, you will go into that Broadcast but nothing will happen, there is a Label at the bottom that says we are Disconnected from OpenTok.

Sessions: You may have not realized, but this part is already done. When a new Broadcast gets saved, the beforeSave function we created in the Cloud Code should run and attach a sessionId property to each Broadcast object. Use the Parse Data Explorer to confirm this.

Tokens: There are two pieces we need to correctly generate a token: owners and calling the getBroadcastToken Cloud function

Owners: In the shell, the Broadcast object has no concept of an owner. We will be creating anonymous users through Parse so that every user has some sort of identity, and then assign a Broadcast’s owner when a new one is saved. Then we will use that information to help display which Broadcasts can be published to by the current user in the Table View. Lastly, we will use that owner information to decide whether we should publish or subscribe when we have opened a particular Broadcast. Follow the code examples below.

Add Line 9 above to make sure an anonymous user is created.

Add Lines 9-14 above to add an owner property to new Broadcasts before they are saved, and specify that they are publicly readable but only writable by the owner.

Edit the line matching line 8 to enable a ★ icon next to the Broadcasts in the Table that is owned by the current user.

Create the method on lines 3-5 as a helper to find out of the current Broadcast is owned by the current user. Then use that method in the if statements that wrap the code on lines 12-15 and 23-25.

Calling getBroadcastToken in the Cloud: The last part is to call our Cloud Code function, which returns asynchronously, and then we use the result to finally connect to the OpenTok session.

Add the method call on lines 11-18 which contains a block that connects to the Session.

Build and Run: If all went well, you should be able to Run and play with the completed Broadcasts App. If you have any problems you can compare with the completed version of the App.

But wait, there’s more…

Why limit ourselves to using Parse for iOS Apps? You can also use their JavaScript API to save you the trouble of writing a backend for your web Apps, too. With the security model benefits I highlighted before, that might not be such a bad idea after all.

Nearly 7 months ago, we publicly announced that the OpenTok API would extend its reach to native mobile application developers by publishing the OpenTok iOS SDK. In the time since, we have tightened the performance of the SDK runtime for iOS devices and spent a good deal of time learning about how best to deliver video to the mobile platform. While iOS commands a large portion of the mobile app market, it is intuitive that we should build similar SDKs for other popular platforms outside of the browser. It is a pleasure to announce that we are developing the OpenTok Android SDK, to allow native Android developers to bring live video chat to their apps.

The Android OS opens a wide set of devices and system API versions to with which to work. While we do not expect to target the entire Android device landscape, most mobile devices built in the past two years that run Android OS should be able to support video chat in a native app. As we learn the baseline of devices and OS versions that developers wish to target, we will work to ensure a quality experience with the OpenTok API for the broadest set of mobile devices possible.

A demo application using the OpenTok Android SDK is available on GitHub, which developers can download and run on their Android devices today. The standalone SDK is hosted separately. We’re collecting data on which devices work, so go ahead and give it a spin, then let us know how it went. Finally, tell us about your app! We are interested in what kind of application you’re working on, devices you’re interested in getting video chat to work with, and Android OS API versions those devices are able to support.

Over the last few months we’ve been hard at work on a new variant to our iOS Video SDK, which we’re dubbing the OpenTok WebRTC for iOS SDK.

In the world of video WebRTC is a really big deal. The quality increase we’ve seen in WebRTC video versus our current Flash SDK is pretty phenomenal. For instance, video latency is typically less than 250ms under most network conditions. This is important to maintain a flowing conversation and avoid talking over other people on the call. Video quality is also noticeably better. The framerate and resolution are higher and adjusted dynamically over time to take advantage of the bandwidth and device capabilities that’s available between the clients.

We’ve seen some pretty exciting results under typical network conditions (broadband WiFi or 4G). The latency (or lack thereof) is instantly noticeable. The days of talking over people due to delays in video transmission are going away.

In addition to these improved stats we’re seeing with our WebRTC implementation, we’re getting a couple additional heavily requested featured implemented in our new iOS SDK. This new SDK introduces support for Peer-to-Peer calls and orientation support with mobile clients. This new Framework will allow you to make video calls between iOS devices and with our new (v2.0) JavaScript SDK in the browser; and it’s fully WebRTC compliant!

When we started looking into implementing peer-to-peer video on our iOS SDK, we were faced with some challenges. One of the big features that sets our platform aside from other video products today is the ability to interoperate between web browsers and mobile clients. We wanted to keep this functionality, but having it work with our current JS/Flash offering wasn’t a viable solution. We started working on a WebRTC implementation of our API back in July for Chrome and this began to open some doors for us.

Google has been one of the driving forces behind the WebRTC standard. They’ve made several acquisitions in order to leverage some key technologies in the standard. In doing this, Google has also open sourced the WebRTC source code. We decided this may be the best place to start when adding peer-to-peer video to iOS. Not only was it already implemented in Chrome – it was also being kept up to date for us by Google and the open source community.

That being said, this wasn’t exactly easy. We faced many challenges when we decided to get WebRTC running, let alone compiled, on iOS devices. The challenge was getting that very large codebase, which was not intended to run on mobile devices (at least yet) to compile for the iOS platform. This required modifying the WebRTC codebase fairly extensively to enable specific code paths that didn’t expect the iOS platform to be executing. Parts of the codebase, including an iOS implementation of the capture pipeline (getting frames from the camera down to the VP8 encoder) had to be built from scratch. Once we got it running, we faced some problems keeping it compatible with the version in Chrome (it’s a pretty rapidly changing codebase). For instance, the implementation of PeerConnection has changed (in a breaking fashion) at least a handful of times since we began this journey back in July.

Once we got it stable and started doing testing, we also realized that this SDK was not going to be able to run on older devices. The key reason is because the current iteration of WebRTC uses a different video codec than we use in our original iOS SDK (VP8 vs H.264). What this meant for us is that we couldn’t rely on hardware accelerated video encoding for this new SDK out of the box. All video encoding and decoding is now taking place in software running on the CPU. This unfortunately is out of our control at the browser implementation. However, with our iOS SDK, we have the ability to dig into the codebase and modify it to our liking.

Adding H.264 support between iOS devices is an effort we have already begun. Once completed, we’ll be able to rely on hardware H.264 video encoding between iOS clients which means the CPU consumption on those devices will go down significantly. At this point in time, we’re limited to newer, dual-core devices for our WebRTC stack (such as the iPhone 4S, 5, iPad 2, the new iPad and 5th Gen iPod Touch). This is something we’re very aware of as a pain-point for developers that want to add live video to their iOS apps. We hope that support for older devices (iPhone 3GS and iPhone 4) will come in the future, though.

We’re excited to release this new SDK (currently a new project on our GitHub page). It’s yet another option we’re offering to enable developers around the world to add live video to their iOS applications. This new Framework raises the bar for acceptable video quality and we’re looking forward to seeing even more use-cases take advantage of what it has to offer. Dive right into our docs to get started. You can be up and running your own version of FaceTime in less 30 minutes.

Related Posts:

OpenTok on WebRTC: Offering the technology of tomorrow, today