--and one, and give a warm welcome to Shwetank Dixit.

[APPLAUSE] Hi everyone.

My name is Shwetank.

This talk is going to be about WebRTC, but more from a front-end perspective.

We'll see what are the latest parts of WebRTC? What are the good things? What are the things which need to be improved, as well as what are the things, which are coming up in the future? And we'll get a sense of what the overall state of WebRTC is.

First a little bit about me.

My name is Shwetank I work in developer relations in Opera, and I've been keeping a close eye on WebRTC, right from the days of when we just had one part of WebRTC called getUserMedia.

We at Opera also made a bunch of demos with getUserMedia back in the day on a site called shinydemos.com.

You can check it out.

And a little bit more information about me.

I grew up mostly in India, and when I was making this talk, I got a few memories flooding in of my first memories of using some kind of technology for communication.

And I remember these things.

we, back in the early '90s, we used to-- at least in my county, we didn't have landline phones everywhere.

So my family, for example, didn't.

And we had to go to these places to pay a lot of money to make a long distance call to a few of our relatives.

And I remember my mother saying, OK, hurry up, hurry up.

And those of some of my few cherished childhood memories, you could say.

One more memory than I had was concerning something like this.

How many people over here have a cellphone with some kind of lock screen? Like you enter a code or gesture something? I've encountered stuff like this, back to my childhood.

In case you don't know what it is, it opens up like this.

And this is because when people finally started getting some landline phones, they were still a little bit new.

And the children in the house they were like, yay, we have a landline phone.

And they started to make crank calls and dial random numbers and talk to people, without getting the concept that it actually cost money.

So at the end of the month, the parents would get the phone bill.

And they would be like whoa.

They used to install some kind of lock, so they're actual physical phone locks, to prevent the children to dial unless one the parents were in the house.

So the reason why I wanted to mention these things is because I want you to just see how far we've come.

And part of the reason why and how far we've come is because the worldwide web.

It completely changed communications forever.

It changed the way in which we communicated, the speed and frequency in which we communicated, and the cost at which we communicated as well.

But something even-- actually, in some ways almost equally revolutionary happened, at least in developing countries, which is the emergence of mobile phones.

Now I mean-- previously, there were parts of people-- there were people in parts of the world, which were very remote, and they had no way of having an online connection.

But now with mobile phones, they can have it and not even think about landline anymore.

So these things have been really revolutionary, when it comes to communication.

And the question arises, what's going to be the next thing? Maybe it's not going to be revolutionary, but at the same time it's going to be really, really awesome.

And that's WebRTC.

why? Because it's real time, peer to peer communication.

And the thing is I haven't really seen a lot of people who are front end developers really getting and talking about it.

But when you do, whenever you have someone who really takes a look at WebRTC, you see that there are extremely excited.

I've seen very few web people talk about it.

And that's why I wanted to talk about it, from the perspective of a web guy.

Because as I said, not a lot of people talk about it.

There was this survey of.

what's the biggest barrier to WebRTC adoption? And if it was, the number one answer would have been lack of browser support.

Then that would have been fine because with every emerging technology, there's some browsers that support it first and some later on.

But the number one answer over here was lack of awareness.

So how many people over here have built stuff with WebRTC? OK, very few.

That's what I'm talking about.

WebRTC is for web developers.

There has been real time communication technologies, which have existed for a long, long time.

But these have always been for native app developers and all the telecommunication guys.

But this is the first time that we, as web developers, can make something peer to peer, real time on websites.

So this is really, really awesome.

So let's take a look at the various parts of WebRTC.

the first is Mediastream.

The second is RTCPeerConnection.

And the third is RTCDataChannel.

These are the three big pillars of WebRTC.

And Mediastream is pretty much just getUserMedia.

And we'll take a look at getUserMedia because this is one of the things that as front end developers, we will be using quite a lot.

So when it comes to the webcam access and the output generally speaking, sites don't really do that much.

There's a site called Sqwiggle, which uses some kind of filters and effects to create some kind of fun output.

But generally speaking, it's just a cookie cutter rectangle, and that's it.

But you can do a lot.

Generally speaking, a lot of people have this misconception that to do any kind of manipulation on getUserMedia, you have to put it into a canvas and then do something.

But that's not true.

You can actually use CSS with getUserMedia to do a lot of cool effects.

So I just made a small demo to illustrate this.

Let's see.

So as to see over here, you have the normal camera output.

It's fine.

We can apply CSS filters over here pretty easily, without using any kind of canvas or whatever.

You can use 3D transforms to change this around.

You can use CSS masks to have a round cut, or you can even use border radius.

Blend modes, you can do like this.

And this is stuff that you had to use Photoshop for.

And now you can do it real time video using just CSS.

I haven't seen a lot of people give attention to it.

But I think it's something that we should also explore.

It's really, really nice.

So let's go down.

One often ignored part of getUserMedia is the sound, the microphone access.

So you can actually hook up the sound too, and hook it up with the Web Audio to do some really cool stuff.

People have done guitar tuners and that kind of stuff.

You can also do sound powered actions and navigations.

And of course, first I want it, as a demo to make something which just a clap based interface.

You clap once, it does something.

Clap twice, it does something else.

But I thought that would be a little bit too cliche.

Let's do something a little bit more fun.

I once again got back to nostalgia.

In the '90's, I used to watch two sitcoms.

One was Seinfeld.

The other was Friends.

How many people watch Friends over here? So maybe you'll get this.

I hope it works.

Pivot.

Pivot.

[APPLAUSE] So I kind of cheated over here.

I wasn't actually using speech processing.

I was pretty much just analyzing the volume, and if it's above a certain volume, then it uses 2D transforms to rotate it.

And once it goes over 360 degrees, it replaces Ross' image with Chandler.

So you can do a lot of stuff using getUserMedia mic access.

There's a project called Pocketsphinx.js,

which uses normal web audio API to do some rudimentary speech processing.

You should check out.

It's still early days, but it's something worth checking out.

How many people know about the Trojan room coffee pot? A few.

So for the people who don't, this is the story of the world's first web cameras, since we're talking about webcams and stuff like that.

Back in 1991, I think, the early '90s, there were a few computer programmers in the University of Cambridge who really liked their coffee.

So much so that they used regularly doing the day, they used to go up and check the coffee pot.

If it was full, they would get the coffee.

If it wasn't, they had to go back empty-handed.

Of course, this was a problem which needed to be solved with computer science.

So what they did was, they wrote a client and server program and hooked up a camera to watch the coffee pot live, 24/7 and have a 1/20 by 1/28 gray-scale image.

So whenever the coffee pot was full, it's only then that they would get up and go and take the coffee.

So the world's first webcam was actually a surveillance monitor to watch coffee, which I found really amazing.

So of course, you can make surveillance stuff as well using getUserMedia.

People have made stuff like security cams, baby monitors.

People have made stuff like emotion recognition libraries, in which you can smile and it detects it.

If you frown it detects.

People have made stuff like hotspot recognition.

For example, if you wave your hand in one direction, it does something.

If you wave your hand in the other direction, it does something.

And once again, I made a small thing, which isn't complete yet.

But I think it demonstrates the concept of hotspots.

So let's hope this works.

So you have a keyboard.

I call this the airboard.

Let's see.

I hope this works.

[APPLAUSE] Go back.

Where's the mouse? So yeah, you can do all those kind of crazy stuff.

And of course, you can make-- it has some accessibility or application in some games as well.

You can do this.

Now with getUserMedia, let's take a deeper look.

We all know how to use getUserMedia? everyone knows how to use getUserMedia over here? You'll use Navigator.getUserMedia

with prefixes of course.

Now exceptions over here.

There's a problem because if there's an editor, OK.

But what if there's an exception? So generally speaking, the Mediastream editor, the editor column, I just hang those Mediastream editors.

That's it.

But if you go into the success function, and then some kind of exception happens, then you have to do that a tri catch block within the success function.

And that's a little bit icky.

I haven't seen a lot of people do this.

So in the future, what's going to happen is we're shifting getUserMedia to navigator.mediaD

evices.getUserMedia.

This change is still being done in the spec level.

But it's pretty much-- consensus has been built around this.

And this is in recognition of the fact that computers can have multiple media devices.

You can have multiple cameras.

You can have multiple sound outputs or mics.

And you need a way to detect, which one do you want to use.

So you'll have a function also similar to something called enumerate devices, in which you'll list all the output devices or input devices you want to use.

And of course, getUserMedia will be in the media devices on it.

But it also is going to be promised based.

I mean, that's the discussion right now.

And almost everyone is convinced that, yes this is the way to go.

So this is good.

Pretty much every API which is coming up in the last one year is using promises, and it makes sense for us to do promises as well.

So this is good.

There's still debate.

There's a few people who are against this.

There's some people who are for this.

Mainly the discussion, however, is what you do with navigator.getUserMedia.

So do we have the same callback based approach? Or do we have an approach, in which we just shelve navigator.getUserMedia but

have navigator.MOSgetUserMedia,

with the prefixes and webkit getUserMedia? My opinion though is that since it's already in prefixes, everyone knows that it's open to change.

Let's not go with the callback based approach.

Let's move to promises.

And let's have navigator.MOS and

webkit getUserMedia for people who want to do callbacks.

But overall, we want people to use navigator.mediaD

evices.getUserMedia

with promises.

I think that's the way to go.

If you want to join the discussion, you comment on the mailing this.

These things are still being discussed.

There are other concerns with getUserMedia as well.

You cannot really use getUserMedia in low light.

You cannot use zoom.

You cannot use focus.

You cannot use Flash.

These are major concerns.

Paul Kinlan he went the whole week just using web apps.

And he wrote about it.

And he wrote a lot of stuff.

This is the stuff you said about getUserMedia.

It doesn't offer any kind of advanced features, like focus, flash or zoom.

And it's clearly been designed just for P2P applications like web chat.

rather than a dedicated camera experience, which puts us way below native applications, when it comes to making something as simple as a camera application.

So this is something which we, in the web platform, need to improve.

Fortunately there is a spec going on.

Work is being done, which could provide that.

Is called the Mediastream Image-Capture spec.

And it provides all these kind of things, red eye reduction, exposure, brightness, even zoom, which is pretty cool.

One important thing in this spec is the take photo function.

So right now what's happening is if you want to take a photograph using getUserMedia, it's kind of like a hack.

It's not exactly a photograph.

It's just a frame grab.

You have video.

And you just capture a frame.

That's it.

So in take photo, it's a little bit different because you can specify that I want to take an actual photograph with this kind of sharpness, with this kind of zoom, with this kind of exposure.

So I think this is something which is really, really needed.

And the sooner it gets implemented in browsers, I'll be really happy.

But this isn't enough.

You're still in the spec right now, currently.

There isn't any support for focus and flash And if you make a camera application, no matter how advanced it is, but you can't even focus or can't use flash, it's going to be useless.

So I found a bunch of bugs and talked to the people on a meeting this on and on having a provision like this.

And if you agree, I'll request you guys to do the same.

One more thing, with asking for mic and camera access is the fact that users don't seem to notice the permissions dialogue in a lot of these browsers.

So if you go to any kind of site which uses getUserMedia right now, any kind of big site which does any kind of video chat, you'll find stuff like this.

You'll find this big red arrow, pointing upwards, saying hey, notice this thing over here.

Click over here.

In GoToMeeting Free, for example, in Firefox, they actually have an entire illustration like this, asking the person to click.

So this is something which is a problem.

Yeah and appeared in once again.

And this is in Opera.

But this is further complicated by the fact that permission styles are different in different browsers and in different platforms.

So Chrome on desktop might have the permissions dialogue on the top, but Chrome on mobile might have the permissions dialogue at the bottom.

Opera has the permissions dialogue, in mobile, in the middle, and Firefox upwards.

So it's very difficult to test all the time, and try to see, OK where the permissions dialogue is in which browser, in which platform, and where to point the user to say, hey click over here.

And if you don't do that, then they might not even notice it.

So this is something of an issue right now.

I hope we, as browsers, get a consensus and do a better job of it.

One thing which helps though is HTTPS because with HTTPS, the browser remembers the permission, which means that you only want the user to come once and click Yes.

The next time the person comes to your site, the browser will not ask for permission for enabling the camera.

One more thing, which is being discussed, and this is a little bit-- there isn't clear consensus right now.

But one of the things on the mailing list, which is being discussed right now is making getUserMedia HTTPS only.

And this is because it's a highly sensitive API.

So any kind of man in the middle attack could be there and compromise the security.

We have seen a lot of specifications in the recent past, moving away to something which is HTTPS only.

For example, service workers is one of the most widely known.

And in my personal opinion I think it makes sense.

We have to place the user's above short term discomfort of developers.

So what do you think? I will be happy to know your views as well.

So this was about taking a good hard look at getUserMedia.

What are the problems? What are the great things? What are the things which are coming? But the other part of WebRTC is RTCPeerConnection.

And it's pretty much the heart of WebRTC.

But as a web developer, who has no background in telecommunications, reading about this and watching the tutorials on YouTube or something, and reading the spec it is a little bit discomforting.

Because sometimes people assume that you are from a telecommunications background.

They sometimes have terminologies related to that.

So as a web developer you feel kind of like this.

You know, what the hell? And the thing is, you're inundated with so many acronyms.

There's so many acronyms in WebRTC.

In fact, there's so many acronyms in WebRTC that a person actually made the time to create something like WebRTCglossary.com, which

lists all the acronyms that you encounter in WebRTC and what they mean.

So this is actually pretty helpful.

I would advise you to check it out.

So I just wanted to briefly mention how to do this, in the words of how would web developers understand it.

So the first thing, when you want to make a connection to the other person using WebRTC, you cannot just make a connection like this.

There are firewalls in between.

There are NATs in between.

So even before you try to make a connection, you want a way to punch through these firewalls, to punch through these NATs.

And this is because this is a big problem nowadays.

So you have a protocol called ICE, interactive connectivity establishment.

And what this ICE protocol says is you can use something called a STUN server.

Once again, an acronym.

And what it does is something very simple, which is it just tells you what's your public IP address and port.

That's it.

And generally 85% of the times, it works.

But about 15% of the times, it doesn't.

And in those cases, you have to use something called a turn server.

A turn server is just a relay server.

So what it does is it just takes stuff from one side to the other and vice versa.

So in this case, the connection won't be exactly P2P p because you'll be using a server in between.

But at least the connection will be made.

And if you specify in your options a stun server, as well as a turn server, then it'll try of course, stun first so that you make a P2P connection.

If it fails, then it'll fall back to turn.

And if you're wondering, all these stun servers, and turn servers, where can I find more information? There's a project called the free ICE project, where it lists all the freely available STUN server that you could use in your project right now.

But the thing is, this was about punching through NATs and firewalls.

But how do you actually make a connection? And this is something, which I have struggled with.

Because a lot of people who explained this, explain it in the terminology or someone from a telecommunications background who already knows this stuff.

So I thought that maybe I can explain a little bit differently.

So WebRTC doesn't really say what kind of protocol or service should you use.

You can use WebSockets, XHR, whatever.

But the main thing is to get a certain piece of information from one side to the other, so that afterwards you can make a proper connection.

So what happens is you use a central server.

You have to use it once so that it exchanges information.

And once the P2P connection is made, the server just steps away.

It just plays the role of a matchmaker.

So what happens is in WebRTC, there's a concept of a caller and a callee.

So the caller, it it makes something called a session description.

And it doesn't matter what's said over here.

Basically what it is information about your audio and video codecs, and what type of qualities.

That's it.

So you set the local session description and you create something called and offer based on it and send it to the server.

The server receives the offer, and sends it to the other person.

The other person receives this offer, sets it as its remote description, then creates its own session description, which has information about his own codecs and this and that.

And it sends the answer back.

So the first person now receives the answer, sets the remote description now.

Both sides have the remote and the local description set.

Now we can begin the process of the connectivity checks, punching through firewalls and that kind of stuff.

And afterwards, we get something called ICE candidates, which we can use to create a P2P connection.

So this is, in a nutshell, how signaling works in WebRTC.

And if you think that this is a little bit too complicated, then there are libraries that you could use as well.

The stuff like simple WebRTC, PeerJS rtc.io.

But one of the things that you have to realize, and that I also found with WebRTC is that in a normal WebRTC connection, peer to peer connection, things will not scale well.

For example, if you have just two local people in a call, that's fine.

If you have three people in a call, that's going to be somewhat of a problem.

If it's four, then it's even a bigger problem because what happens is each and every WebRTC connection is encrypted, which means that the more number of peers that you add in a call, the more amount of streams that you have to encrypt in real time.

So after a while, it becomes too much load on the CPU, especially if you're on a mobile connection.

So for example, appear.in, they

have a limit of eight people, and even then it kind of struggles.

The best thing would be around four people.

But CPU is not the only thing you have to be concerned about.

WebRTC also uses VP8 for its video codec-- at least in the browsers, which support it right now-- and the opus codec for audio.

So for, like a 720p video, if you want to send, it's going to take about one to two Mbps of bandwidth.

So for a typical HD call, you have to be prepared for sending about 2.5 Mbps of bandwidth upstream.

And that's going to be somewhat of a problem because a lot of network connections cannot really handle this.

So in those cases, I think it's better to err on the side of caution and actually have a lower resolution video being sent rather than a higher one right.

So this is about bandwidth.

One more thing about WebRTC and signaling and dealing with all these things as a developer is debugging.

And this is somewhat disappointing right now.

When you make a WebRTC call, you can go to WebRTC/internals in Opera or Chrome, and you get a bunch of useful information about the call.

You also have something called the get stats method, which basically is just a dump of the entire call.

But these things are a little bit hard to work with.

What we really need right now and what really is missing from the platform right now is proper developer tools, in the actual browser developer tools to deal and to analyze WebRTC properly.

And this is something which I would happily like.

One more thing is that when you're dealing with WebRTC as a developer, no matter what you do, you're at the mercy of the network.

No matter how well you test your application, in the end, your users in the wild, they'll have their own network connections, which might be crappy, which might be good.

They might be on a mobile phone, which might be crappy, which might be good.

And dealing and debugging is going to be somewhat of a pain.

So this is about debugging.

The third leg of WebRTC is data channels.

And of course, it's about delivering data, whether it's text or binary data.

And the best way to look at data channels is to compare and contrast it with WebSockets.

So data channels, they provide a very high performance P2P, very low latency connection to the other person.

Now even with something as fast as WebSockets, you have a server in between.

So there is some kind of time, which will be used to send it to the server and the server to send it to the other side.

But with WebRTC data channels, it's P2P.

So you know, it's less latency.

If you work with WebSockets, you'll feel right at home with data channels.

It works pretty much exactly the same way, almost by design.

You want people who are familiar with WebSockets to feel really at home when it comes to data channels.

So you have seen event handler, the same functions, and that's good.

But the really cool thing about data channels is that you can make it work exactly like WebSockets, like exactly the same, but you don't have to.

And this is where things get interesting.

Because data channels actually uses this kind of protocol stack.

They use SCTP for access to primary protocol stack, which is a transfer protocol just like you have TCP or UDP.

It's done using DTLS, which is used for encryption, and it goes in the end over UDP.

Nowhere over here that you'll notice is TCP being mentioned.

TCP is used for WebSockets.

But data channels don't have to abide by those rules.

TCP is ordered and reliable in terms of packet transmission.

But you don't have to do that with data channels.

So you have a few options in data channels to configure it almost exactly as you want it.

And this is really, really cool for stuff like games.

So you can make it such that you know, it works exactly like TCP.

You can just say ordered true and reliable true, and that's it.

But if you want, you can make it work exactly like UDP.

You can just say, I don't want it ordered, and I don't want any kind of retransmission of, or retries if a packet is missed.

And you can also have something like partially ordered so you can say that I don't want it reliable, but I want it partially reliable.

In the sense that, if a packet is not given, if it is a packet is missed, they try five times before giving up.

Or try for three seconds before giving up.

And this is where you have really fine grained control over the kind of data that you send and how it's sent.

So this is really, really cool.

And this kind of differentiates it from WebSockets.

In the wild you have projects like Peerflix, PeerCDN, WebTorrent.

And the surprising thing about-- and these are the people who are making it-- and surprising thing for me, when it comes to WebSockets-- sorry about WebRTC is the fact that when people were talking about WebRTC, when it was just being made, people were always talking about video chat and that kind of stuff.

But some of the most innovative stuff, some of the most groundbreaking stuff, has actually happened on the data channel side.

Now we have Peerflix, which is like streaming torrents.

Yeah you have Feross, who has made a peer CDN, which is like a peer to peer CDN.

Amazingly cool.

So data channels is something which you should really, really explore.

Now we come to browser support.

So when it comes to browser support, Chrome, Opera, Firefox, we supported.

But when it comes to Apple, we don't know anything.

So I just expect a few years from now, Apple will be like, yeah now we support WebRTC.

And half of the world will be like, huh? And the other half of the world be like, I knew it.

So let's see what happens over there.

On the other hand, you have Microsoft.

And Microsoft has been a little bit more open about its plans.

They have a site in which people can request user features.

WebRTC is the fourth most requested feature over there.

If you look at their platform status site, you'll see that media captured in streams is in development, which means that getUserMedia is probably coming soon.

But when it comes to WebRTC spec and making the actual connection and that kind of stuff, that probably is not going to happen for the current version.

We're at 1.0, probably

would not happen.

They are showing good signs, positive signs for something called the ORTC spec, which is going to be WebRTC 1.1,

you could say.

So what is that? Back when WebRTC was being standardized and stuff, there were a group of people who said that they're not truly happy with it.

And they made something which-- they made a community group called ORTC, ORTC community group.

And they worked on a kind of a spec on their own.

Later on, they discussed this thing with the original WebRTC group.

And a lot of the stuff is now being integrated.

So you could say that ORTC is kind of like WebRTC 1.1,

rather than a competing specification.

And one of the things which is one of the highlights of WebRTC 1.1, or ORTC, is

the fact that you don't need this SDP exchange, which happens right now.

This will make things-- should make things a little bit less complicated.

At the same time, it offers a little bit more advanced features, a bit more low level stuff.

So you can do is to stuff like simul-cast and all the stuff that you can't really do right now with WebRTC.

A little bit more advanced stuff, and low level stuff.

And just as an outside observer, I've seen people talk about compatibility, and you know to make sure that if you implement ORTC, you shouldn't break existing WebRTC apps.

So what is the work being done over there? People are talking about making some kind of spec library, so that existing WebRTC applications can use that and not break.

But at the same time, people want to use ORTC, or WebRTC 1.1, can dig in

and use the more low level stuff, to have more power and applications.

So in the end though, the end user doesn't really care about what technology are being used.

You could use any kind of technology, but the experience has to be good.

And this is where WebRTC provides you with more options.

The kind of power that you have with WebRTC, you didn't have before.

I just remember, I was talking about in the beginning of the presentation, about calling my grandfather by going to these PCO booths.

And just recently I made an application-- like a test application-- and I put it up on my private server.

And I asked one of my relatives, who lives with my grandfather, to just go to the site using his mobile phone.

And he did.

And I had an actual conversation.

And I could see my grandfather's face.

And that was just so cool, trying to see and trying to think, oh you know how far we've come.

I made something in an afternoon, and I could see my grandfather's face.

And I could do it for free.

So this is something which is really, really awesome.

So WebRTC provides you with the kind of stuff that previously wasn't available before.

For example appear.in,

kind of money that's spent on the server cost was really, really low.

I think it was about $300 to $400 for the entire year.

That's amazingly cheap because the bandwidth is really paid by them, if it's peer to peer connection.

So it just lowers the barrier for entry, when it comes to making really, really kick-ass, world class communication services.

So people have already made peer to peer CDN's.

There's going to be someone who's going to make the next peer to peer social network, probably.

There's going to be someone who makes the next peer to peer video sharing site.

The possibilities are just up to you.

So I hope that you go ahead and play with WebRTC.

And thank you for listening.

[APPLAUSE] Come and join me in my lounge.

We can do a few questions.

All right.

So my experience of WebRTC, I've had a lot of issues with reliability of the connections and stuff.

And is this related to some of the real time problems that we were hearing earlier, or I am I just bad at code? No, it is related to the kind of problems that were talked earlier in this talk.

A lot of the stuff that WebRTC does, it kind of hides the amount of complexity which is required in making a peer to peer connection.

There's a lot of stuff that the browser does under the hood that is just not exposed to web developers because we don't want them to know.

But getting the connection, and that to appear to be a connection, is hard, especially behind firewalls, behind NATs.

So sometimes it's real work.

But sometimes you have to use some other techniques.

So we've seen protesters in Hong Kong recently sort of used, create a mesh network for communication.

Is just something that WebRTC could do? Could we do this with the web? Could we build that kind of network? Yeah you could.

You don't necessarily have to use a server which is located somewhere else.

You can use any kind of thing.

So yeah, it is pretty much possible to do this.

There's also a project called Named WebSockets, which one of my colleagues, Rich Tibbett is working on.

And what it does is create a way to do totally local connections and use WebRTC to do it.

And it also has a provision for web applications to talk with other things, like native applications using Bonjour and stuff like this.

So this is a really, really cool project.

I'll ask you all to check it out.

Named WebSockets, it's on GitHub.

What do you need that's in terms of centralization, in order to make that work? Would they, if you wanted to create a mesh network, would you just have to create stun and turn network on your small network and you're good to go? I'm not really sure about this.

But with the Named WebSockets thing, it is completely possible to do it because it creates a local service.

And then you can use that local service on the local network.

And it has automatic peer discovery as well.

So I think that would be a better way to do the stuff that you're talking about.

If I'm using a turn server, and that's one where the full data is going through another server, right? What are the security implications of that? Yeah, so a turn server, the only job is to relay stuff.

So there is no decryption happening at the turn server.

And if you want to use a turn server, there's an open source turn server called, I think, the RC57661 that you could use.

So it's open source.

You can see how it works.

But in general, a turn server's job is just to relay stuff to the other side.

So it's not actually possible for it to do decryption? Decryption, no.

That's really cool.

What about a stun server? I mean that's just negotiating the IPs.

But is there a security issue there? Is the-- I mean, are you kind of leaking information about calls being made? Yeah so you have to expose some kind of information, which is like your public IP.

But you know, that's needed, right? So what can you do? That is needed.

If you're under a firewall and you have like a private IP address and a public IP address, you need some way to negotiate that kind of stuff.

So is there anything stopping us building a true mobile phone app, just in the browsers? Or have we got everything we need to do that today? I didn't get exactly what you're talking about.

So if I wanted to recreate the dialer app on my phone but just in the web, assuming that I'm just going to call the people on the web, are we there with that? Is that everything, everything in the browser? There's actually a library called sipML5, in which you can use it to dial people, actually dial people, and actually make phone calls and stuff like this.

So there are a number of projects going on, in which you can hook up WebRTC with conventional communication technologies to do some really cool stuff.

You can hook it up with sip.

You can hook up with XMPP, whatever.

So in that case this there's some kind of server that's acting as a WebRTC client.

I guess it's dealing with the old telephone.

Yeah, dealing with the signaling and everything.

So are there libraries already that it acts as a WebRTC client, but on the server? Does that stuff exist? There are libraries like what I talked about, in which they can make phone calls on your behalf.

You just connect it through.

So those things are possible right now.

When we're dealing with some of your demos, we're processing video, and I guess that processing was also happening on the main thread? Was that using Canvas? It was using Canvas.

And Canvas, as you know, doing something in real time with the camera is expensive.

So you can use stuff like web workers and shared workers, and stuff like that.

I didn't use it right now, but it is a good idea to do it.

I guess in that case, you would be-- because you would still receive the frame in the mainframe.

But you'd pass it off and then pass it back.

It was a suggestion that came from the audience.

What was really interesting is that, would there be a way-- I guess it's a little bit like service workers, to have the information arrive into another thread, where you can maybe transform it that stage before you send on.

I don't know if that's been thought about before.

I don't know.

I thought of the offer, it was I just wondered if it was thought about.

Is there a freeJS equivalent for RTC? Is there a library that, sort of, takes out that really low level, into something that's going to be easiest to deal with? Yeah so when it comes to the signaling part, you have stuff like RTC IO, or simple WebRTC.

If you just want to make a proper video chat, you can do that using that.

When it comes to the stuff using computer vision stuff like, emotion recognition and that kind of stuff, there's a library called Clam Tracker, I think, in which you can do emotion recognition.

There's a library called JSfeat, in which you can detect different features in computer vision, and do stuff with it, like in a basic edge detection and that kind of stuff.

So there's a number of libraries on a number of facets of WebRTC, whether it's computer vision, whether it's signaling, whether it's just normal video chat.

If we're having problems with WebRTC adoption-- you showed the graph there-- if this move's going to happen to force it to be over HTTPS only, that's going to hurt adoption a lot more, isn't it? Yeah but in the end, in my opinion is, you-- and my opinion is, by the way, changed.

initially.

I was against having getUserMedia as HTTPS only.

But then Anne van Kesteren, he made some really good points, which is you can't really defend against a man in the middle attack.

And getUserMedia's such a privacy sensitive thing.

Someone getting access your camera, that is something which is pretty hard core.

So you have to take all kinds of steps for it.

And I think with all of the different moves that are happening in web standards right now, with service workers going HTTPS only and a few others, I think this is the right time to make it HTTPS only.

But this is just my opinion.

Other people might disagree.

We had exactly the same discussion with the service worker stuff.

Like I really was fighting for it to be, no we want to be able to caches on HTTP.

We want this to be.

But yeah, when you can see the kind of attacks people can do, there's no-- it's a pretty big bug, the app cache is over HTTP.

In fact, maybe HTTP in general is just a bug, right.

It should've been secure from the very start.

So Microsoft are considering it.

They're starting to do stuff.

Are they cautious with this stuff because of their work with Skype, a product they own? I'm not a Microsoft guy, so I can't say.

But what as an outside observer, just looking at the stuff which is happening with ORTC and the new stuff being discussed, they're pretty active.

They're taking a pretty good look.

They have some good points in discussions, as well.

So I think they're taking a pretty serious look, when it comes to WebRTC and having it in the browser eventually.

So at least their taking a serious look.

We don't know about the other browser.

Looking at some of the API, the change the API from navigator getUserMedia, to navigator media devices getUserMedia, how about navigator.mediadevices.get?

Can we just have .get?

Can we make this decision right now? Let's do it.

That's brilliant, sorted, OK.

This is how Sander's group should operate, just two people petrified on stage.

Well I'm petrified.

Don't know about you.

The image capture API, that stuff looked really interesting.

What about video capture? It seems like we want an equivalent to do that as well.

There's an API-- I didn't have time-- but there's an API called the Mediastream Recorder API.

I haven't, to be honest, taken a good hard look at it.

But there is something called Mediastream Recorder, in which you can-- there is a provision of recording stuff as well.

And of course, there is the media devices API, or something like this, in which you overload the input element, so that if you're on a mobile phone, for example, and you say input type is equal to something, I think, camera or something.

There's a whole article on devOpera that you can take a look at.

And what it does is it overloads the input element.

So if you click on it, it opens up the native camera application.

And then you can do whatever you want.

You can do use flash, or you can use zoom or whatever.

But it's just ceding control to the native application.

And as web developers, we don't want to cede control to a native application to take pictures.

We want very, very low level control on what kind of brightness, what kind of zoom we have, using JavaScript.

We don't want to just give control away like this.

Well I suppose it make sense to have both.

We definitely need the low level stuff but.

If I wanted to build something for a site, just upload your avatar, or click here to take a photo, I guess-- In those cases, it's fine.

But in some other cases, like if you make a dedicated camera application, you want all of these things in JavaScript.

That's my opinion.

So when you've got-- you're having a RTC call with many people at once, does the turn server act like a multicast? Or is that the simulcast thing you mentioned? Simulcast, I'm not-- I regularly get confused between simulcast and multicast.

So don't ask me about this.

But yeah in general, a turn server's only job is to just send it to the other side.

And that's it.

And the more number of people there are in the call, the more activity has to be done by the turn server .

You end up with, if you've got five calls going through a turn server, you end up with five streams potentially coming from different turn servers, I guess.

There's also a solution called an MCU that you could use, which is like a big fat really, really powerful server in between that you could use to offload this kind of stuff to that particular node.

That's more like what Google are doing, with the hangout stuff.

They're probably using an MCU.

That makes sense.

Or some other, more powerful stuff.

You mentioned improving debugging.

Are there any plans there? What do you think these debugging tools might look like? What do we need? We need-- well some of the stuff is, Google is doing, as you know-- I don't actually.

In the WebRTC internally as well, they just have their own prefix things that they use for measuring different called parameters.

So we need this inside the developer tools itself, rather than going to a particular page, or using a particular API, I think.

If it's like one more tab in the developer tools, I think that'll be really cool.

That's excellent stuff.

That was absolutely brilliant.

Ladies and gentlemen, Shwetank Dixit.

[APPLAUSE]