Sean DuBois - 008: WebRTC & Go makes Pion: The best way 2 build sub-2nd and secure P2P real-time UC

okay welcome to this next session we

have shown to be a year talking about

one of the most exciting new projects in

WebRTC environment which is spy on which

I'm really interested about mostly

because it breaks the so called WebRTC

monoculture that you may have heard

about so everybody uses live web RTC for

everything and it's nice instead to see

a WebRTC stack that is done from scratch

building new things from scratch again

so please gentlemen amaze us now try my

best

well thank you very much and thank you

for putting on this con con it's

fantastic to see such a focus on open

source so I'm Sean I have one of the

original founders of PI on today we'll

just talk about the WebRTC

implementation first off even though I'm

standing up here I'd like to thank

everyone who's contributed it's very

easy to see me as you know as working on

it but this is really what makes a

difference there's some people on here

who put in you know hundreds of hours

and some people who have done fantastic

of telling oh here's a little bug and

here's how to fix it so this is a little

out of date we've had a couple

contributions in the past week but yeah

this is the lifeblood of the project so

first off what is PI on PI on it's sort

of a collective for building RTC and

peer-to-peer projects there's no

commercial ownership I work on this on

my personal time it's not really a

project has any funding right now I I

also don't plan on making any money off

of it I'd like to expand out in other

places but I'd like to see it owned by

the community more than anything else so

how it works right now is if you commit

to PI on and you merge something you get

a commitment and that's all it takes and

to merge to match you just need an

approval from another contributor so all

that it takes in a week's time you can

fix a bug and be merged by someone who

already contributes and from then on you

can approve other people and other

people can approve you

so there's you're not there's no

deadlock on a single single person

merging pr's like me - not just WebRTC

because everything is 100% and go we had

to implement all these individual parts

so DTLS ice turn all these individual

things

so this is a quite undertaking and on

top of that this is available for other

projects I've seen a lot of like small

embedded device IOT wants to use just

DTLS or SCTP and the other fantastic

thing is there's many people who are

very passionate about these individual

technologies we have one person who just

loves DTLS they want to implement all

the rfcs they know about it than than

anyone I've ever talked to and so that's

why what I really want to encourage is

ownership of these each individual parts

and get the people that are passionate

about them I'm personally passionate

about WebRTC as a whole so it's great to

see people jumping in individual parts

and there's other stuff in the future

like an RTP engine so some people just

want to send RTP over UDP like now we

want to have a pluggable jitter buffer

that's agnostic to how whatever you want

to do it's still a growing community

soap ions only existed for a year in a

month or so and in that time we have

about 45 contributors and ten active

large and small companies use it we have

a lot of startups where we have there's

tio have a lot of large companies where

we have them pulling down with it large

projects and stuff like that and diverse

use cases which is one of the ones

that's most interesting to me so I'll

hear people doing the classical things

that I'm used to they're building

another source and bringing it in and

taking them out to WebRTC and you'll

have these really weird ones like I I

want to make a customisable Raspberry Pi

security cam system which just there's

nothing I'd ever thought of but it's

great that we can empower use cases like

that so in to have you so the community

not just for PI on we have a slack

channel with like 250 people and we love

free

standards other projects really I'd like

to just bring the key the WebRTC

community closer together I hear that

people say it's it's hard to find one

place to talk and the second is the

knowledge base that we're trying to work

on

so because we implemented DTLS and all

those things we want to write down that

deep knowledge not just about the WebRTC

API but about how all those individual

technologies work so one of the first

benefits you'll know notice is pion is

100% and go so if you ever written

before some of this will be boring but

if if it's all new to you hopefully this

will convince you that goes worth your

time go fast I would say like

performance somewhere between plus plus

and Java and that speed that matters put

this element so if you've ever worked on

systems everything like that the nice

thing is go doesn't have any then if you

want to start using PI on all it is is

that second command just go get high on

RTC it'll pull everything down and

because it's a hundred percent and go it

doesn't matter what bar you get a brand

new minimal decid install you can have a

map you can have any embedded system

it'll work and it's also sub-second

build time so you pull that down

completely and it's very easy to

optimize the nice thing which is so

simple and fast there's really not a lot

of to worry about you just have to track

allocations and make sure you don't do

the usual bad things portable and simple

deploys if you're building WebRTC

products you've probably run into this

before you waste a lot of time debugging

and testing you'll you'll have users say

they have an individual issue and you

want to deploy and have them test again

how do you do that if you have it takes

30 seconds way out

so all that it takes is setting those

two environment variables and from my

Mac I can target Linux are my kin like

argot darwin i can target anything i

want just set those things build your

project and it builds a nice binder and

i can deploy it on it anywhere we have

people that are deploying to android

devices we have people that are

deploying to building little binaries so

they can throw a Mac of Windows and

Linux because they have little desktop

applications that are doing peer-to-peer

maintainable really important for an

open-source project because the language

is so simple you about we do software

development at scale let's make it as

simple as possible if you be if you know

Python you've probably been and learn go

in a weekend it's so easy to get other

people involved so simple and

opinionated the other fantastic thing is

the concurrency and cryptography space

so easy to get started and again in that

same vein the language is opinionated so

because I don't have to worry about a

large language C++ that there's not a

subset of go that you need to learn that

you know the entire thing that you can

fit the entire language and pages of

spec memory safe so this is one that I

had a lot of impact on me chromium has

this rule of two so way that chromium is

designed it's multi-process so if you're

doing some sort of traffic you can't

have it live in a high privileged

situation so I'm you're not allowed to

have user information and process

foreign foreign inputs in the same

process because it fails this rule of

two so most the time you'll process

something and then you'll send it over

RPC to another process that's important

because you don't want to expose user

information and you'll see this like

recently with whatsapp and the rtcp

parsing go doesn't fall into the same

trap you don't have to worry if there's

an if there's a bug in our tcp parsing

or there's a bug in some other subset

the worst thing that happened is the

entire application will you know sig

abort and throw out you don't have to

worry about what can come from that and

the other whole fuzzing and code bridge

something that other people brought to

the project but it's very interesting to

see is how can we go in and make sure

that even though we don't have to worry

about these same types of problems we

still don't want these runtime crashes

so

if you already know how to use WebRTC

you know you know how to use pine it's

just like a go implementation of the API

you already know and love great offers

set remote description tracks pulling

off ice candidates you already know how

to use it and my opinion is that that

when you come to a point where you're

building your application you're gonna

want to scale it you're gonna want to

have to fit into these custom use cases

and this is why I'm happy that we adhere

to this existing API I switch over to

Lib WebRTC or a i/o RTC we should

encourage that like we should have that

kind of flexibility so that people can

use whatever project they want and it

makes every project better and the other

thing I've come to appreciate is the

weather to see API is well designed you

have all these things you can use design

your own API but then you'll say oh I

need these specific stats or I want to

know something about you know the RTC

DTLS transport you don't have to worry

about reinventing the wheel you don't

have to worry about looking up these

proprietary information about PI on if

you already know how to use WebRTC

these problems are solved for you and

then further the actual definition of

the project is it split into subsystems

how often do you pull down a new version

of a library and you find out that Oh

everything's fantastic I have this great

new feature but there's a bug in ice

that drives me crazy and I you know I

just how do I roll back the nice thing

apply on is it split into all these

little packages and so you can pull down

you know version 2.1 of Pi on but you

can pin to 2.0 of ice so you can roll

ahead and you might not get the newest

features in ice but at least you can

keep rolling forward and and be sure

that the rest of it is working you can

also add custom behavior a lot of

companies have these unique use cases

that I want to empower so let's say you

want to do interesting unique statistics

about ice you can use 99% of Pi ons

codebase but then just have your little

custom ice component and as long as you

adhere to the same public API that that

we had that we require you can add any

custom behavior you want so you want to

do analytics you want to do some some

custom behavior let's say you're working

with the weather to the implementation

that's busted and you want to anyone

enable that this is this enables that

there's a little package graph that

shows soap ion leverage you see it's

really just using all these individual

components and wrapping them up so the

main repo itself is a simple you know

taking the STP and parse it taken that

startup DTLS but because that the at the

base Weber to see is just a collection

of all of these interesting complicated

protocols but WebRTC itself is isn't

handling these parts so these are these

are one of the trade-offs that I think

is worth making

but it'll be confusing coming from other

implementations is the media coding is

dev controlled so our API just emits RTP

packets and consumes RTP packets so if

you're doing the coding yourself and

you're gonna think about the things like

I need to emit another keyframe because

there's been loss and stuff like that

you're gonna have to do it yourself but

why that's important is it enables

things like having distributing pre pre

recorded content you know there's a lot

of these custom use cases that aren't

enabled by existing WebRTC

implementations if I have a video file

on disk can just open that and Payan

iterate the frames and send it off those

are the kind of use cases we want to

empower so even though we don't have the

coding built in and you're gonna have to

listen to the feedback from from PI on

it enables all these custom things like

we have this project gunk that does wet

rtmp to WebRTC in HLS and a lot of those

use cases are really popular quic is

another one so right now we have quick

support via ortc so you it's not in the

SDP and it doesn't work with chromium on

this landed in December before the

chromium version came out but I'm

working on moving over to chromium

version soon and then we also have

features that don't find in WebRTC so

this was started by hugo and this is a

fantastic idea is we have all these

things that aren't addressed by WebRTC

yet like how can we make that happen and

so on top of so you can just pass to the

when you create a new rtcpeerconnection

that you're used to and have some of

this proprietary behavior so things like

I want to control how long it

candidate or I want to reject all host

candidates and force server reflexive

this is where we let those things happen

I'm amazed at the unique use cases we

have people that are stuck with broken

firewalls that you know this port range

doesn't work for some reason and we have

these so there's about like 10 different

flags or toggles and eventually it'll

add up but at this time I hope that we

can push these into the WebRTC spec

themselves but these are a lot of these

and unlike the custom mdns behavior I

saw on the mailing list there was talk

about moving this into WebRTC itself

this is one that has saved me a lot of

time is user-defined logging so because

with live WebRTC if you go to debug a

certain issue you can't really log in a

certain subsystem you can't dump that

certain part so with PI and you can see

an example at the bottom I want to get a

trace log of ice but every other

component I just want to see all debug

logs so if you have if you have an issue

in ice you can set one environment

variable and all of a sudden you can get

a trace and print out all the ice

packets or DTLS or sccp since every

individual component respects these

environment variables it makes debugging

incredibly easy and it also doesn't have

to go to standard out it can go to any

sort of user callback so in the rtcp

constructor you can specify okay here's

this callback and then you just have to

satisfy you know trace debug warned and

that's it so a lot of people are sitting

out to their whatever logging system

that they prefer and wasm this one I

don't know enough this was contributed

by Alex and this is a the idea is you

should be able to write your code and go

and then be able to target the browser

as well so one code base you can target

those and I think this is fantastic

many times do we like write these custom

signaling protocols and we have an

implementation in in C++ and we have an

implementation in Java scripts and

they're all broken in different weird

ways this will help keep your signaling

code sane so you only have to worry

about writing at once and it'll work

everywhere and it's like on a you

already have this wide variety of

platforms that go targets and the

browser is just one more so be net is

the idea that if you're frustrated with

your application falling down in

real-world scenarios

so to the pion constructor you can pass

a topology and on top of that you can

pass the network and it can so you say

oh I have three peers and one is behind

they you know an app one is doesn't

allow to see PE only and stuff like that

so you can set all of these network

conditions jitter and loss and then you

can run it and just in software you can

simulate exactly the conditions that

your application that you can perform

under you should you don't have to go

out to the real world and try to set

things on the you have these nice

applications that let you set on your

own loopback these network conditions

but this will let you build complicated

things and this actually helps us has

helped us trace down a few bugs in pi in

itself in our test suite we defined like

here I'm gonna have two PI on WebRTC

instances talk to each other over UDP

but we're gonna drop 80% and then you

find weird bugs and weird places because

it's fantastic for learning I'm gonna

probably mispronounce his name but you

taco did a fantastic job of building

this complicated system now I sit down I

see like oh what happens when I have to

symmetric Nats talk to each other it's

just my understanding of things has

gotten so much better from being able to

just simulate all this stuff instead of

having to read it or hope that I can

figure it out on my own and then what

are the use cases that Python targets

the first one that you'll come to think

of is like the standard WebRTC servers

so one of the saw was WebRTC G speech so

user wrote this thing where you can sit

down and it's just a single go binaries

dot slash go and run it and it's a web

UI you hit start and I talked it

captures your audio over WebRTC and

sends it to the Google speech API and

returns back with you what you sent I

mean best part is how easy the deploy is

is a single static binary the HTML is

contained in it and I just downloading

it goes you hit that start button that

sort of feels like

web ttyl so I think the even though the

media gets a lot of love the data

channels is what it is going to enable a

lot of interesting use cases this one is

someone built something so that you can

share your terminal with a web browser

or you can have two terminals shared to

each other over WebRTC you really just

have to share the SD piece and then you

have two terminals that are provided to

each other so on the left and the right

you know if you wanted to help someone

debug an issue in a remote data center

or you want to connect to peers to each

other this is how you do it this is also

great a lot of small companies I've seen

you know you'll have your SSH jump box

you'll SSH into a server on the outside

and then you'll SSH into the inside I'm

this gonna be great you don't have to

expose any you can keep your network a

little simpler and there's a project how

many people are tired of having to start

x11 to jump into a conference call a ski

RTC was a fun little thing that someone

wrote where you can just download a

single binary it's statically linked it

works on Mac right now just type you

know dot slash to ASCII RTC and you're

popped up and you see someone else's

face and you have data channels as well

so you can talk to each other I think

it's great that there's this conception

that WebRTC is really for the browser I

really want to see more things that

target not just the browser but can use

anywhere and then RTC tunnel is a

project that someone wrote and a while

back was one of the first projects that

was using PI on and it was to build

Network tunnels from your browser to and

a back-end and really you're just

tunneling TCP traffic in setp

and they had all these crazy cool ideas

so how nice would it be if you're trying

to use a rediff server and all of a

sudden you can just pop it open in your

browser and via WebRTC influence the

state of the server and my idea like

they talk about like building a

browser-based IDE so instead of having

to dip your stick where here's your

chance that you can own your own little

VPS and a data center and be able to and

be able to operate on it and how

fantastic gonna be you know how often do

you you don't want to download some

binary and run it locally to test it

especially if it's like some open source

projects you just want to see how it

at the command-line here's a dumb little

one that I had fun for just my own fun

with like 80 lines of go using an

existing project for go C be able to do

some basic like image detection and show

motion detection and all I'm doing is

really just I send the video buffers to

go see B and then I take the video

buffers back and display them to the

user and it sort of feels like magic

that with only 80 lines because Payan

and go C V abstracts away so much that

I'm able to do this but there's so many

cool project you can do on the back end

you can do like motion detection you can

do like speech attacks all these

interesting things here's one that's

dear to my heart is the cloud gaming so

someone wrote an NES emulator and they

ship they ship the video frames back

from the VPS whenever data center you

want and then they send the key events

to that it's a fantastical project I

liked and I'm hoping that we can see it

expand even more fantastic I think the

internet connections get bet and the

technology gets better we're gonna see

more things like this and I'd love to

see that the sought that the user

maintains control of the software I know

there's projects coming out the idea is

that you'll you'll rent a game from some

company and then you'll play it on their

servers but here's an opportunity that

we can build something that you maintain

and own the software but you still can

play in a remote data center so getting

started we have a couple interesting

examples we have the normal like the

data channels example where you know you

plop two SDPs the minimal SFU so if

think about the API if you wanted to

build like an S fu in your browser you

would have you would spin up to

rtcpeerconnection 's you would send your

video to one peer connection and then

that peer connection would connect out

to other people that's really all that

that is and then the ghost CV we talked

about saving good disk you see a lot of

people that they have like they want to

do security systems and they want to

slurp up that video and save it and then

we have built into the code base we have

IVF and opus so you just use those and

then all these already popular in vim

open-source projects like Janousek

GStreamer like how can you hook into

these existing systems and then the

other nice thing is the go doc so with

go you have all of the sense it's all

static so just jump in see the code

exactly as it's defined so you could go

to go dot dot dot org and see how Payan

is driven and see the actual definitions

of all the structs and all the

configuration so if you've ever written

JavaScript you know that it's sort of

fluid you know you have these great

resources like the Mozilla developer

Network and other stuff like that but

there's not it's not really a true

source of information it's something

that someone has to write the nice thing

about this is this is generated directly

from the code so you can jump in and

browse around and it has this nice

generated generated code output and of

course pi is an open source project so

we we need you more than anything here's

some interesting problems I always have

people asked like how can I get involved

these are the things that I'd like to

see so better congestion control so we

have congestion control and SCTP and

then we expect the user to do some of

that on their own in the media coding

we'd like to see this RTP engine so

we've had a couple companies that have

went and implemented themselves we have

to go and implement a good one ourselves

so we have like a basic static one but

solving that problem having like a

dynamic jitter buffer that can grow to

latency and we'd like to see something

where you can pick one ahead of time

there's a lot of use cases where you

think oh I I don't care about packet

loss because I want this you know this

needs to be real-time 200 milliseconds

as a minimum you have others you're like

I don't care it can grow and shrink but

I need perfect video quality and then

another important one is getting that

end-to-end testing which is not one of

the fun problems but definitely some we

need and then web design and user out I

think we have a lot of go developers and

we have a hard time finding joy design

and and building like beautiful web

interfaces we have a lot of people that

like to jump in the channel the first

thing I'm like how do I do things and so

to me it seems obvious because I've been

working on the project but I need that

help of that user perspective on the

outside if you've never used iron before

but you're interested in getting

involved in an open-source project I

think this is a fantastic opportunity to

come and give your feedback and say

this is how you can do a better job of

presenting information and helping

people learn and then we want to hear

from everyone I don't I don't if you're

an open-source project if you have a

company that you're trying to build a

custom use case you don't have to be if

the conversation doesn't have to be in

public feel free to email me personally

or feel free to send me a PM on slack I

just want to hear from people so we

could make the project better I know a

lot of people have come up to me and

passing and told me that use it I'm

always surprised at the interesting use

cases but I I can't figure it out if I

don't hear from you directly so yeah so

that's it thank you very much thanks

Sean any any questions for Sean

so just curiosity the does the wasn't

get compiled down and use the browsers

implementation of the WebSocket or

assets still use the go implementation

of WebSocket web RTP

yeah yeah it uses the browser

implementation so there's like a you we

have these tags in the code base where

at the top of the header we'd say like

use the J this like use the JSF

limitation or use our implementation so

yeah you

we're lucky we like in the cases like

the getusermedia and all that stuff that

all uses the browser implementation yeah

okay nice well I wasn't here right in

the beginning but you put up all the

list of contributors and you've got such

a lot of contributors in quite a short

time so what's the trick with that yeah

I think the the biggest trick is giving

people that ownership so one of the

things is when someone joins they have

as much commit right as me after the

first time they commit it hasn't burned

me yet and I really very much believe

that that's that's the kind of model we

need i I feel I see a lot of people that

get burned out pretty quickly because

it's friction added by me and it's

usually pointless friction you know I I

stress and fret that you know the code

that contribute isn't gonna be good

enough but I think a lot of these things

are a two-way door like maybe they're

your first your first contribution isn't

that great but I know that as we go on

like we'll just make things better and

better I think as long as the project

moves quickly and we test rigorously

like we can avoid that I think a lot of

times like a lot of the stress about bad

contributions comes from the project

itself not having great testing are

great infrastructure and I was really

guilty of that at first as well

when Payan first started a lot of times

the user made this great contribution

this big commit that fix a lot of things

but added another bug and we got all

stressed out and acted like it was a big

deal but the reality is situation you

just need to fix it quickly and catch

things are away any other questions then

let's give Sean a big round of applause

thank you

[Applause]