►
Description
Introduction to Media Streaming Mesh - Giles Heron, Cisco
Media Streaming Mesh is a new concept for enabling real-time media applications in Kubernetes. These applications are usually based on RTP rather than TCP and hence are incompatible with web-proxy based service meshes. In addition they often use out of band TCP control channels (e.g. RTSP, SIP etc.) to negotiate UDP port numbers, and this is incompatible with the kube-proxy ClusterIP NAT. There is, however, a desire from the developers of these platforms to move away from monolithic applications towards the micro-services architecture, and to be able to leverage a cloud-native infrastructure providing load balancing, encryption and observability. Media Streaming Mesh enables this.
A
I
certainly
am
so
my
name's
giles
heron
I'm
going
to
talk
about
media
streaming
mesh,
which
is
a
new
open
source
project
that
we've
spun
up
with
in
cisco
and
we're
looking
for
others
to
collaborate
with
us
on
it.
The
strap
line
here
is
real-time
media
in
kubernetes.
So
really,
this
project
came
from
a
recognition
that
kubernetes
was
more
focused
on,
seems
to
be
more
focused
on
web
applications
than
on
real-time
media,
and
that
was
really
what
we
wanted
to
address.
A
That's
a
very
different
behavior
from
what
you
see
in
streaming,
where
you
tend
to
have
one
canonical
source,
pushing
pushing
data
out.
But,
as
I
say,
we
particularly
wanted
to
focus
in
this
project
on
the
real
time
space,
and
initially
we
looked
more
generally
just
anything
real
time,
so
it
could
have
been
gaming,
for
example,
as
well
as
media.
A
What
we
discovered
was
that
google,
in
fact,
have
a
project
called
quilkin.
I
don't
know
if
anyone
here
has
come
across
it
doing.
Udp
proxies
for
gaming
and
when
we
looked
at
media
and
specifically
at
rtp,
we
decided
to
focus
there
so
we're
very
much
focused
on
real-time
media.
Anything
rtp
based
there
are
various
challenges
there
which
we'll
come
on
to,
and
I
guess
the
other
thing
to
mention.
I
said:
that's
a
fuzzy
application
taxonomy
and
I
mean
by
that
it's
fuzzy
in
both
of
these
dimensions.
A
So,
for
example,
you
know
when
is
something
interactive
and
when
is
it
streaming
if
you're
on
a
zoom
or
a
webex
call?
Actually,
the
experience
is
very
interactive
one,
but
what's
happening.
If
you
look
at
the
packet
layer,
you
know,
as
you
have
all
these
streams
going,
each
user
is
effectively
streaming
their
video
to
all
the
others
and
in
the
real
time,
non
real
time.
Well,
there
are
two
of
us
working
on
this
project
so
far
we're
both
europeans.
A
Unfortunately,
the
other
one
is
actually
based
in
the
u.s.
So
it
might
well
be
that
I'm
watching
a
football
match
on
my
cable
tv
subscription,
so
I'm
seeing
things
as
they
happen.
More
or
less.
Those
of
you
of
my
kind
of
age
will
remember
the
days
of
analog
tv
where
things
were
literally
up
to
the
second
and
the
news
would
count
up
towards
the
hour
beforehand.
A
You
know
we
don't
quite
have
that
now
with
decoding
digital
tv,
but
my
colleagues
might
be
watching
over
the
top.
So
his
challenge
is
going
to
be
that
if
I
text
him
and
say
wasn't
that
an
amazing
goal
he'll
be
like
well,
I
haven't
seen
it
yet.
What
are
you
saying
so
real
time
itself
can
be
a
bit
fuzzy,
and
I
think
that's
really
the
contention
here
that
if
we
look
at
you
know
real-time
media
over
the
internet,
clearly
things
like
webex
and
zoom
prove
we
can
do
effectively
real
time.
A
A
So
what
we're
trying
to
do
here
effectively
is
do
all
the
same
things
that
a
service
mesh
would
do
for
you
in
terms
of
web
traffic,
so
things
like
observability
things
like
security,
so
we
want
to
do
things
like
spiffy
aspire,
we're
going
to
use
things
like
srtp
to
encrypt
our
traffic,
because
you
know
you
writing
an
app.
You
shouldn't
have
to
care
about
that
stuff,
that
that
should
be
a
deployment
consideration,
but
do
all
of
that,
while
achieving
very
low
latency.
A
So
you
know
an
rtp
proxy
is
going
to
take
a
packet
in
fire
one
or
more
packets
out.
It's
not
going
to
be
buffering
and
in
terms
of
deployability
keeping
the
footprint
really
low,
and
I
guess
that's
you
know
that's
particularly
pertinent
to
what
we're
talking
about
today
in
terms
of
edge
and
really
that
was
where
this
project
started,
with
looking
at
rtsp
cameras
being
deployed
at
the
edge
and
where
you
didn't
really
want
to
deploy
the
whole
istio
envoy
stack,
which
I
think
is
about
40
megabytes
per
pod,
that
the
sidecar
adds
rhomboid.
A
So
you
can
use
rtp
for
that,
for
example,
but
really
focusing
on
these
video
use
cases,
whether
in
the
sort
of
tv
production
space
and
streaming
over
the
internet,
but
also
things
like
the
retail
and
industrial
edge,
particularly
around
things
like,
as
I
say,
rtsp
cameras
doing
analytics
at
the
edge,
but
also,
of
course,
you
know,
real-time
collaboration.
So
if
you
were
watching
this
in
real
time
rather
than
with
https
lag,
it'll
be
that
kind
of
stuff.
A
So
in
terms
of
the
video
applications,
I
guess
we're
looking
to
see
where,
where
might
this
technology
be
useful?
So
you
know,
if
you
think
about
it,
you
might
have
cameras
again.
Take
the
football
example:
cameras
out
the
football
match
streaming
back
summer
in
the
studio
you'll
be
mixing
stuff
together,
you'll
be
bringing
in
you
know
the
score
from
other
games
the
score
in
this
game.
A
That
kind
of
thing
and
then,
of
course,
you
need
to
encode
it
to
send
it
out
whether
it's
going
over
the
air
or
whether
it's
going
out
over
the
internet
and,
of
course,
because
of
sending
it
out
over
the
internet.
There
might
be.
You
know,
various
different
bit
rates,
various
different
encodings,
various
different
protection
schemes,
and
so
all
of
that
you
might
want
to
do
as
a
cloud-based
thing
and
then
delivery
out
to
the
caches.
And
then
you
know
the
final
hop
to
the
user.
A
A
A
Even
if
your
final
hop
of
your
football
match
is
being
delivered
to
users
using
dash
or
hls,
it
might
be
that
you
at
least
want
the
caches
themselves
to
be
right
up
to
the
second,
and
there
are
things
you
can
do
there,
for
example
stream
it
over
rtp
but
send
it
over.
Two
paths,
maybe
add
f,
that
sort
of
thing
to
make
sure
the
caches
are
up
to
date
and
have
the
content
where
it
might
get
interesting.
A
But
when
we
come
to
that
final
hot,
you
know
really
that's
where
a
lot
of
the
challenges
are
going
to
be
so,
for
example,
if
I'm
watching
it
on
my
television
set,
which
is
streaming
over
the
internet.
Well,
how
do
I
update
the
software
on
that
when
you
know,
when
can
we
get
software
updates
to
support
rtp
over
quick.
A
But
the
ultimately,
what
we
want
to
do
is
take
in
an
rtp
stream
that
might
be
regular,
rtp
of
udp
and
then
stream
it
out
over
quick
to
those
users.
You
end
up
modifying
the
protocols,
so
now
he's
looked
at
some
of
his
video
process.
Typically,
what
you'll
see
is
you'll
have
a
tcp
control
plane
that
control
plane
will
then
negotiate
the
udp
ports
that
you
use
for
your
actual
media
streams.
A
What's
interesting
for
anyone
who's
watching
this
whole
media
space
in
the
itf
is
there's
a
lot
of
activity
now
around.
Do
we
do
rtp
over
quick
or
do
we
create
new
media
protocols
that
run
over
quick?
So
there's
proposals
from
the
likes
of
facebook
and
twitch
things
like.
I
think
it's
rush
warp.
I
think
it's
another
one.
They
they
sometimes
do
really
cool
things
like
they'll,
take
one
video
frame
and
send
it
over
a
quick
stream,
so
you'll
only
get
head
of
line
blocking
for
that
one
video
frame
not
for
the
whole
thing.
A
So
rtsp
security
cameras.
Typically
you
might,
I
guess
there
are
these
two
really
extreme
ends
of
the
spectrum.
One's
going
to
be
you've
got
a
lot
of
small
sites,
so
maybe
it's
coffee
shops
you'd
be
a
large,
a
large
coffee.
What
would
you
call
them
retailer?
Who
has
tens
of
thousands
of
coffee
shops?
A
You
might
only
have
one
or
two
cameras
in
each
shop,
but
you
might
still
want
to
have
centralized
monitoring
capabilities
as
well
as
perhaps
having
analytics
in
those
individual
stores,
or
you
might
be
an
airport
or
a
factory
where
you
have
thousands
of
cameras
in
one
site,
and
the
challenge
you
have
is
that
you've
got
both
human
and
machine
viewers.
If
you
want
to
call
them
that,
so
you
could
have
ai
apps
that
are
receiving
the
video
streams
and
doing
things
like
checking.
Are
people
wearing
face?
Masks,
I'm
not
cause,
I'm
speaking,
you
know.
A
Is
there
something
that
looks
like
a
confrontation
kicking
off
in
the
coffee
shop
and
they
might
then
be
streaming
out
using
something
like
mqtt
and
doing
all
that
analytics
at
the
edge
so
that
the
central
site
can
then
see
okay
something's
happening?
I
now
need
to
view
it
again.
Those
could
be
humans
for
them,
viewing
it
at
the
central
site.
A
The
challenge
is
because
you
don't
know
ahead
of
time
how
many
consumers
you
have
for
each
stream.
So
it's
going
to
be
naught
to
many
typically
or
one
to
many,
then
today,
what
very
often
people
do
is
they
use
udp,
multicast,
so
they'll
configure.
If
you
have
one
of
those
little
tsp
cameras,
you
can
configure
it
to
just
multicast
out
the
stream
rather
than
running
rtsp.
A
Is
you
then
end
up
putting
state
into
your
network
for
each
of
those
multicast
streams
and
if
you've
got
tens
of
thousands
of
cameras,
that's
tens
of
thousands
of
star
commodity
or
s
common
g
entries
in
your
network,
and
you
really
don't
want
to
do
that,
whereas
the
proxies
inherently
support
that
fan
out,
so
you
can
send
off
to
one
subscriber
it
might
be
in
that
kind
of
edge
case.
Like
I
said
in
a
coffee
shop,
you've
got
some
kind
of
ai
apps
consuming
it
locally.
A
But
if
that
app
then
detects
that
somebody
isn't
wearing
a
face
mask
or
two
people
look
like
they're
having
an
argument,
then
what
it
might
do
is
send
a
trigger
and
something
at
the
central
site
might
start
viewing
that
and
that
just
gets
replicated
out
through
the
proxy
so
effectively
we're
multicasting
at
the
application
layer.
And
of
course
we
can
do
that.
Really
adding
for
an
item.
A
So
how
are
we
building
it?
Well,
as
I
mentioned
earlier
in
terms
of
architecture
and
footprint,
we
really
wanted
to
minimize
the
footprint
of
this.
So
we
split
what
we're
doing
into.
I
guess
three
main
components:
we
have
a
data
plane
proxy.
That's
there
in
red,
the
rtp
proxy
we'll
come
on
to
how
we
want
to
build
that
the
bits
we've
already
started
building
and
are
all
the
other
bits.
I
guess
we
haven't
we're
literally
about
to
start
on
that
data
plane
proxy.
A
But
of
course
we
want
to
build
this
in
a
framework
where
you
only
have
to
add
that
code
for
that
particular
control,
plane
and
again
really
want
to
have
a
plugable
model
so
that
we
can
provide
that
framework.
So
anyone
can
contribute
that
control
plane
and
then
we
have
a
little
stub
that
runs
in
each
part,
but
that's
much
much
smaller
than
something
like
an
envoy
proxy.
Hence
calling
it
a
stub.
A
A
So
when
I
started
the
project
on
my
own,
I
didn't
know
how
to
do
any
of
this
stuff,
because
I'm
a
network
guy
or
software
guy,
so
I
was
literally
creating
my
own
yaml
files
where
I'd
add
like
init
containers
and
I'd
run
my
sidecar
proxy
that
I
was
using,
then
I'd
run
it
privileged,
and
this
is
just
to
solve
those
sort
of
issues.
So
in
this
case
you
know,
if
you
have
this
custom
key,
that
saying
inject
the
stub,
then
the
stub
will
just
automatically
get
injected.
A
Likewise,
in
terms
of
the
ip
tables
rules
that
are
going
to
direct
traffic
into
the
stub,
then
those
will
get
triggered
off
the
same
rule.
In
that
case,
we
actually
have
a
cni
plug-in
that
runs
on
every
node.
It's
a
chained,
cni
plug-in,
so
you
can
still
run
your
calico
or
psyllium
or
whatever,
as
your
your
main
cni,
and
this
just
gets
chained
into
our
lip
tables
rule.
A
Writing
all
this
in
golang,
using
grpc
to
communicate
between
the
different
components
and
I'd,
say:
we've
got
the
control
plane
up
and
running
now
the
the
stub
I've
mentioned,
calling
it
stub
just
because
it's
pretty
small
it's
about.
I
think
it's
about
500
lines
of
rust
code,
so
pretty
minimal
and
really
the
main
function.
It's
doing
is
terminating
tcp
connections
from
the
app
and
then
just
multiplexing
over
grpc
towards
that
control
plane,
which
will
help
us,
as
we
start
to
do,
control
plane
redundancy,
because
then
we
can
move
things
around.
A
We
deploy
this
as
a
demon
set,
so
we
can
do
east
west,
north
south
all
through
the
same
proxy.
So
anything
that
comes
in
from
outside
we'll
hit
that
we'll
hit
that
proxy.
In
that
case,
we'll
have
a
stub
alongside
to
handle
the
control
plane
for
that
effectively.
It's
because
it's
terminating
the
udp
layer
and
effect
sort
of
terminating
rtp,
we're
not
going
in
and
looking
at
the
clocks
and
those
sort
of
things,
but
we
are
terminating
that
socket.
A
So
that
means
we
can
do
things
like
v4
to
v6,
or
you
know
ip
tunneling
convert
ntus
et
cetera
that
we
kind
of
get
that
for
free,
it'll,
also
support
rtp
over
tcp
or
quick,
and
we
can
support
proxy
at
those
lower
layers
if
we
need
to
a
key
thing.
Security.
Wise,
of
course,
is
this
then
minimizes
our
attack
surface,
because,
if
north-south,
if
the
only
thing
we're
exposing
is,
for
example,
the
rts
people,
then
that's
really
minimized
the
attack
surface.
A
A
A
So
that
feels
like
the
right
way
forward
and,
of
course,
yeah.
We
can
have
multiple,
multiple
application,
pods
all
using
that
same
property
and
of
course
we
can
have
multiple
control
plane
instances
and
we
might
want
to
spawn
a
proxy
for
each
one
or
or
have
a
shared
one.
We
can
figure
that
out,
but
what
I
mentioned
internally,
you
know
it
should
be
a
filter
chain
that
should
be
entirely
reconfigurable
reprogrammable
for
the
user.
A
A
A
A
A
A
So
we
can
see
here
is
we're
streaming
video.
Through
the
system,
we've
got
a
couple
of
different
setups,
one
using
envoy,
which
will
force
everything
through
tcp,
of
course,
one
using
node
ports,
which
in
effect
will
force
everything
through
tcp.
Because
of
this
challenge,
with
dynamic,
dynamic
port
numbers
for
the
native
rtsp
over
udp
one,
I
had
to
use
a
separate
vlc
player
because
the
plug-in
we
had
for
chrome
wouldn't
support
that
for
some
reason.
A
A
Now,
normally
this
is
running
a
very
lightweight
vm.
In
fact,
I
think
you
saw
the
domain
name
come
up
at
the
beginning.
You
can
even
look
at
that
yourself.
If
you
want.
The
only
thing
is
because
it's
lightweight
we,
we
couldn't
put
an
analytics
app
behind
it.
So
normally
you'd
see
the
red
box,
for
I
haven't,
got
a
mask
on
the
green
box
for
has
one
on,
but
that
model
was
so
big.
I
couldn't
run
it
on
this
little
vm.
A
So
you
can
see
those
struggling
with
this,
but
if
you
really
jack
the
loss
up-
and
of
course
this
is
insane-
you
would
never
see
30
packet
loss
normally,
except
perhaps
at
a
conference
like
this.
But
if
we
jack
it
up
to,
I
think
it's
30.
We
try
here
what
you
should
see
yeah.
Sometimes
the
streams
will
just
die
because
of
the
packet
loss.
A
A
A
A
And
I
said
this
was
using
a
sidecar
proxy
that
was
a
a
single
proxy
rather
than
this
decomposed
one
we're
building
now,
but
the
behavior
should
be
the
same,
and
you
can
see
that
you
know
with
the
forward
error
correction.
It's
quite
happy
even
at
30
packet
loss,
as
I
say,
not
something
you
typically
see
real
world,
but
just
as
a
proof
point
that
this
kind
of
approach
can
work.
A
A
I
think
that's
probably
about
it
yeah,
so
just
in
summary,
you
know
the
goal.
The
goal
for
this
project
really
is
for
real-time
media
apps
to
be
first-class
citizens
in
this
cloud-native
world.
We're
building
so
be
able
to
have
the
same
tooling,
use
the
same
pool
of
developers
etc
and
still
do
these
real-time
apps
and
hopefully,
as
I
mentioned
earlier,
drive
the
cost
for
that
down
such
that
you
know.
When
I
watch
football
over
the
internet,
I
won't
have
to
wait
for
the
goals
anymore.
A
You
know
we
have
a
website
for
it.
It's
up
on
github,
some
of
the
repos
are
still
private.
I
think
the
the
stub
and
the
control
plane
we're
about
to
push
out
publicly
over
the
next
few
days,
but
really
it's
just
an
appeal
that
you
know
if
anyone
wants
to
come
and
work
work
with
us
on
this,
you
know
that
would
be
really
really
good.