►
From YouTube: Intro + Deep Dive: Kubernetes IoT Edge WG - Steven Wong, Cindy Xing, Dejan Bosanac, & Kilton Hopkins
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Intro + Deep Dive: Kubernetes IoT Edge WG - Steven Wong, VMware, Cindy Xing, Huawei, Dejan Bosanac, Red Hat, & Kilton Hopkins, Edgeworx
Intro and Deep Dive: Edge Computing challenges and solutions with Kubernetes This session will investigate and catalogue challenges encountered when Kubernetes is deployed in Edge and IoT applications. We'll start by describing two basic approaches: deploying nodes to the Edge with a central control plane; and deploying whole clusters to the Edge. This will be followed by a deep dive into Kubernetes architectural features and constraints in the context of both approaches. We'll see which course makes the most sense for some specific use cases. Next we'll discuss some common challenges to successful deployments, such as resource limits and network availability, and provide some guidance on how to deal with them. There are opportunities to contribute to the evolution of Kubernetes to better serve edge use cases. We will close with details on how you can get involved with the community effort to help this happen.
https://sched.co/MPl4
A
Okay,
hi
everybody
we're
going
to
get
started
here.
This
is
an
extended
session
with
both
an
intro
and
deep
dive
to
running
IOT
and
edge
applications.
Utilizing
kubernetes
we've
got
four
speakers
here
from
different
regions
of
the
world,
I
think
we're
standing
up,
so
you
can
see
who
they
are
but
Cindy.
Why
don't
you
introduce
yourself
just
briefly?
Okay,.
B
My
name
is
Cindy
Shing
I'm,
a
Software
Architect
at
the
future
way,
so
I
helped
a
future
way
to
build
the
public
cloud
infrastructure
specializing
in
IOT
edge
computing,
currently
I'm,
leading
the
open
source
project
called
the
cube
edge
and
also
I'm
co-chairing
with
this
gentleman's
for
the
kubernetes
IOT
edge
work
group.
So
before
future
way,
I
worked
in
Microsoft
for
over
12
years
have
a
lot
of
experiences
in
building
distributed
system.
B
C
So
my
name
is
Dan
I'm,
a
software
engineer
with
Red
Hat
working
in
the
messaging
and
IOT
team
at
let's
get
primarily
working
with
a
within
the
Eclipse
IOT
working
group
on
couple
of
interesting
projects.
One
of
the
most
notable
is
is
the
cloud
scale
messaging
for
the
IOT
also
yeah.
Last
year
we
started
the
the
edge
working
group
in
IOT
edge
working
group
within
the
kubernetes
community
to
try
to
explore
this
topic
a
little
bit
further.
A
D
Home
hi
I'm,
Kim,
Hopkins
I'm,
a
software
engineer
and
architect
and
co-founder
of
n
CEO
of
a
company
called
edge
works,
which
is
an
edge
computing
company
and
I'm.
The
original
designer
of
the
Eclipse
io
fog,
open-source
edge
computing
layer
and
we
Ataturk's
are
the
still
the
core
maintainer
us
of
that,
and
so
that's
part
of
the
the
Eclipse
foundations,
Eclipse
IOT
and
edge
division.
B
So,
let's
talk
about
the
use
cases,
I.
Think
most
of
you
has
the
idea
of
the
benefits
of
edge
computing.
Then,
if
we
look
at
all
the
common
use
cases,
probably
we
can
summarize
in
the
following:
for
example,
you
you,
you
might
have
a
lot
of
IOT
devices
and
at
the
edge
and
then
somewhere
you
want
to
have
a
gateway
to
manage
all
the
devices.
The
benefit
that
you
can
gain
from.
It
is
mostly
the
low
networking
latency
and
then
you
can
remote
remotely
manages
fleet
of
devices.
B
Then
the
other
would
be
like
say,
for
example,
you're
doing
artificial
intelligence
at
the
at
the
edge.
For
example,
you
have
a
machine
manufacturer
and
then
currently
a
lot
of
companies
are
using
OPC
UA
to
do
the
data
collection
using
time
series
DB,
and
then
you
want
to
have
the
data
to
be
locally
analyzed
and
then
upload
some
of
the
data
to
the
remote.
So
in
essence
you
want
to
remotely
manage
your
data
and
also
have
the
data
uploaded
to
the
cloud,
and
then
you
have
aggregated
analysis
further.
B
So,
from
a
requirement
perspective,
we
would
say
you
see
like
I,
want
to
remote
manage
my
notes
or
resources.
I
want
to
remotely
deploy
or
orchestrate
my
applications
and
also
consider
because
I
have
a
fleet
of
edge
nodes.
The
scalability
can
be
huge
and
then
the
first
one
would
be
the
network
connectivity
between
the
edge
node
and
the
cloud
might
not
be
reliable.
That's
the
and
the
fact.
B
B
So,
in
summary,
if
you
look
at
this
architectural
chart,
the
add
node
can
be
categorized
in
two
buckets:
why
we
call
it
infrastructure
edge
for
the
infrastructure
edge,
most
likely
you're
going
to
run
a
whole
cluster
at
that
point,
and
then
your
IT
admin,
our
user,
can
just
talk
to
your
infrastructure
edge
and
then
scheduling
your
application
or
manage
your
resources.
Then
the
other
type
of
edge
required
device
edge.
B
Basically,
it
can
be
a
worker
node,
which
is
part
of
a
bigger
cluster
where
you,
the
controlling
of
your
class
or
can
either
be
in
the
infrastructure
edge
or
it
can
be
at
the
cloud.
So
in
general
you
can
have
a
combination.
Basically,
you
can
have
a
control
plane
in
the
cloud
and
then
your
device
edge
attach
or
you
can
have
a
hierarchical
infrastructure.
Basically,
your
cloud
manager
infrastructure
edge
and
then
your
infrastructure
edge
manage
further
the
device
device
edge.
B
So
in
summary,
as
I
mentioned,
there
are
three
kinds
of
architectural
approaches
we
listed
out
some
reference
architecture.
For
example,
you
are
heard
about
the
Ranger
p3s
in
our
mind.
That's
the
way,
like
you
run
a
whole
cluster
at
the
edge
it's
lightweight,
and
then
you
can
add
as
an
IT
admin,
you
can
your
resources
and
scheduling
your
applications
there.
The
second
one,
the
reference
architecture
is
Cuba,
which
is
the
one
I'm
currently
leading.
So
the
concept
for
Cuba
is
your
control.
Plane
is
at
the
cloud,
but
your
worker
node
is
at
the
edge.
B
So
with
it
you
can
have
a
single
pane
of
glass.
Then
you
can
manage
all
your
resources,
that's
the
second
one!
How
about
the
third
one?
The
third
way
is
the
hierarchical
so
that
it's
similar
to
the
Federation
concept.
Basically,
your
control
plane
is
on
the
cloud
and
then
in
the
middle
you
probably
have
another
layer
of
of
control
plane,
which
is
the
infrastructure
edge.
From
that
perspective,
we
see
virtual
cubelet
is
open
source
project
which
can
meet
this.
B
Need
so
next
I'd
like
to
do
some
comparison
among
all
those
three
architectures
and
see
what
had
the
pros
and
cons
and
what
other
scenarios
are
a
better
fit
for
all
those
three
architectures
then,
before
that,
before
we
drill
down,
let's
look
at
the
current
kubernetes
cluster
in
the
data
center,
so
you
have
multiple
nodes
talking
to
a
master
node
and
then
your
couplet
on
the
under
on
the
node
has
a
long
connection
with
the
worker
node.
So
assuming
the
network
connection
is
very
reliable,
okay
and
then
also
like
for
the
current
kubernetes.
B
The
scalability
support
is
up
to
5,000
nodes,
which
means
the
beveled
is,
to
that
extent,
okay,
and
then
the
third
one
would
be
when
crew
blade
talks
to
the
master.
It
watches
all
the
parts
on
the
controller
and
then
was
the
meta
data
flow
down
to
that
specific
node,
a
figure
out.
What
are
the
parts
bounded
to
this
node
and
then
I
will
do
car
doctor
to
start
the
application.
So
this
is
the
current
datacenter
coronated
solution.
Then,
let's
see
what
about
that's
a
k3s?
B
What's
the
requirement
to
run
Cornelius
cluster
@h,
you
probably
want
to
think
about
at
your
edge.
You
might
only
have
a
single
machine
right,
so
in
that
case
you
don't
want
to
spend
a
lot
of
resources
to
deploy
your
control
plane.
So
we
want
to
have
a
lightweight
control,
plane
and
a
lightweight
agent,
so
I
think
from
that
perspective,
the
case
3s
can
be
a
reference
architecture.
B
B
So
let's
see
about
a
cool
badge
cooperage
that
as
I
mentioned,
it
has
the
cloud
part
and
then
the
agent
part
for
the
cloud
part.
Besides
the
current
kubernetes
components,
it
added
two
components:
wise
card
controller,
the
other
is
a
cloud
hub
and
then
on
the
agent
side
there
are
a
bunch
of
components
just
to
I
want
to
mention
for
the
cube
edge
agent
at
runtime.
The
memory
footprint
is
only
10
Meg,
so
the
benefit
of
the
qubit
is.
B
You
can
have
a
cluster
with
worker
node
in
the
data
center
and
also
you
can
have
worker
node
on
the
edge.
But
on
the
cloud
you
basically
have
a
single
plain
glass
of
pain
to
manage
your
node
on
the
cloud,
as
well
as
the
node
on
edge
without
knowing
where
those
worker
node
is
located,
and
then
you
also
benefit
from
from
the
lightweight
agent.
And
then
the
third
thing
I
want
to
mention
is
cube.
Edge,
dresses
the
network
at
this
connectivity
issues.
B
So
if
you're
interested
in
next,
whenever
I
have
a
deep
dive
session
talking
about
more
in
detail
about
Cuba,
so
the
third
approach
will
be
the
hierarchical,
so
hierarchical,
meaning
you
have
a
control
plane
on
the
cloud
and
then
you
have
an
infrastructure
edge
which
is
a
cluster
at
the
edge,
and
then
your
infrastructure
at
node
managed
further
a
lot
of
worker
nodes.
So
then,
let's
see
you
virtual
couplet
can
be
referenced
architecture
because
behind
the
virtual
couplet
you
can
have
a
cluster,
for
example,
in
the
data
center.
It
can
be
assured.
B
Assure
you
assure
cluster
or
forget
it
like
AWS,
but
for
the
for
the
hierarchical
IOT
edge
scenario,
it
can
be
an
actual
outer
edge.
So
that's
another
architectural
solution,
so
in
summary,
I
have
a
table
listed
here
when
to
use
and
where
to
use.
So
if
you
want
to
have
a
single
class
class
of
pain
and
really
lightweight
of
control
plane,
then
you
can
think
of
cube
edge
where
you
have
your
control
plane
in
the
cloud
and
your
worker
node
at
the
edge.
B
But
on
the
other
hand,
if
you
want
to
have
strong
autonomy
of
resource
scheduling,
you
can
consider
case
3s,
but
the
the
thing
challenge
for
running
a
cluster
at
the
edge
is
you
don't
have
like
a
central
management
of
all
those
clusters
that
come
to
the
third
option
is
like.
If
you
have
multiple
clusters
right
at
the
edge,
then
you
would
need
to
think
about
a
hierarchical
solution.
So
basically
that's
what
I
have.
A
A
Your
resources
at
the
edge
locations
are
typically
constrained
by
power.
Physical
size
cost
and
they're,
really
not
necessarily
comparable
to
what
you've
got
at
a
public
clouds
or
a
large
on-prem
data
center.
You've
also
got
the
network
limitations.
The
challenge
of
completely
unattended
operation
at
these
remote
locations,
as
well
as
physical
security
and
we're
gonna,
get
into
these
things
in
detail
later
in
the
presentation
today
with
regard
to
network
some
of
these
challenges,
don't
necessarily
have
solutions
or
certainly
not
solutions
within
kubernetes
itself.
A
They
might
involve
using
external
tools,
and
some
of
them
are
more
like
situations
involving
triage
where,
if
the,
if
you're
just
going
to
have
guaranteed
periodic
network
outages,
you
want
to
have
ways
to
live
with
them,
we'll
get
into
a
deep
dive
of
what
you
can
do
about
it.
First
of
all,
we'll
talk
about
resources.
A
A
A
Your
workloads
may
have
a
wide
range
of
priorities
and
there's
a
need,
whether
you
appreciate
it
or
not
at
first
for
doing
triage
on
these.
Likewise,
your
network
capacity,
limited
variable,
subject
to
outage
and
I'm
gonna,
give
it
go
in
now
to
some
examples
of
what
to
do,
or
in
this
case,
what
not
to
do
yeah
if
you're
rookie
to
kubernetes
you
often
go
to
these
workshops,
see
these
examples
that
show
you
how
simple
it
is
to
deploy
a
containerized
app,
a
pod
and
they'll.
A
Often
the
fact
is
when
you
go
into
a
pod
specification,
or
in
this
case,
a
deployment.
There
are
a
lot
of
things
that
are
technically
optional
and
you
can
leave
out
and
when
you
go
to
that
intro
session,
to
make
this
thing
quick
and
easy,
they
will
leave
it
out.
In
this
example,
I
hope
you
could
read
that
in
the
back,
but
there's
a
resources
space
in
this
pod
spec
and
things
work
perfectly
well
when
you
leave
that
out
the
what
goes
there.
A
If
you
do,
if
you
choose
to
fill
out,
the
resources
section
is
a
declaration
of
how
many
CPU
cores
you
would
expect
to
use
how
much
memory
you'd
expect
to
use-
and
you
know
if
you
choose
to
leave
this
out
after
your
pod
gets
deployed.
Your
pod
could
experience
anything
from
resource
starvation,
one
pod
negatively
affecting
another,
basically
being
a
noisy
neighbor
on
whatever
worker
node
this
got
deployed
on
and
when
this
happens,
when
resources
run
out,
let's
suppose
you
get
an
out
of
memory
could
condition
if
you
don't
declare
any
usage
or
any
priorities.
A
This
is
completely
random.
Behavior
who's
going
to
be
killed
first,
if
every
workload
you're
running
is
like
this,
it's
not
it's
completely
unpredictable.
Is
this
likely
to
be
okay,
I'm,
guessing
no
I'm,
guessing
that
some
of
the
things
you're
running
at
the
edge
are
more
important
than
others
and
you
might
have
distinct
preferences
for
what
might
happen?
A
So
what
can
we
do
about
this?
Well,
first,
let's
look
at
where
all
these
pieces
are.
It
actually
is
fairly
complex
in
the
moving
parts,
and
this
is
the
simplified
view,
but
you
start
with
your
pod
specification
done
by
the
developer.
Kubernetes
puts
the
pods
that
are
ready
to
be
run
or
deployed
into
a
scheduling,
funnel,
there's
an
admission
control
process.
Potentially,
if
you,
if
you
manage
to
declare
it
with
priority,
where
some
jobs
in
the
queue
are
able
to
leapfrog
ahead
of
others.
A
But
if
you
once
again,
if
you
don't
indicate
the
preferences,
the
kubernetes
scheduler
has
no
idea
that
you
might
have
these
and
it
is
just
first-come
first-serve.
So
this
goes
to
scheduling
which
determines
what
not-it's
run
it
runs
on
and
it
takes
into
account
the
resources
on
the
nodes.
You've
got
it
might
be
one
in
edge,
but
it
could
be
more
and
if
you
declared
what
you
need,
it
can
do
the
right
thing
with
regard
to
placement.
A
Finally,
once
this
thing
is
running,
the
cubelet
is
responsible
for
actual
enforcement.
In
other
words,
I
declared
I
need
X
amount
of
memory,
but
I
went
way
over
there
and
we're
out
of
memory,
and
somebody
needs
to
be
clubbed
on
the
head
and
taken
out.
This
is
a
deeper
dive.
It's
really
the
same
things
going
on
in
that
first
aerial
view,
but
it
goes
into
greater
detail
of
where
some
of
this
takes
place.
The
pod
manifest
admission
control.
A
Moving
on
to
the
scheduler,
the
cubelet,
the
container
runtime
gets
a
bite
at
this
process
and
finally,
some
of
these
enforcement
mechanisms
happen
inside
the
Linux
kernel.
In
some
of
these
cases,
like
out
of
memory
resource,
there
are
actually
two
places
that
potentially
enforce
things,
and
let
me
go
back
here.
I
want
to
make
sure
I
point
out
that
this
is
a
session
covering
the
broad
subject
of
IOT
and
edge
on
kubernetes.
But
there
was
a
really
great
presentation.
A
year
ago,
at
cube
Konig
in
Copenhagen
there's
a
link.
A
A
If
you
have
no
specification,
there
is
a
mechanism
to
have
a
global
default,
and
if
you
don't
specify
that,
then
your
pod
priority
for
everything
is
none
is
zero,
which
means
every
everything
is
the
same,
which
means
it's
random
and
you
really
shouldn't
do
that.
You
really
want
to
take
the
effort
to
give
kubernetes
a
clue
as
to
what
you
want
to
have
happen.
Orthogonal
to
this
there's
something
called
class
of
service
and
every
workload
is
going
to
fall
into
one
of
three
classes
of
service,
guaranteed,
burstable
or
best
efforts.
A
But
if
you
don't,
some
of
them
have
fully
specified
things.
They
actually
will
get
a
higher
class
of
service
and
they'll
get
preferential
treatment.
If
you
only
specify
part
of
these
specs
say
you
have
a
pod
and
I
gave
specs
for
one
container
in
the
pod,
but
two
more
containers
are
sitting
there
with
none
of
these
specified.
You
fall
into
the
burstable
category,
leave
everything
else,
all
everything
off
and
you
get
best
efforts
and
these
determine
which
pod
gets
killed.
A
First,
when
you're
out
of
resource
understand
that
kubernetes
itself
is
monitoring
this,
but
on
a
very
slow
control,
loop.
Measured
in
minutes,
typically,
the
Linux
kernel
is
there
and
if
kubernetes
is
too
slow
and
the
Linux
kernel
determines
that
the
Linux
OS
is
out
of
memory,
it
can
independently
kill
something.
It
has
no
knowledge
of
these
classes
of
services
so
that
results
in
the
random
behavior
network
capacity
is
similar
to
C.
You
know
the
resources
I
talked
about
a
minute
ago.
A
The
covered
allocations
of
CPU
and
memory,
but
network
is
another
critical
resource
and
it
can
be
limited.
It
can
be.
You
see
the
speed
test
here
asymmetrical
where
the
opera's
down
isn't
necessarily
uniform
and
different
workloads
are
likely
to
have
different
priorities
and
behavior,
and
this
means
that
you
should
have
different
policies.
A
Just
like
the
resources.
It
is
entirely
possible,
but
probably
not
smart,
to
ignore
your
option
of
defining
network
policies
and
just
go
out
there
and
get
random
behavior,
but
with
kubernetes
you
have
the
option
of
creating
these
resources
called
network
policies,
and
they
deal
with
things
like
what
traffic
is
allowed
by
default.
A
Every
pod
is
going
to
be
able
to
communicate
with
every
other
pod,
but
that
more
than
likely
is
kind
of
a
looser
security
policy
than
you'd
like
to
have
can
potentially
when
things
go
wrong,
produce,
noisy
neighbor
problems
and
just
be
kind
of
a
bad
day.
I
will
caution
you
that
kubernetes.
If
you
didn't
already
know
this,
the
networking
layer
is
implemented
as
a
plugin.
A
So
some
of
these
examples
I'm
about
to
give
it
some
of
these
plugins
may
not
have
all
the
options
and
others
are
very
rich
and
maybe
even
have
sidebar
options
for
implementing
network
security
policies
and
observability,
but
I'm
just
going
to
give
you
one
example
to
point
out
that
it's
important
to
do
this.
So
the
when
you
define
a
policy
that
what's
going
to
go
on,
is
based
on
source
and
destination,
IP
addresses
and
ports
and
there's
an
example
here,
defining
limits
on
the
ingress
and
you
can
do
egress
as
well.
A
If
you
have
the
plug-in
that
would
support
this
kind
of
option,
and
probably
if
you're
doing
edge
and
IOT,
you
need
to
be
on
one
of
these
network
layers
that
give
you
all
of
these
options
once
you've
made
this.
These
declarations
understand
that
the
actual
enforcement
of
your
preferences
is
going
to
be
taking
place
at
multiple
levels
that
it
starts
with
the
pod
bandwidth,
annotations
and
the
prior
slide
covered
restricting
who
gets
to
talk
to
who.
But
there
is
something
called
a
bandwidth
plug-in
that
may
be
available.
A
If
you
choose
the
nut
write
network
layer
that
can,
in
addition
to
specifying
who's
allowed,
to
talk
to
what
you
get
to
allocate
bandwidth
caps
on
egress
and
ingress.
This
is
implemented
by
the
traffic
control
system
in
the
Linux
operating
system
and
under
that
the
Linux
network
namespace
comes
into
play.
This
is
an
example
of
what
you
can
and
should
do
out
of
pod
spec,
where
in
this
example,
I've
called
out
a
spec
for
ingress
bandwidth
and
egress
bandwidth.
A
A
So
if
you
like,
go
step
out
into
the
hall
go
but
come
back
in
five
minutes,
maybe
we're
gonna
resume
this,
but
feel
free
to
chat
with
your
peers
for
the
next
five
minutes
for
and
separate
yourself
by
these
use
cases,
the
use
cases
are
remote
office
and
retail.
So
this
would
be
people
at
at
cube
con
Seattle
an
example
of
this
remote
office.
Retail
would
be
the
chick-fil-a
sandwich
chain
had
a
proof
of
concept.
They
talked
about
of
standing
up
a
3-node
kubernetes
in
every
fast
food
outlet.
A
I
think
something
like
branch,
bank
branch
offices
or
retail
stores
would
fall
into
that
use
case.
The
second
use
case
sensor:
data
collection
analytics
that
would
be
grabbing
sensors,
perhaps
doing
some
filtering
of
them
before
they
get
published
up
to
a
public
cloud.
I'd
also
declare
that
things
like
video
data
collection
with
a
little
processing,
we'll
call
it
that
use
case
physical
device
control
would
be
industrial
automation
where
you've
got
sensors
that
have
running
tight
control
loops
that
need
low
latency.
If
you're
in
that
look
who's
ever
holding
up
that
sign.
A
There
are
people
doing
gaming
out
at
the
edge
where,
for
best
experience
in
the
game,
they're
trying
to
run
workloads
closer
to
the
actual
participants
in
the
game
and
then
finally
telco
edge
cloud
where
there
are
telcos
that
are
interested
potentially
in
using
kubernetes
to
manage
things
running
in
cell
towers
or
legacy
landline
switching
offices,
and
that
would
be
that
use
case
if
you're
in
the
other
I
don't
know,
come
up
and
sit
in
the
front
row.
So,
let's
Ricci
and
5-minute
break
so
I've
got
2:30
so
at
2:35
we'll
start
talking
about
data
plane.
A
C
So,
at
the
beginning
of
the
presentation
we
had
that
we
said
that
we
usually
have
like
a
three
common
challenges
when
it
comes
to
to
the
edge
computing
right,
so
infrastructure
challenges.
How
do
we
put
our
computer
resources
at
the
edge
sites?
Then
we
talked
about
a
lot
about
the
control
plane
challenges.
So
how
do
we
then
scheduler
workloads
do
to
these
to
these
compute
resources,
but
something
I
want
to
talk
a
little
bit
now
is:
is
the
data
paint
challenges
so
once
our
our
workloads
are
successfully
deployed?
C
You
know
you
know
something
like
a
very
distributed
infrastructure
like
this.
How
do
we
communicate
between
between
these
services
and
how
these
services
communicate
with
the
central
clouds,
how
they
can
possibly
communicate
between
themself,
and
it's
on?
The
first
look
looks
like
a
very
complex
problem
and
in
the
first
thing
instincts
to
everybody
is
to
try
to
solve
this.
This
on
the
infrastructure
level.
C
On
the
you
know,
l2
l3
levels,
trying
to
put
a
lot
of
VPNs
everywhere,
try
to
use
DNS
to
resolve,
resolve
services
and
do
this
discovery,
but
you
know
in
a
fairly
complex
network
eat
it.
It
quickly
becomes
very,
very
challenging,
so
that's
one
of
the
solution
for
that
is,
is
to
go
up.
Go
on
on
the
application
level,
use
some
tools
that
that
have
been
around
for
a
while
and
can
can
can
be.
It
can
be
done
to
solve
some
of
these
problems.
C
One
of
those
tools,
so
just
one
of
the
solutions
is
mqp
1.0
protocol,
which
is
basically
a
messaging
protocol,
which
supports
a
lot
of
features.
So
first
thing
is
to
say
is
that
AMQP
is
very
loaded
term.
It
has
a
lot
of
history
and
it
changed
over
time.
So
you
know
once
people
first
hear
about
mqp,
the
first
Association
is
rabbitmq
and
the
brokers,
but
mqp
1.0
is,
is
a
completely
different
kind
of
protocol.
I
know
that
it
should
have
been
called
differently,
but
this
is
where
we
are
today.
C
One
of
the
the
major
features
of
amqp
1.0,
which
makes
it
a
good
fit
for
solving
some
of
these
challenges,
is
that
it's
a
pure
peer-to-peer
protocol,
meaning
that's
it's
it.
Doesn't
it
doesn't
assume
the
desert
does
look
like
a
client-server
communication
or
order
or
a
broker
in
between.
Of
course
you
can
implement,
but
you
can
implement
all
kind
of
these
different
communication
patterns
like
like
multicast
or
or
or
fan
out
or
or
request
response.
So
just
to
give
you
a
quick
idea
of
these
different
kind
of
communication
patterns
is.
C
This
is
what
you
usually
call
direct
messaging
right,
so
we
have
two
endpoints.
One
is
the
producer
of
the
message.
One
is
the
consumer
of
the
message.
Producers
sent
the
message
directly
to
the
client.
The
client
accepts
the
message
and,
and
that's
delivery
is
the
Islam
right
so
think
about.
It
is
like
HTTP,
where
you
have
like
a
request:
direct
response.
That's
like
a
direct
communication
right,
but
when
you
talk
to
people
about
messaging,
everybody
assumes
a
message
broker
message
to
queue
in
in
between-
and
this
is.
C
So
first,
the
producer
sends
the
message
to
the
broker
broker
owns
that
message
at
some
point
and
then
the
producer
is
done
right
and
it's
the
job
of
the
broker
to
to
send
that
message
to
one
or
multiple
consumers,
depending
on
the
Monda
semantics
of
that
particular
communication
that
that's
going
on
right,
but
because
of
the
the
symmetric
symmetric
property
of
the
nqp,
we
we
were
being
able.
We
will.
C
We
were
able
to
develop
a
new
component,
which
is
which
we
call
the
the
message
router
and-
and
if
you
think
about
message-
router,
it's
it's
think
about
IP
router
right
for
the
IP
addresses,
but
only
on
the
AMQP
level
on
the
level
of
the
messages
and
what
that
means
in
particular.
So
what's
the
difference
between
the
the
broker,
storing
forward
messaging
or
the
direct
messaging
with
the
router?
C
Is
that
in
this
case,
router
never
owns
this
the
message
it
just
routes,
the
necessary
packets,
to
be
able
to
achieve
like
pops
up
or
point-to-point
communication
between
the
different
endpoints
that
are
communicated
now
connected
to
the
single
router.
All
this
is
currently
implemented
in
in
couple
of
components.
Upstream,
we
have
Apache
ActiveMQ
Artemis,
which
is
which
is
in
QP
1.0
message
broker,
not
just
what
in
Kipp
1.0,
but
it
serves
that
purpose
very
well.
C
We
have
a
apache
QP
dispatch
router,
which
we'll
see
in
action
a
little
bit
later,
which
implements
this
routing
component
in
the
system,
and
we
have
a
apache
cupid
proton
project,
which
is
basically
umbrella
project
for
all
kind
of
AMQP
clients.
You
may
wish
to
connect
to
this
kind
of
network,
so
this
is
just
the
basic
components
in
basic
theory
behind
it.
C
But
now
things
become
a
little
bit
more
interesting,
because
if
you
go
back
to
the
like
a
h
kind
of
deployment
section,
the
the
nice
thing
about
the
routers
is
that
they
can
form
the
very
complex
networks
themselves.
So
the
router,
the
routers
themselves,
have
ability
of
out
out
to
discovering
the
other
other
routers
and
they
can
provide
a
provider.
A
complex
network
over
which
we
can
send
messages
in
a
very
very
you
know,
distributed
geographical
area,
for
example.
C
So
there's
a
lot
of
different
use
cases,
one
that
I
always
like
to
point
out,
which
is
which
is
very
important
for
if
we
go
back
to
the
origin
of
these
stories,
so
how
are
we
gonna
solve
all
these
edge
challenges?
So
if
you
take
a
look
look
at
these
pictures,
so
this
is
how
we
can,
for
example,
enable
edge
to
edge
communication
without
with
two
things
to
notice.
First
thing
you
can
see
here
that
edge
nodes
here
doesn't
have
any
incoming
traffic
the
incoming
TCP
connections
here.
C
So
the
the
points
of
this
arrows
shows
that
the
routers
that
are
hosted
on
the
edge
sites
are
connected
to
TCP
connections
to
the
router
in
the
cloud
right.
But
once
these
connections
is
it
established
on
the
TCP
level,
the
full-blown
duplex
AMQP
connection
entropy
flow
can
go
to
these
connections
on
that
layer
right.
C
So
the
service
a
simply
needs
to
communicate
to
the
service,
see
on
the
on
the
gamecube
level,
addressing
saying
like
service
II
wanted
to
consume
messages
from
the
addressee
and
and
service,
so
that
information
will
be
forwarded
to
to
the
routers
network
and
in
the
router.
Here
we
remove
that,
and
it
will
note
that,
for
the
for
the
message
sent
on
the
other
C,
it
needs
to
send
that
message
for
us
to
router
B
in
the
router.
B
will
note
how
to
how
to
forward
that
message
further
round
through
the
through
the
network.
C
Right
and
with
this
kind
of
architecture.
We
solve
two
problems,
so
first,
there's
no
need
for
for
for
a
VPN
to
get
to
this,
and
the
second
one
is
that
is
that
all
the
dressing
is
done
on
purely
on
the
own
on
the
l7
address
space.
So
there's
no
need
to
do
any
kind
of
any
kind
of
dns
resolutions
in
the
in
the
process.
I'll
just
go
quickly
because
I
am
assume
using
a
lot
of
times.
So
with
this
we
can
then
do
like
a
point-to-point
flows
to
the
network
completely
independently.
C
We
can
implement
a
very
efficient
multicast
to
the
system,
sending
one
message
to
a
lot
of
consumers
through
the
system
and
and
also
these
routers
implement,
implement
the
the
best
path,
optimization
routes.
So
they
will
try
to
find
the
optimal
part
of
the
message
to
flow
through
the
system.
But
if
something
goes
wrong,
the
the
alternate
path
will
be
available
there
and
when
the
original
or
original
connection
comes
back,
we'll
have
will
have
everything
everything
back
to
normal.
So
right
now,
I
would
like
to
demo
this
a
little
bit.
E
C
What
I
want
to
show
here
is
a
little
demo
that
we,
so
we
have
of
this
for
first
tops,
is
basically
for
physically
different
OpenShift
clusters
right,
so
this
first
one
is
easy
in
the
private
server.
The
other
one
is
is
the
AWS
instance,
which
is
you
instance,
hosting
the
the
operative
cluster,
and
these
two
hour
represents
are
my
two
clouds,
which
are
interconnected,
we'll
see
a
little
bit
later
and
then
then
we
have
two
more
two
more
clusters.
C
This
money
is
behind
the
corporate
firewall
firewall,
and
this
one
is
actually
running
on
this
laptop
on
the
on
the
conference
Wi-Fi
in
the
mini
shift,
and
these
two
are
representing
my
two
edges,
so
I
must
say,
like
I
use,
different
clusters,
just
to
show
that
you
know
this
work
in
this
environment
completely
physically
different
clusters,
but
it
it
can
be
used
on
the
devices
edge
as
well.
So
it
doesn't
doesn't
need
to
be
the
whole
cluster
to
run
the
router
right.
C
You
can
run
it
on
on
a
single
node
on
different
nodes,
so
this
is
just
to
show
things
and
many.
This
is
how
the
dispatcher
outer
console
looks
like
so
this
little
white
thingie
here
is
actually
our
console
connected
to
the
to
the
cloud
cloud.
One
one
one
of
the
nodes
in
will
call
one
how
we
set
up
set
up.
C
So
first
taking
point
here
is
that
you
can
see
how
routers
can
discover
themselves,
and
even
if
this
console
is
connected
only
to
us
a
single
single
cloud,
we
can
visualize
the
whole
topology
of
the
whole
network
of
the
routers
going
on
here.
So
next
we
have
this.
Yellow
things
are
the
services
I
have
two
very
simple
types
of
services
here
so
once
and
we
will
demonstrate
like
our
PC
traffic,
so
we're
trying
to
do.
You
know
a
command
and
risky,
and
you
see
our
control.
So
we
have
really
two
simple
services.
C
One
is
called
upper
and
other
is
called
a
reverse
right,
so
one
will
receive
the
text
uppercase.
The
text
and
other
will
reverse
the
text.
Okay,
so
that's
that's
as
easy
as
it
gets
and
we
gave
the
colors
to
each
of
these.
These
instances
of
the
service
is
running
everywhere,
just
to
be
able
to
visualize
physically,
where
the
responses
are
coming
from.
So
that's
that's.
The
whole
idea-
and
this
little
guy
here
is-
is
the
client.
It's
basically
one
one
single
page
here
which
we're
gonna
use
to
to
demonstrate
these
calls.
C
So
there's
the
address
call
all
here
and
if
I
send
a
message
to
all
I
should
receive
the
response
from
all
the
all
the
services
running
all
over
the
place
right
so
interesting
image
and
this.
So
this
is
I'm
calling
this
from
the
from
the
I
think
this
client
is
actually
connected
to
the
AWS
instance
of
the
openshift
and
it
called
the
service
running
on
the
mini
shift
on
my
laptop
over
the
over
the
conference
Wi-Fi
without
any
VPNs
or
firewalls
or
DNS.
C
C
Define
yeah,
we
can
do
point-to-point,
which
message
will
be
sent
to
a
single
instance
of
a
single
service,
and
it
will
try
that
the
closest
service
to
the
caller
to
instantiate
it.
So
there's
a
lot
of
things
more.
We
can
do
with
this,
but
I,
don't
know
how
I'm
on
time.
At
this
point
two
minutes
nice
so
I
have
a.
Let
me
clear
this.
C
C
So,
if
I
generate
a
very
low
load
here,
what
I
want
to
show
you
is
that
everything
all
all
the
load
will
be
handled
by
the
by
the
service
running
on
that
particular
node
as
well,
so
no
traffic,
we
believe
the
single
node,
so
we
will
handle
all
the
all
the
requests
locally.
So
let's
hope
it
will
work.
So,
let's
me
start
and
then.
C
We
see
that
all
the
all
the
requests
are
handled
by
the
read,
in
instance,
of
the
service,
which
is
exactly
what
we
wanted
that
the
instance
running
on
on
the
same
node,
where
the
on
the
same
cluster,
where,
where
the
where
the
logs
and
service
is
running
right,
so
you
need
to
stop
this
now.
Okay,
bear
with
me
for
a
second
stop
and
I
will
I
will
instruct
this
service
to
start
to
to
generate
much
more
load
here
now.
C
We
can
see
that,
with
this
particular
configuration,
it
said
that
that
the
red
service
will
only
get
a
part
of
the
load
and
then
the
rest
of
the
Lord
will
be
offloaded
to
the
other
sites
to
do
the
services
being
located
or
on
the
other,
on
the
other,
basically
plasters.
So
that's
that's.
What
I
wanted
to
show
you
today
regarding
regarding
this?
So
maybe
now
we
can
go
back
to
the
presentation
if
I
can
find
the
PowerPoint
yeah.
C
With
the
with
the
operator
for
the
QP,
Dispatch
router
and
there's
a
so,
this
demo
is
in
the
rough
shape,
still
right,
we're
working
on
it.
So
you
can
find
the
source
here
and
you
can
play
right
around.
But
what
certainly
we
will
do
in
the
future?
Try
to
document
it
more
and
and
try
to
present
it
more
so
that
more
people
can
play
with
it
and
instead
of
writing
these
simple
services
try
to
put
it
in
in
the
content
in
the
proper
context
of
the
of
the
edge
computing,
so
yeah.
D
Okay,
great
so
I
can't
express
enough
affection
for
that
protocol.
Amqp.
That
is
fantastic
stuff
so
and
by
the
way,
if
anyone
wants
to
continue
the
group
discussions
that
we
were
having
on
the
break
there
cuz,
it
seemed
pretty
lively.
We,
after
the
this
session,
were
all
up
for
being
available
outside
of
the
the
conference
room
to
take
those
discussions.
So
this
is
a
topic
that
would
would
normally
be
you
know
a
talk
in
itself
and
so
I'm
going
to
do
my
best
to
to
bring
it
together
in
the
final
20
minutes.
D
There's
a
joke
that
I
like
to
tell
how
many
of
you
have
heard
this
before
the
s
and
IOT
stands
for
security.
Anybody
have,
and
if
you,
if
that's
new
to
you,
feel
free
to
reuse
it
it's
it's
it's
true!
It's
it's
true,
though,
so
the
gist
of
security
at
the
edge
is
that
it's
a
completely
separate
environment.
It.
It
can't
be
added
as
an
afterthought.
So
if
security
is
normally
the
sort
of
thing
that
you
put
your
system
through
as
a
final
check
and
then
penetration
test
or
whatever
and
say,
ok,
it
works.
D
Now,
let's
get
the
security
in
place
and
launch
it.
This
at
the
edge
is
going
to
kill
you
and
that's
because
the
edge
environment
is
is
very
different,
so
you
want
to
be
designing
security
into
the
edge
infrastructure
from
the
start,
and
so
the
best
way
to
do
that
is
in
this
kubernetes
IOT
edge
working
group.
We
have
begun
to
assemble
and
are
actually
nearing
completion
of
a
white
paper
that
exposes
the
challenges
of
security
at
the
edge.
D
So
there
are
some
key
differences
about
the
edge
environment
that
makes
security
a
particular
challenge.
We
just
saw
a
example
of
diverse
networks.
Some
things
are
behind
firewall.
Some
things
are
over
VPN
VPNs,
not
a
good
idea
by
the
way,
usually
at
the
edge,
but
some
things
are
on
cellular.
Some
things
are
through.
D
You
know
three
different
NAT
layers
because
they're
inside
a
an
advanced
manufacturing
facility,
and
you
have
to
try
to
bring
all
of
those
things
together,
but
still
in
some
way
represent
the
identities
of
the
things
in
a
way
that
allows
you
to
do.
You
know,
dataflow
control
and
so
on.
Also
there's
no
guarantees
of
continuous
power
so
out
at
the
edge.
D
You
might
have
things
that
are
running
on
battery
or
solar
charge
battery
and
when
things
go
offline,
the
perfect
opportunity
to
hack
is
then
because,
as
it
comes
back
up
on
things,
look
strange
anyway
and
you
might
even
be
fed
some
initial
commands.
You
know
to
allow
you
back
on
a
network,
that's
a
great
time
to
be
present
and
trying
to
get
access
to
a
system,
and
you
have
intermittent
connectivity.
So
if
you
can
guarantee
that
everything's
flowing
all
the
time
over
a
high-speed
land,
then
you
can
maybe
watch
the
traffic,
but
what?
D
If
the
traffic
comes
in,
bursts
or
is
offline
for
two
days
in
a
row
because
of
a
of
a
cell
tower
outage
or
there's
a
dust
storm
and
so
on?
You
have
to
take
these
things
into
account
and
at
the
edge
you
have
direct
physical
access
to
Hardware.
So
how
many
of
you
believe
that
if
you
wanted
to
go
and
attach
a
keyboard
and
a
mouse
to
someone's
cloud
server
that
you
would
be
able
to
find
where
it
is
and
get
in
and
do
this
anybody
feel
like
you
are
up
for
this
challenge?
D
It's
not
gonna
happen.
How
many
of
you
feel
that
you
could
go
and
put
a
ladder
on
a
light
pole
and
access
a
camera?
That's
part
of
a
smart
city
deploy
anybody
figure.
You
could
do
that
if
there's
no
police
around
yeah,
it's
a
completely
different
environment.
We
have
to
take
this
into
account
the
hardware
at
the
edges
heterogeneous.
So
it's
really
easy
to
buy.
You
know
hundreds
or
thousands
of
servers
to
go
in
your
data
center,
all
from
the
same
supplier,
all
with
the
same
spec,
but
at
the
edge
you
have
mix
and
match.
D
You
have
different
machines.
You
have
boards
that
have
been
in
place
for
dozens
of
years
and
you
would
like
to
access
those
boards
over
serial
port
with
some
new
IOT
gateway
that
IOT
gateway
that
you
bought
is
different
than
the
one
was
bought
two
years
ago
and
so
on
and
so
forth,
and
we
have
a
lot
of
non
tcp/ip
communication.
Anybody
ever
program
with
Bluetooth.
Does
anybody
ever
done
any
Bluetooth
development?
D
You
know
Bluetooth
is
non
tcp/ip,
Nats
short-range
anybody
use
Lauro
an
for
IOT,
that's
good
stuff,
great
long
range,
decent
bit
rate,
not
tcp/ip,
and
so
then
you'll
also
have
multiple
vendors
they're
very
different
than
you
know,
racking
out
your
data
center
and
you'll
need
to
handle
security
in
offline
mode.
So
a
lot
of
times
we
take
a
look
at.
We
have
some
kind
of
advanced
AI
based
security
traffic
during
in
the
cloud.
D
Let's
take
a
look
at
what's
coming
into
our
data
center
and
we
start
to
sniff
out
things
that
don't
quite
look
right,
but
at
the
edge
you
actually
need
to
be
able
to
handle
security
down
on
single
or
multiple
node
level
off
off
of
the
the
cloud
connection
and
at
the
edge
you
have
very
low
latency
locally.
This
is
one
of
the
advantages
of
the
edge,
but
you
have
a
higher
latency
to
the
cloud
and
back,
and
so
you
have
to
take
into
consideration
that
you
might
not
get
answers
as
fast
as
you
want.
D
So
there's
a
kind
of
a
summary
or
an
overview
of
the
types
of
security
challenges
at
the
edge
not
said
yet
to
be
complete,
but
complete
enough
for
us
to
be
talking
about
it,
and
so
the
gist
of
it
is
you
have
to
trust
the
edge
hardware.
This
is
the
hardware.
That's
running
your
edge
computing.
You
have
to
be
able
to
trust
the
devices
connected
to
that
edge
hardware.
D
You
have
to
find
some
way
to
deal
with
operating
systems
at
the
edge
and
the
fact
that
they're
complex
beasts-
you
have
network
concerns
and
then
of
course
now
say
that
all
of
those
things
are
in
place:
edge
micro
services
themselves,
post
some
new
challenges,
we're
going
to
talk
about
how
to
or
some
of
the
things
that
we've
exposed
around
edge
maker
services,
so
starting
with
trusting
the
edge
hardware.
So
I
already
mentioned
that
physical
security
is
not
guaranteed
at
the
edge.
D
In
fact,
I
would
say:
physical
security
is
actually
very
difficult
at
the
edge
and
there
have
already
been
plenty
of
hack
attacks.
I've
suffered
one
myself
as
an
IOT
advisor
to
the
city
of
San
Francisco,
and
there
were
some
incidents
that
that
occurred
a
number
of
years
ago
really
opened
my
eyes.
You
have
people
going
out
with
screwdrivers
and
wrenches
and
saying
I
think
I
can
get
that
get
at
that
box.
What
do
I
think
I
could
do
with
it
if
I
got
it.
D
So
the
hardware
root
of
trust
is
a
starting
point
for
a
lot
of
people.
You
know
a
TPM
2.0
module
is
kind
of
the
de-facto
hardware
root
of
trust
solution.
If
I've
got
that,
then
everything
that's
riding
on
top
of
that
hardware
is
said
to
be
good.
That's
not
actually
true,
because
hardware,
root
of
trust
is
just
a
starting
point.
This
basically
tells
you
that
the
board
that
you're
operating
on
is
the
board.
D
You
think
it
is
and
looks
to
be
still
in
the
same
condition
that
you
last
trusted
it
in,
but
building
on
top
of
that,
that's
like
that's
like
the
the
basement
foundation
of
a
house
right
what's
built
on
top
of
that,
still
needs
to
be
good
quality,
but
you
do
have
a
that
foundation.
So
what
about
the
condition
of
the
hardware?
So,
if
I
have
some
orchestration,
that's
happening
with
the
clips,
IO
fog-
and
it
says
within
these
GPS
coordinates
I
would
allow
the
following
micro-services.
D
So
what
do
you
do
if
your
edge
Hardware
node,
so
an
IOT
gateway?
All
of
a
sudden
has
a
USB
flash
drive
attached
to
it?
That's
usually
a
warning
sign,
and
if
it
was
your
laptop,
you
would
be
very
concerned
because
who's
getting
up
to
your
laptop
and
plugging
in
their
USB
Drive,
but
for
an
IOT
gateway
that
said
a
remote
oil
pump.
D
But
what
do
you
do
then?
If
you
go
scope
our
off,
then
maybe
let's
take
the
thing
off
the
wall
take
at
home
and
get
the
data
when
they
get
home.
What
you
really
want
to
do
is,
if
at
all
possible,
find
some
way,
so
your
challenge
is
find
some
way
to
detect
that
there's
compromise
and
react
to
it
in
a
way
that
saves
you
can
I
wipe
the
data,
can
I
wipe
off
the
micro
service
images.
These
would
be
good.
D
Things
can
I
make
sure
that
everything
is
encrypted
and
then,
therefore
you
know,
even
if
they
get
the
hard
drive,
it's
not
a
problem.
Okay,
great
but
then
I
need
to
have
some
monitoring
of
the
whole
system,
and
so
what
about
authenticity
of
the
hardware?
So
in
some
cases
you
have
hardware
that
you
trust.
But
then
you
come
to
find
out
later
that
there's
security
flaws
and
the
hardware
itself,
possibly
even
some
way
that
someone's
doing
remote
monitoring
of
the
hardware
and
you
weren't
aware
of
it.
D
So
no
the
sources
of
your
hardware
and
know
that
what
you're
putting
out
in
the
field
not
only
has
a
TPM
2.0
module,
but
that
all
of
the
setup
on
the
board
is
what
you
intended
to
pay
for
and
there's
nothing
there.
That
is
waiting
for
a
BIOS
update
through
a
backdoor
or
anything
like
that.
Just
be
careful
of
supply.
D
Trusting
connected
devices
gets
really
tricky.
So
these
inexpensive
battery-powered
sensors,
the
kind
that
you'd
put
ten
thousand
in
a
warehouse.
They
don't
tend
to
come
with
identities.
The
same
way
that,
like
a
you,
know,
a
mainboard
of
a
computer
does
so,
whereas
a
rack
server
is
going
to
have
all
the
the
software
identity
stack
with
the
little
battery-powered
device.
How
do
you
know
which
one
is
which?
D
And
how
do
you
trust-
and
that
being
said,
if
you
have
10,000
of
them,
who
is
going
to
assign
the
identities
to
the
deploy,
that's
a
lot
of
numbers
and
where
are
they
going
to
plug
them
into
for
you
to
say?
Oh,
that
ID
belongs
that
ID
doesn't
belong,
so
you
have
to
consider
the
cost
and
the
labor
involved
in
getting
these
systems
deployed.
D
So
if
there's
data
flowing,
you
want
to
protect
that
possibly
encrypt
it
as
near
as
possible
to
the
source,
but
what
about
commands
going
back
out?
So
some
really
interesting
commands
are
shut
the
water
valve
off
or
open
the
gate
to
the
to
the
warehouse,
which
is
a
mostly
automated
facility.
If
I
can
issue
that
command
remotely,
once
I
figure
out
how
to
get
into
the
system,
then
I
can
basically
just
line
somebody
up
and
say:
okay,
you're
ready
to
steal
in
five
four
three.
D
D
Device
management
is
actually
a
backdoor,
so
think
about
it.
This
way,
if
you
have
a
device
cloud
and
in
that
device
cloud,
you're
saying
of
all
my
IOT
devices
a
several
hundred
of
them.
This
is
the
battery
life.
That's
remaining.
This
is
the
network
address
of
the
device.
This
is
the
firmware
I've
got
and
you
use
that
information
to
make
some
decisions,
such
as
oh
we're
almost
out
of
battery
here.
Let's
go
ahead
and
move
the
workload
over
to
where
we're
getting.
D
You
know,
sensors
that
have
more
more
data
coming
in,
because
the
battery
is
still
alive.
If
somebody
spoofs
that
information,
then
you
can
force
decisions
on
the
back
end
and
a
lot
of
the
device
management,
API
systems
or
cloud
systems
they're
open.
All
you
really
need
is
an
API
key
and
you
can
get
that
off
of
any
one
device.
So
steal
a
device,
you
steal
a
way
to
report
the
data
in
report,
the
data
and
falsely
decision
start
getting
made
and
nobody's
any
the
wiser.
D
So
the
operating
system,
this
complex
thing
so
who's
heard
of
secure
boot
before
who
use
a
secure
boot
in
any
of
your
deploys.
It's
tough,
secure
boot
is
a
hard
thing
to
get
right
because,
basically
it
says
I'm
on
some
hardware
that
hopefully
I
trust
and
I'm,
going
to
execute
the
boot
sequence
like
this
and
I'm
gonna
check
that
I'm
only
loading
the
drivers
that
I'm
supposed
to
load
and
I'm
only
running
the
binaries
I'm
supposed
to
run
them
only
opening
the
the
network
configuration
I'm
supposed
to
open
and
then
you're
done.
D
You
basically
report
I
securely
booted
what
happens
after
boot.
So
then
you
get
some
software
that
decided
to
get
if
gif
itself
an
update,
and
you
trusted
that
software
you
installed
it,
but
now
the
update,
that's
God
is
doing
something
funny.
Secure
boot,
won't
save
you
so
secure
boot
is
yet
another
step,
so
Hardware
rid
of
trust,
secure
boot.
These
are
the
first
two
layers.
What
comes
after
that?
D
You
have
to
know
what's
running
and
you
have
to
know
that
that
the
thing
that's
running
is
pure,
so
some
attestation
of
what's
happening
on
the
edge
node,
and
you
need
to
continually
monitor
this
because,
when
you're
in
the
the
manufacturing
facility
and
you're
kind
of
getting
everything
ready
and
your
all
of
your
binaries
are
loaded,
it
looks
fine.
It's
when
you
go
out
to
the
field
that
you'll
have
a
change.
So
it's
not
enough
to
say
that
the
image
looks
fine.
D
It's
gonna
change
afterward
when
you,
if
you're
actually
gonna
get
hacked
so
constantly,
monitor
monitor.
So
this
one
is
kind
of
a
pet
peeve
of
mine
in
something
I
like
to
point
out
a
lot.
So
if
you
have
a
private
key
and
you
put
it
into
an
edge
device
and
that's
meant
to
be
the
identity
right
so
can
it
can
attest
I.
Am
this
thing?
What
happens
if
someone
gets
ahold
of
that
device
and
they're
able
to
get
ahold
of
that
private
key?
D
This
is
essentially
like
taking
your
house
keys,
putting
them
on
a
park
bench,
and
you
might
as
well
put
an
address
tag
on
it
too.
That
says
where
you
live,
because
you
have
now
given
yourself
a
false
sense
of
trust
and
you've,
given
the
keys
to
your
kingdom
out
into
the
world.
What's
the
answer
to
this,
it's
actually
really
challenging
something
related
to
using
keys
and
asymmetric
key
pairs,
but
maybe
giving
them
out
when
you
know
the
device
or
the
edge
node
is
trusted
already.
So
something
that's
more
dynamic.
D
I
know
that
flashing,
fixed
private
keys
into
edge
nodes
is
very
dangerous.
That's
that
things
for
that's
for
sure
and
so
component
firmware
vulnerabilities
is
also
a
problem,
so
your
OS
is
running
and
you're
talking
to
the
serial
ata
chipset
there
who
made
that
does
it
have
ulnar
abilities?
Are
you
riding
on
this
for
all
of
your
edge
to
point?
Probably
not.
You
probably
have
10
20
different
vendors
based
on
the
variety
boards
you
have.
How
are
you
gonna
check
for
security
vulnerabilities
in
all
of
these
chipset
firmware
versions
that
are
on
the
board?
D
It
just
starts
to
become
a
nightmare,
so
much
of
a
nightmare,
in
fact
that
some
organizations
in
the
United
States
at
NSA,
for
example,
requires
the
authority
to
turn
off.
All
of
the
security
features
baked
into
all
of
the
components
of
the
board,
and
this
is
so
that
they
can
override
and
say
well.
At
least,
we
have
visibility
instead
of
trusting
some
of
the
stuff,
that's
been
put
out
there
that
might
might
have
flaws
in
it.
D
Hopefully
this
is
all
getting
you
scared
right
because
it
gets
me
scared,
and
so
what
about
security
updates
of
the
OS
on
the
security
updates
are
great
until
somebody
gets
in
the
middle
of
them.
So
this
is
a
golden
opportunity
to
cause
problems.
Also,
are
you
doing
security
updates
of
your
edge
nodes?
In
some
cases
you
put
a
pump
controller
out
in
the
oil
field,
and
once
you
put
it
out
there,
you
want
to
conserve
bandwidth.
D
Maybe
the
bandwidth
that
you
are
paying
for
is
on
satellite,
very
expensive,
and
you
say
patches
recommended
for
Ubuntu
nah.
That's
like
a
lot
of
money.
We'll
do
it
next
week
right-
and
this
is
because
you're
trying
to
make
a
cost-benefit
analysis
right
of
consuming
bandwidth,
and
so
then,
what
ever
happens
on
the
OS
usually
produces
an
audit
trail
and
some
logs
that's
a
great
place
for
people
to
hack.
D
It's
also
something
that's
typically
missing
from
the
edge
backup
if
I
can't
get
access
to
the
audit
trail
or
logs
and
I'll
never
know
what
happened
out
there.
Something
has
to
take
care
of
these
things
and
treat
them
like
data,
because
you're
not
guaranteed
protection
right,
no
physical
protection
of
your
of
your
edge
node.
So
something
has
to
treat
log
files
as
first-class
citizens
now
so
on.
The
network,
another
pet
peeve
of
mine
is
open
ports
at
the
edge.
D
So
there
have
been
some
hack
attacks
executed
against
energy
grid
systems,
where
somebody
really
just
kind
of
rolls
up
near
the
the
transformer
facility.
So
you've
got
these
kind
of
fenced
off
areas
outside
cities,
where
you
do
power
transformation
and
there
have
been
hack
attacks
where
people
roll
up
start
scanning
for
wireless
network
connections
that
are
available,
especially
SCADA,
and
then
they
just
begin
to
say
well.
D
Based
on
what
I'm,
seeing
there's,
actually
an
open
socket
here
that
I
can
talk
to
you,
I'm
just
gonna
I'm,
going
to
issue
a
couple
of
bytes
commands
and
see
if
I
can
get
it
to
feed
me
back
some
data
or
do
something
it's
basically
just
sitting
open,
because
nobody
thought
that
anybody
would
ever
be
out
there
with
a
laptop
and
it's
a
different
wireless
adapters.
Trying
to
do
these
things.
But
it's
starting
to
happen
and
fixed
VPNs
are
like
fixed
private
keys.
D
So
you
definitely
do
not
want
to
put
edge
hardware
out
there
that
contains
VPN
credentials
that
if
somebody
gets
access
to
the
edge
hardware,
they
can
get
on
your
VPN
because
that's
like
saying,
I
have
a
warehouse
and
I
have
a
data
center
and
I
have
a
retail
store
hack,
any
one
of
them,
and
you
get
the
other
two
for
free
right.
We
don't
want
this
scenario.
D
Network
access
control,
so
identities
on
the
network
are
pretty
easy
to
establish
and
in
a
fixed
environment.
That's
good
at
the
edge.
You
have
things
roll-on
and
roll-off
if
a
truck
pulls
up
to
my
warehousing
facility
and
it
wants
to
join
my
my
you
know
my
network
here
and
start
doing
some
edge
compute
data
sharing.
You
want
to
dynamically,
give
it
access,
but
then
you're,
not
gonna,
have
a
network
engineer
on
standby.
At
all
times
saying
you
know,
let's
let
this
identity
in,
so
you
tend
to
open
things.
D
That's
not
the
right
approach,
but
you
have
to
think
how
can
I
dynamically
a
check
identities?
And
so
then
we
always
talk
about
verifying
what's
happening
at
the
edge
up
in
the
cloud
insanely.
Is
that
the
right
device
did
it
give
me
genuine
data?
What
about
the
edge
saying?
Am
I
talking
to
a
real
control
plane?
Because
we
have
this
this
notion
that
the
masternode
is
going
to
be
like
walled
off
and
protected,
but
it
was
a
conversation
actually
happened
at
the
Eclipse
booth
earlier
today
was
really
interesting
about
smart
cities.
D
If
you
get
access
to
the
master
node,
you
control
the
city.
This
is
kind
of
a
future
we're
heading
towards
so
think
about
it.
The
edge
node
should
say
something
changed
on
that
control.
Plane,
I'm
not
liking
it.
You
know,
I'm,
just
gonna
go
into
offline
mode
until
I
can
get
rid
of
era
fication
that
would
be
a
betcha,
smarter
and
better
edge.
D
Attacks
of
the
transport
layer
is
very
interesting
for
Bluetooth
Laura
when
all
of
these
local
wireless
things,
if
you
just
blast
a
bunch
of
noise,
you
can
often
interrupt
the
transmission
and
prevent
any
data
coming
through
that's
a
great
way
to
like
turn
off
water
flow
or
cause
something
to
overflow
or
cause
fuel
line
transmission.
To
stop
all
stuff
that
you
can
do
just
by
blasting
white
noise,
and
this
is
a
problem
that
we
don't
have
a
solution
for
I.
Can't
think
of
any
way
to
you.
D
Can't
radio
shield
things
so
much
that
they
don't
get
interference
because
then
they
don't
get
transmissions,
not
sure
how
to
solve
it,
and
this
is
a
term
that
that
actually
came
out
of
a
years
ago.
Working
group
that
was,
if
think
it
was,
there,
was
edge
works
and
it
was
Cisco
and
it
was
Samsung
and
and
GE.
But
back
when
GE
digital
was
like
really
in
the
space
and
more
of
a
thing
and
so
denial.
The
thing
attacks
are
kind
of
like
this.
You
have
a
device
and
it's
reporting
data.
D
Maybe
it's
a
temperature
and
it's
reporting
it
over
Bluetooth,
Low,
Energy,
I,
don't
actually
have
access
to
the
device
and
I
don't
think
I'm
gonna
get
it
because
it's
actually
encrypted
data.
That's
transmitting!
These
are
all
really
good
architectural
implementations,
but
I
can
ask
it
to
connect
so
many
times
that
it
runs
out
of
battery
and
so
you're,
basically
denying
all
of
the
things
of
the
world
from
participating
in
the
solution.
This
is
something
that
we
don't
really
have
a
good
solution
for.
Okay,
last
section
and
I
did
I
get
to
watch
from
Steve.
D
So
when
you
are
running
micro
services
at
the
edge,
you
have
to
make
sure
that
the
micro
Service
images
are
the
ones
that
you
intended.
There's
a
lot
of
solutions
for
this
in
the
cloud
arena
and
all
we
really
have
to
do
is
make
sure
that
you
can
do
these
checks
checks
at
the
edge
and
do
them
quickly
or
maybe
an
offline
mode.
You
have
API
keys
and
things
find
a
way
to
securely
deliver
them
to
the
edge
instead
of
having
them
baked
in.
D
What
do
you
do
if
you
have
microservices
running
at
the
edge
that
were
unauthorised?
If
somebody
watchdogging?
Is
there
a
whitelisting
capability?
And
so,
if
that's
not
built-in,
then
I
could
basically
deploy
micro
services
just
by
getting
on
the
edge
node
and
you
might
not
know
controlled
access
to
resources
means
serial.
Ports
are
available
to
micro
services.
Are
they
available
to
all?
Should
they
be?
What
about
Wi-Fi
connectivity?
D
What
about
the
Bluetooth
radio
that
should
be
controlled,
guaranteed
remote
shutdown
if
I'm,
remote
and
I'm
managing
an
edge,
node
and
I,
say:
ok,
repeel
right
I
want
to
turn
off
these
micro
services
and
I'd
like
to
wipe
the
data,
because
we're
decommissioning
this
IOT
gateway
or
decommissioning.
This
video
camera
that's
been
been
taking
the
video
feed
and
doing
AI
I
need
to
know
that
that
stuff's
wiped
off
before
I,
basically
consider
it
no
longer
part
of
my
edge
compute
system,
and
you
want
to
match
the
right
micro
services
to
the
right
edge
hardware.
D
So
it's
no
good.
If
you
deploy
a
micro
service
that
checks,
employee,
IDs
in
a
factory
and
says
you
can
or
cannot
be
in
the
sector,
no
good
to
run
that
micro
service
at
the
edge
if
it
runs
in
the
women's
restroom
and
so
then
there's
no
real
control,
because
it's
not
at
the
right,
targeted
location
so
match
the
right
edge.
Hardware
to
the
right
micro
services
and
the
last
one
is
micro
service
is
that
the
edge
are
great
but
they're,
largely
unmonitored.
D
And
what
happens
if
you
deploy
one
that
just
start
sending
your
data
to
some
back
channel
and
this
happens.
People
write
micro
services
and
they
put
them
out
there
for
free
and
so
hey
use
this.
This
is
an
MQTT,
the
broker,
slash.
You
know,
analysis
system.
That
will
give
you
a
great.
You
know
edge
iot
system
and
you
look
at
it
you're
like
whom.
D
Well
I
don't
have
access
to
the
source
code
and
I'm,
not
really
sure,
where
they're
sending
all
my
data,
so
maybe
I
should
think
twice
about
that
cool,
so
the
gist
of
it
is
in
the
end,
it's
going
to
require
a
multitude
of
approaches
and
everybody's
going
to
have
to
get
involved.
So
what
we
really
need
to
do
now
is
start
thinking.
D
Security
first
question
the
status
quo,
so
just
because
it
works
in
cloud
and
data
center
doesn't
mean
it's
right
for
the
edge
and
just
remember
the
edge
environment
is
so
different
that
unless
you're
absolutely
certain
you've
explored
all
that
challenges,
you
should
assume
that
things
aren't
yet
safe
and
then
start
to
get
involved.
That
way,
asking
the
right
questions
before
we
start
giving
the
right
answers:
cool,
Steve,
okay,
thank.
A
Let
me
wrap
up
here
by
pointing
out
that
this
session,
you
just
attended,
is
being
run
by
this
IOT
and
edge
working
group
within
the
kubernetes
project.
We
have
two
meetings
per
month.
The
reason
we
have
two
is
one
is
done
for
the
convenience
of
Europe
Eastern
Europe
Asia
Japan,
the
other
one
for
North
America.
A
Once
you
get
synced
on
to
the
right
one
of
these
and
I'm
sorry
that
I've
observed
that
some
people
join
at
the
wrong
time
for
the
the
wrong
four-week
cycle,
but
look
at
them
here
take
a
picture
of
the
the
top.
Therefore,
that
shortened
link
you
can
get
this
whole
deck
to
find
it
later,
and
these
are
zoom
meetings
interactive.
A
A
Once
you
join
the
heading
for
the
group,
you
can
discover
all
of
this
stuff
at
this
point.
I
think
we've
well.
It
looks
like
we
hit
our
deadline
if
another
speaker
walks
in
the
door
we'll
take
it
out
in
the
hallway
out
of
courtesy.
But
at
this
point,
if
anybody's
got
questions,
I
think
we
have
an
extra
mic
I'm,
not
sure
where
it's
at
okay
so
raise.
A
And
then,
if
not
I
think
we
had
to
break
up
some
of
these
groups.
So
if
we
have
no
questions,
let's
reassemble
in
the
hallway
divide
it
up
into
those
popular
groups
and
you
let's
get
to
meet
each
other.
Okay,
we've
got
one
okay,
so
we
were
likely
to
be
able
to
continue
any
questions
or
you
want
to
go
into
these
focus
groups
again.
Let
me
know:
there's
one.
F
C
So
did
the
roadmap
for
the
first
six
months.
We
try
to
cover
this
material
for
intro,
try,
try
to
figure
out
the
use
cases
and
things,
and
things
like
that
and
for
the
last
couple
of
months
we're
basically
focusing
on
what
we
presented
here
so
focusing
a
little
bit
on
security,
focusing
on
on
the
messaging
trying
to
deep
dive
in
into
these
things.