►
From YouTube: Edge-Native: The New Paradigm For Operating And Developing Edge Apps - Krisztian Flautner, Cisco
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Edge-Native: The New Paradigm For Operating And Developing Edge Apps - Krisztian Flautner, Cisco
Speakers: Krisztián Flautner, Frank Brockners
“Cloud native?” Check! Apply the same principles at the Edge? Hmmm! How do I operate Apps across 1000s of locations, which are often hidden behind layers of NAT? How do I run AI apps on nodes that are too small to fit the AI model? How to make it operationally simple? Lets discuss and demo! We’re all familiar with “cloud native” -but once we start to operate applications at the edge, we have to adopt a new set of principles and evolve our cloud-native paradigms. We deploy Apps at the edge to achieve lower latency or higher performance, to comply with data sovereignty regulations, to reduce transit cost or to perform near real-time decision making on local data sources. Developing and operating Edge apps requires us to answer questions like: How do I operate Apps across 1000s of locations, which are often hidden behind layers of NAT and have spotty cloud connectivity? How do I run computation heavy tasks, like AI apps, on a set of nodes where each node does not have sufficient CPU and memory to run the entire model? How do I deal with a heterogeneous environment, with x86 and ARM-based devices? Which additional tools do I need to assure compliance to data-privacy rules, run AI models that just don’t fit a single compute element, or perform federated learning in an efficient way?
A
B
Hate
to
disappoint
you,
but
I'm,
not
Frank
and
I'm,
not
a
distinguished
engineer,
but
I
do
work
at
Cisco
and
apologies.
Frank
couldn't
be
here.
He
he
had
the
bug
and
couldn't
get
on
the
plane
from
Germany,
so
I
stepped
in
I'm,
Chris,
floatner
and
I'm,
actually
in
the
edge
group
on
the
business
side,
doing
the
product
definition,
but
working
on
some
of
the
products
that
actually
are
described
here.
B
So,
if
you've
been
in
the
industry
for
a
while,
you
know
that
there's
always
a
pendulum.
Sometimes
people
are
talking
about,
centralized
Computing,
sometimes
decentralized
Computing.
Sometimes
the
client
is
in
the
focus.
Sometimes
it's
the
server
and
this
pendulum
kind
of
goes
back
and
forth
every
10
15
years
and
there's
a
new
kind
of
Norm
in
the
industry.
Now
we
think
that
there's
a
this
is
actually
happening
now,
as
well.
B
At
least
a
little
bit,
there's
still
a
lot
of
momentum
towards
the
cloud,
but
there
are
some
users
that
are
not
swinging
back
towards
the
edge
to
solve
certain
problems
that
that
they
think
the
edge
is
actually
more
suited
for
we'll
get
into
that.
What
type
of
problems
these
might
be,
but
what's
different
here,
is
that's
not
a
paradigm
switch
in
a
sense
that
people
are
expecting
a
lot
of
the
goodness
that
they've
gotten
used
to
from
cloud
computing,
but
they
now
expect
a
very
similar
Paradigm
to
actually
exist
for
Edge
compute
as
well.
B
And
what
are
the
reasons
for
this?
Some
are
related
to
just
costs.
How
much
does
it
cost
to
actually
move
something
to
the
cloud
and
move
it
back
and
that's
an
issue
in
certain
applications?
Real-Time
performance
regulation?
These
are
all
kind
of
driving
a
migration
towards
the
edge.
But
I
would
add
one
more
to
this
and
in
general,
which
is
just
about
robustness.
How
do
you
actually
come
up
with
complex
distributed
applications
that
are
actually
robust
and
run
partially
at
the
edge
partially
at
the
cloud,
but
delivering
the
right
kind
of
user
experience.
B
So
the
challenge
now
is
not
to
actually
completely
detach
from
the
cloud
and
come
up
with
a
new
paradigm
and
kind
of
go
back
to
the
good
old
days
of
desktop
Computing,
but
rather
take
a
lot
of
learnings
from
the
cloud
and
apply
it
to
at
the
edge
and
make
distributed
resources
easy
to
use
the
I
guess
the
key
metrics
for
that
are.
Can
you
make
it
easy?
Can
you
hide
the
complexity?
That's
actually
inherent
in
the
edge
deployments.
B
Can
you
do
that
quickly
without
having
to
set
up
a
lot
of
the
resources?
Can
you
automate
a
lot
of
these
processes,
and
can
you
make
it
very
familiar
to
all
those
developers
that
have
used
to
the
convenience
of
the
cloud
and
the
tooling
that
actually
they
used
to
day
to
day,
but
before
we
can
get
into
that,
we
have
to
talk
about
which
Edge
we're
actually
talking
about,
and
if
you
go
to
Edge
talks,
often
it
can
be
quite
confusing.
B
B
So
we're
used
to
having
simple
devices
at
the
edge
that
are,
they
could
run,
maybe
one
application
or
two
and
now
actually
it
was
a
single
device,
but
actually
what
we
see
is
a
need
for
actually
multiple
devices
that
provides
some
level
of
redundancy
some
level
of
scalability
at
the
edge
nodes.
The.
B
B
Historically,
some
of
this
stuff
has
been
incredibly
manual
and
done
by
people
that
we're
not
computer
scientists,
but
they
were
actually
you
know,
construction,
Engineers
or
building
a
lamp
post
and
things
like
that,
and
they
were
expected
to
do
some
level
of
manual
configuration
now.
You
can
kind
of
figure
out
what
went
wrong
with
a
lot
of
those
projects,
but
in
this
world,
what
we'd
like
to
do
is
to
move
move
to
a
completely
automated
system
that
can
actually
be
done
remotely,
but
with
the
help
of
people
that
actually
are
at
the
age.
B
So
a
deployment
scenario
is
on
this
picture.
Quite
simple:
really
you
have
a
set
of
endnodes,
sensors
and
actuators
that
communicate
to
one
or
more
devices
at
the
edge
which
then
actually
communicate
with
some
back
end
in
the
cloud.
What
makes
this
use
case
complicated
is
not
the
ACT,
not
that
part
of
it,
but
the
fact
that
you
actually
now
have
to
multiply
this
n
times
and
you
want
to
actually
make
that
work
want
to
manage
them
centrally
and
want
to
have
a
consistent
user
experience.
B
B
B
B
Etc
now
concurrency
is
the
bit
that
gets
more
complicated
because
we're
used
to
kind
of
scaling
things
up
by
the
processes
or
the
containers,
and
things
like
that.
But
here
you
actually
have
to
scale
the
sites
and
and
that's
a
very
large
lever
that
you
have
if
you
go
after
large
deployments
now.
This
disposability
is
another
key,
because
if
you're
managing
lots
and
lots
of
sites,
suddenly
you
can't
afford
to
care
about
all
of
them
all
the
time.
So
you
you
should
think
about
them
as
disposable
resources
that
can
be
brought
up.
B
Another
area
I
would
highlight
is
actually
logs.
So
when
they
wrote
this
decade
ago,
logging
was
the
thing
I
think.
If
you're
doing
things
at
scale,
you
suddenly
have
to
start
moving
towards
metrics,
because
otherwise
you
just
get
lost
in
the
noise
of
all
the
data
that
these
computers
are
generating
for
you.
B
So
let's
dig
into
these
a
little
bit
more.
So
if
you
look
at
the
concurrency
on
the
scale
side,
the
picture
on
the
left
just
shows
you
the
scaling.
If
you're
actually
scaling
processes,
you
want
to
run
more
of
them.
You
scale
up
and
essentially
The
Edge
Computing
Paradigm
gives
you
another
dimension
of
scaling,
which
is
the
number
of
size
you
have.
B
So
sites
that
are
referred
to
are
just
an
abstract
notion
of
location,
and
it's
you
know
it's
a
collection
of
compute
nodes
that,
in
a
in
a
certain
location,
now
Frank
actually
points
out
that
they
don't
have
to
be.
It
can
be
any
collection.
You
can
organize
these
sites
in
any
logical
way
that
you'd
like,
but
often
it's
a
very
useful
thing
to
aggregate
them
based
on
where
they
are
actually
located,
and
the
other
key
here
is
that
these
are
multi-purpose
nodes.
B
So,
on
the
disposability
point,
this
is
really
driven
by
the
user
experience
that
we
want
to
enable
now
one
of
the
property
of
these
sites
is
that
it's
very
likely
that
there's
no
I.T
person
there
there's
no
devops
person
to
actually
maintain
these
sites
and
the
debugging
process
will
be
very
much
like
we.
We
were
used
on
Windows
and
control
deleted
when
it
doesn't
work
here,
somebody's
going
to
push
the
switch
and
the
site
needs
to
come
back
up
if
there
was
a
failure
and
it
better
actually
work
right
out
of
the
box.
B
B
I
think
I've
already
mentioned
the
backing
services
in
the
heruku
model
on
the
left.
Those
were
attached
to
the
the
deployment
directly
in
the
model
that
we
think
works
for
the
edge.
Some
of
obviously,
some
of
these
backing
Services
may
not
actually
be
at
the
edge,
so
they
you
have
to
kind
of
worry
about
Connections
to
them.
That's,
okay!
B
It
just
gives
you
a
little
extra
work
to
actually
make
sure
that
those
connections
can
work
as
well
now
to
enable
this
type
of
operations,
things
have
to
be
declarative
and
have
to
start
from
the
edge
in
reaching
to
the
cloud.
The
if
you
actually
have
another
operation
model,
which
is
pushing
things
out
from
the
cloud
side,
you
often
will
run
into
problems
when
you
have
to
deal
with
complicated,
Network,
topologies
and
or
misconfigurations.
B
B
I've
mentioned
logs
already,
and
this
is
actually
an
interesting
area
of
research.
What
do
you
do
with
the
information
that's
contained
in
these
logs
now?
A
lot
of
people
just
like
to
dump
them
somewhere
and
in
case
they
need
it
and
then
analyze
it
now.
If
you
have
a
hundred
sites
and
100
sides
times
five
machines
distributed
over
a
relatively
slow
links,
this
aggregation
becomes
a
hard
problem
and
it
may
take
forever
to
actually
look
at
these
logs.
B
When
something
changes
when
something
is
out
of
the
ordinary,
this
is
actually
an
area
that
I
think
requires
some
Innovation
at
the
edge,
because
it's
it's
a
quite
different
model
than
what
we're
used
to
in
the
the
other.
The
other
side
of
the
the
connection
so
actionable
information
information
is
gold
and
the
question
is:
how
do
you
turn
raw
data
into
actionable
information?
B
The
the
picture
on
the
right
is
actually
from
a
paper
that
Frank
and
his
co-authors
wrote
that
actually
tries
to
do
that.
They're
doing
clustering,
algorithms
on
very
highly
dimensional
data
that
they
get
out
of
metrics
and
they
were
trying
to
figure
out.
Actually
what
are
the
import
when
they
should
look
you
know
is:
is
the
machine
working
okay
or
is
there
some
sort
of
a
problem
with
it
and
they
found
that?
Actually,
it's
possible
to
do
this
by
doing
some
fairly
sophisticated
statistical
analysis
on
it?
B
That's
actually
for
all
of
your
sites
together.
That
has
to
exist
and
work
so
that
when
these
sites
come
back
up,
they
know
actually
what
to
do
so.
The
chicken
or
cattle
can
fail
any
time,
but
obviously
your
collection
of
sites
shouldn't
so
I
already
mentioned
that
this
should
be
based
on
some
sort
of
a
pool
model.
There's
a
defined
state
that
the
Assad
should
get
into
and
when
they,
when
they
actually
come
up,
they
should
pull
the
information
that
relates
to
them.
B
B
So
what
are
the
three
things
that
are
in
this
demo?
Their
apps?
There
are
the
applications
that
are
deployed
to
the
sites,
as
I
mentioned.
Earliest
sites
are
just
groups
of
notes,
usually
geographically
grouped,
and
the
app
nodes
in
this
case
are
small
computers
and
you'll,
see
it's
just
a
Raspberry
Pi,
oh
there
it
is,
and
that
is
indeed
Frank's
hand.
So,
let's
and
the
the
point
of
this
is
we
actually
the
Prototype.
B
Started
so
that's
the
the
user
interface
for
Great
Bear,
which
is
a
an
edge
management
platform
that
we're
building,
and
here
you
can
see
a
few
sites
that
have
already
been
defined
in
Europe,
so
they're
one
is
Switzerland,
France,
UK
and
there's
nothing
in
Cologne
and
cologne
is
where
Frank
actually
lives.
So
he's
going
to
show
you
how
you
can
add
a
site
and
actually
right
here
and
I,
think
he's
giving
his
home
address
here.
So
please
don't
SWAT
him.
B
B
B
So
the
application
deployment
itself
is
really
quite
simple.
You
just
have
to
you
know
you
add
in
the
requirement,
in
required
information
about
the
secrets.
You
turn
on
some
configurations
and
and
after
that,
it'll
be
all
okay
and
you
can
see
in
this
example.
These
are
displayed.
So
what
what
he's
doing
there
is
turning
on
the
VNC
displays,
so
that
these
Edge
devices
can
actually
display
something
on
a
monitor
connected
to
them.
B
B
B
A
A
B
So
you
see
all
the
sides
that
have
actually
been
defining
Great
Bear
by
the
way
this
can
be
done
for
every
tenant
separately.
So
in
in
this
SAS
you
get
to
pick
your
own
sights.
You
can
share
and
share
them
with
others,
and
in
this
case
it's
showing.
It
shows
five
sites
across
Europe
and
the
use
case
here
is
really
quite
simple.
B
B
And
specify
the
content
on
the
application,
so
in
this
case
the
application
was
already
launched.
Now
it
just
says:
send
this
new
URL
to
them
and
you'll
see
very
quickly
that
they
start
updating
the
top.
Two
machines
are
actually
beefier
machines.
It
takes
a
Raspberry
Pi,
a
little
bit
longer
to
actually
get
the
updates,
but
they're
all
in
sync.
They
they're
all
decoupled
from
each
other,
and
we
managed
to
actually
launch
a
simple
application.
B
B
We've
shown
you
how
we
can
actually
deploy
things
very
quickly.
There
are
a
couple
of
other
points
I
wanted
to
make
here
about
how
easy
it
is
to
do
this
and
how
it's
actually
familiar
with
the
the
cloud
related
tools.
If
you
actually
want
to
use
this
system
and
the
I
guess,
the
first
point
is
okay:
how
do
you
get
some
apps
in
and
some
example
of
these
apps
that
we
run
on
these
are
related,
we're
actually
very
much
interested
in
a
lot
of
machine
learning
workloads.
B
So
there's
an
edge
AI
component
to
this,
but
when
you're
actually
moving
something
to
the
edge,
you
might
have
to
massage
the
application
itself
it
might.
It
might
be
too
beefy
to
fit
on
the
on
the
device
that
you
have
available
at
the
edge.
So
in
this
case
you
might
actually
have
to
shrink
down
those
models
or
you
might
have
to
do
Federated
learning
because
actually
near
the
edge,
where
you're
trying
to
do
some
learning
people
don't
give
you
direct
access
to
the
data.
B
So
sorry
they
give
you
direct
access
to
the
data
at
the
edge,
but
they
don't
want
you
to
actually
take
the
data
elsewhere.
So
you
deploy
Federated
learning,
learning
on
it
to
actually
enable
that
you
need
to
manage
the
data
flow
between
the
the
application
and
then
the
local,
the
data
sources,
and
sometimes
you
have
to
do
as
I
showed
you
earlier.
Some
of
the
rendering
in
IO
take
care
of
it.
Just
like
you
would
on
a
PC.
So
here's
an
example
of
something
you
can
do
with
Aji.
B
So
you
have
an
AI
model
which
you'd
like
to
deploy
at
your
at
the
sites
at
the
edge,
and
you
have
a
problem
because
you
have
two
new
models:
one
is
a
bit
too
large
for
the
edge
device
and
the
other
one
doesn't
run
fast
enough
and-
and
those
are
two
things
that
you
know
can
easily
happen.
If
you
have
Edge
devices.
B
Unlike
on
the
cloud
side,
you
can't
just
specify
a
beef,
for
instance,
to
have
a
beef,
for
instance,
that
the
edge
requires
somebody
to
go
out
there
get
a
new
board
plug
it
in
set
it
up.
So
here
you
actually
have
to
adapt
your
application
to
the
needs
of
the
edge,
as
opposed
to
the
other
way
around.
B
So
why
do
you
care
about
frame
rates
in
a
lot
of
inferencing
algorithms?
It
actually
matters
impacts
the
quality
of
the
the
inference
itself
in
an
example
that
we
had,
for
example,
depending
on
the
data
stream.
Sometimes
the
the
inferencing
energy
would
see
five
people.
Sometimes
you
would,
if
you
actually
increase
the
frame
rate,
it
would
C6,
because
it
has
more
granularity
on
the
images
and
it
can
actually
pick
up
the
sixth
person
that
was
obstructed
earlier
on.
B
B
Now,
if
you
have
that
specialized
Hardware,
you
still
have
to
figure
out
how
to
use
it
from
software.
So
that's
always
a
tricky
thing.
The
you
could
co-design,
but
it's
it's
an
option
that
you
can
do.
The
other
thing
you
can
do
is
model
compression.
You
take
a
model
and
then
you
try
to
get
the
most
optimal
representation
for
the
given
device.
B
There's
there
are
a
lot
of
tricks
in
the
toolbox
to
actually
do
this,
and
there
are
systems
that
help
you
achieve
this
and
or
you
can
use
your
own
flow
to
actually
massage
the
models
in
place.
The
point
is,
you
have
to.
You:
have
to
have
some
ml
knowledge
to
be
able
to
do
this
or
you
might
try
to
actually
split
a
model
across
multiple
Edge
devices.
B
Now
all
of
these
are
things
that
you
probably
should
do
or
must
do
depend,
depending
on
your
application
constraints
and
the
how
you
actually
do
it
is
that
there
are
a
number
of
papers,
actually
we've
written
and
they're,
but
they're
also
in
the
research
community.
That
can
actually
tell
you
how
to
do
this.
You
have
to
find
kind
of
a
a
part
of
this
network.
That's
easily
cutable
and,
at
the
same
time,
doesn't
have
bandwidth
requirements
that
overwhelm
The
Edge
systems,
but
that
can
be
done
by
heuristics.
You
won't
get
a
perfect.
B
So
again,
the
edge
is
similar
to
the
cloud,
but
it
has
different
constraints
and
it
requires
different
tooling
to
actually
make
the
most
of
it
now.
One
example
of
this
is
is
just
the
the
the
AI
and
actually
getting
a
model
to
work
there,
which
you
can
do
with
some
AI
magic
and
running
through
the
the
workflows
and,
as
a
result,
you
might
take
something
that
actually
initially
was
a
around
one
frames
per
second,
and
with
some
work
you
can
actually
increase
it
to
2.4
now.
B
This
is
just
an
example
of
what
you
can
do,
but
this
is
kind
of
a
factor
of
two
to
three
improvements
in
a
model.
Inference
is
often
possible
with
these
tools.
If
you
need
a
factor
of
10,
you
probably
should
actually
pick
a
different
model
to
start
with,
so
you
have
to
design
for
the
edge
in
some
degree,
but
you
you
get
to
leverage
a
lot
of
the
tools
that
everybody
else
would
be
using
on
the
cloud
side
as
well.
B
So
Federated
learning
is
a
technique
to
achieve
this
and
it
kind
of
works
like
this.
You
have
private
data
in
three
customer
locations,
but
you'd
like
to
have
a
global
model
based
on
all
the
customers.
So
you
try
you're,
trying
to
train
train
a
single
model
globally,
but
we
don't
have
Global
access
to
the
data.
You
just
have
local
access
to
the
data,
and
the
way
this
works
is
that
you
actually
train
three
different
models:
these
preserve
privacy
and
then
there's
a
method
methodology
to
aggregate
it
back
to
the
global
model.
B
A
A
B
B
So
the
summer
is
this:
Edge
native
enables
a
cloud-like
experience
at
the
edge
and
it
enables
a
transition,
hopefully
a
smooth
one,
as
the
pendulum
swings
back
towards
the
Edge
from
the
cloud.
The
the
main
Trend
that
we
see
is
that
people
have
sold
this
in.
In
with
coin
Solutions,
inverted
Solutions
aimed
at
different
verticals,
but
the
edge
native
actually
allows
people
to
kind
of
combine
these
into
a
platform
and
have
a
horizontal
solutions
that
can
actually
work
across
verticals.
So
they
can
use
a
use.