►
From YouTube: Building and Running an Enterprise-grade Serverless Platform on Kubernetes - Ying Huang
Description
Building and Running an Enterprise-grade Serverless Platform on Kubernetes - Ying Huang & Quinton Hoole, Huawei
Serverless platforms provide functions as a service, and have become a hot topic largely because they allow developers to focus on core business logic, leaving packaging, deployment, monitoring, event propagation, scaling and load balancing to the infrastructure. The serverless billing model is simple - pay-per-invocation - which can being significant benefits for many event-driven applications.
Huawei launched its FunctionStage serverless platform, which is built on Kubernetes, in 2017. In this talk we will explain in detail the design and implementation of FunctionStage. This involved both fairly straightforward function packaging, scheduling, auto-scaling, event triggering and load balancing, as well as some significantly more interesting challenges related to container re-use, on-the-fly micro service provisioning, reliable operation and much more. We will demonstrate the use of our system to solve some complex real-world problems in Huawei Public Cloud.
About Quinton Hoole
Quinton is currently Technical Vice President of Cloud Computing at Huawei. Previously he spent five years at Google, where he was an Engineering Lead on the Kubernetes team, and Technical Lead and Manager of Ads Serving SRE. He was also the founding engineer of the Amazon EC2 cloud computing project (2005-2009) where he lead much of the early design, implementation and operation. As a lead engineer at Nimbula (a startup that was sold to Oracle in 2012), lead the design and development of their Cloud IAAS product (both private and public cloud). His experience prior to EC2 includes telecommunications (call center switches, CTI systems, mobile billing systems), financial (data exchanges, point-of-sale systems), travel (internet booking systems, airline and hotel reservation systems) and mobile (aviation GPS navigation, banking etc).
About Ying Huang
Ying is currently a senior software architect at PaaS (Platform-as-a-Service) team at Huawei. She played a key role leading the design and implementation of FaaS (Function-as-a-Service) platform in Huawei. Before that, she worked in Microsoft Azure Identity team as an engineer for two years. She also worked in Amazon as a platform software engineer for seven years.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
B
A
Apologies
to
those
of
you
who
are
servile
as
computing
experts
that
a
few
minutes
might
be
a
little
boring
to
you
and
then
yang
is
going
to
take
you
through
the
architecture
and
design
of
the
highway
service
platform
goes
by
the
name
of
function
stage
she's
going
to
demonstrate
it
for
you
we're
going
to
talk
a
little
bit
about.
You
know
some
of
the
challenges
that
we
faced.
These
systems
are
designed
for
very
big
kind
of
enterprise
deployments.
A
A
A
A
So
it's
a
cloud
computing
model
for
execution
of
functions
there.
The
general
idea
is
that
you
shouldn't
have
to
worry
too
much
about
the
infrastructure.
All
you
do.
Is
you
upload
a
piece
of
code
and
you
chain
these
pieces
of
code
together
in
useful
ways.
The
other
kind
of
key
element
of
service
computing
is:
is
that
it's
all
event-driven,
so
typically,
these
functions
execute
as
a
result
of
certain
events
happening
either
inside
your
infrastructure
or
inside
of
your
application.
A
The
other
key
element
of
it
is
that
typically,
the
charge
model
or
the
resource
allocation
model
is
based
on
on
the
much
more
closely
to
the
usage
than
something
like
VMs
or
even
containers.
Every
time
you
execute
function,
you
you
pay
for
the
resources
associated
with
the
execution
that
function
as
opposed
to
having
some
static
resource
allocation
like
a
virtual
machine
running
permanently,
and
so
for
certain
use
cases.
This
is
very,
very
beneficial
if
you
don't
want
to
have
you
know
virtual
machines
running
all
the
time
if
you
only
occasionally
need
execution
capabilities.
A
The
other
important
aspect
of
it
is
that
it
is
auto
scaling.
So
you
shouldn't
have
to
worry
about
how
much
capacity
you
need
in
order
to
execute
these
functions
and
if
you
have
large
spikes
up
or
down
in
in
your
demand,
your
user
demand.
The
the
platform
should
automatically
take
care
of
that
for
you
and
auto
scale,
and
all
you
really
care
about
is
that
you're
paying
per
function
execution,
so
you
shouldn't
have
to
plan
for
any
of
these
huge
spikes
in
traffic
and
related
to
the
resource
allocation
model.
A
I
just
mentioned
the
billing
is,
is
similarly
tied
directly
to
how
many
invitations
of
your
functions
occur.
So
you
shouldn't
in
theory,
you
shouldn't
be
able
to
be
required
to
pay
anything
at
all
if
your
functions
are
not
executing
and
one
of
the
challenges
that
comes
up
is
well.
How
do
you
do
this?
If
you
have
functions
that
clearly
need
to
be?
You
know
instantiated
on
some
kind
of
virtual
machine,
usually
or
virtual
environment,
and
you
need
to
wire
up
a
whole
bunch
of
networking
to
make
them
all
work.
A
How
do
you
do
this
instantaneously
when
the
first
request
comes
in
and-
and
you
know,
10
milliseconds
later,
you
want
to
response
so
there's
this
cold
start
challenge
which
different
turbulence
computing
platforms
deal
with
in
different
ways,
and
you
should
definitely
if
you're
are
shopping
around
for
a
service
platform.
That's
one
of
the
areas
you
should
look
at.
A
It
happened
to
be
at
the
time
and
we're
effectively
writing
software.
They
were
building
applications,
although
they
were
not
software
engineers,
but
it
was
so
simple
for
them
to
do
this.
They
didn't
have
to
worry
about
compilers
and
debuggers,
and
all
this
kind
of
stuff
they
could
write
macros
in
Excel
and
and
get
quite
a
lot
of
stuff
done.
That
was
traditionally
being
done
by
you
know
software
written
on
mainframes
etc.
At
the
time.
A
So
you
know
the
end
result
of
all
of
this
is
this
is
what
a
typical
service
application
usually
looks
like
in
the
middle.
There
you've
got
a
bunch
of
functions.
These
can
be
invoked
in
response
to
any
events
occurring.
Those
events
could
be
incoming
requests,
application
level
requests.
There
could
be.
Events
in
the
infrastructure,
for
example,
object,
storage,
things
being
uploaded
to
object.
A
Changes
in
a
relational
database
results
from
an
artificial
intelligence
service,
for
example,
and
by
chaining
these
functions
together
through
these
sets
of
events,
the
end
result
of
whatever
the
application
requirement
is
is
fulfilled.
As
you
can
imagine
the
right-hand
side,
there
is
pretty
important
part
of
the
model,
so
you
need
to
have
typically
a
bunch
of
high
level
services
available
to
these
functions.
You
don't
usually
write
those
in
server
lists
or
within
the
functions
you
tie
into
much
higher
level
services.
C
Thank
you
so
next
we'll
talk
about.
How
do
we
do
that
in
highway,
public
cloud,
so
I'm,
sorry,
I,
think
this
is
yours
net
well,
I
can
take
over.
So
current
surveys
have
from
products.
There
are
a
whole
bunch
of
those,
so
you
have
a
lot
of
option
to
choose
from
first.
Obviously,
the
most
used,
the
most
successful
and
their
stock
price
is
very
high.
I'm
alone
also
like
this
is
I
personally
work
in
Amazon
for
seven
years
before
I
know
they
do
things
very
fast.
C
It
design
from
customer
perspective,
so
their
business
model
driven
their
technical
decisions.
So
I
would
say
this
is
a
wonderful
company
to
choose
from,
but
also
consider
others
as
well,
because
different
company
have
their
different
cultures
and
give
you
different
features
so
followed
by
Amazon.
We
also
have
other
Cerberus
crowd
function,
provider,
Microsoft,
Google,
IBM
and
others
like
kubernetes
iron
fishes
are
Muniz,
mention
them
because
they
are
relatively
smaller
than
those
big
giants,
but
Alibaba
and
tension
also
are
some
China
cloud
provider
that
the
nonstop
service
before
they
are
away.
C
However,
I
was
think
we
serious
consider
when
we
do
those
performance
analysis.
Actually
highway
is
getting
much
faster
than
years
so
well,
of
course,
this
is
based
on
a
few
months
ago,
when
we
analyzed
our
product
with
yours,
so
I
wouldn't
say
that
they
did
not
speed
up,
but
we
are
saying:
WOW
way
is
getting
Hedgehog
very
fast
and
we
will
talk
about
how
we
do
that
later.
C
Also,
in
addition,
we
have
say
so
many
big
giants,
radio
and
the
other
company
I
corrupt,
a
robot,
Expedia,
no
strong,
happy,
don't
want
the
financial
companies
they
are
using
Cerberus.
So
consider
when
we
talk
about
it,
security
financial
is
the
most
most
secure
the
business.
Then
they
are
still
picking
up
and
they
don't
have
any
problem.
So,
let's
ghost
who
they
meet,
what
okay,
the
architecture
of
Huawei
Cerberus.
C
So,
let's
platform,
we
call
it
function
stage
in
Huawei,
it
take
the
hardware
resources
provided
by
kubernetes
and
it
posed
in
the
software
resources
saved
in
function,
repository
combining
the
hardware
and
software
resource
together.
We
start
the
function
wrong
time
and
the
actual
user
function
running
in
the
function
wrong
time
hosted
by
kubernetes
port.
C
C
C
The
OS
layer
is
whywe
certified
CentOS,
it's
urn,
enix
version
of
Red,
Hat
I'm;
sorry,
it's
a
Linux
and
it's
a
variation
of
Red
Hat,
but
it's
highways
and
who
certified
and
they
publicly
available
the
web
by
our
service
platform,
the
language-specific
runtime
library
and
the
most
important
part.
The
customer
function.
C
C
How
does
function
function
instant
surrounding
of
our
cloud?
First,
we
are
other
server
list,
application,
user,
I,
don't
know
what
happens
behind
the
screen.
I
am
outside
of
the
crowd,
I'm
owning
sending
they
request
to
the
cloud,
so
they
don't
know
they
behind
architecture.
What
do
we
do
behind
the
screen
here?
The
customer
requests
gets
to
pick
up
by
API
gateway
and
a
preferred
gateway
further
forward.
The
function
call
to
our
function
with
dispatcher.
C
Function:
dispatcher
here
saying:
if
I
don't
have
here,
this
is
a
code
start.
If
there
is
no
such
thing
as
this
function
instance,
we
call
the
kubernetes
api
servers,
they
give
us
our
hardware
resources
and
then
the
function
wrong
time
will
be
started
ways
they
software
pulled
from
a
function
repository
and
then
we
press
dispatcher
forward.
They
request
to
the
actual
runtime.
C
So
if
this
is
a
code
start,
if
this
is
our
warm
start,
meaning
that
the
requestdispatcher
already
found
to
the
funky
instance
from
the
instance
catch,
it
is
integrated
and
then
it
will
directly
forward
the
request
to
the
function
instance.
So
there
is
no
waiting
for
this
part
to
be
start
for
asynchronous
call.
We
put
this
request
into
Kafka
message
queue
and
for
client
HTTP
request.
It
will
be
filtered
by
the
HTTP
trigger
to
see
whether
this
is
actual
or
customer.
C
D
C
So,
first
let
me
define
my
use
case
so
as
a
person
who's
like
travel.
A
lot
and
I
like
to
take
pictures
as
well
I
want
to
share
my
best
moment
with
relatively
not
cresol,
so
crappy
pictures
I
like
to
process
my
pictures
before
I
send
them
out.
Okay,
so,
but
I
also
want
to
travel
light.
I
don't
want
to
carry
my
laptop
just
because
I
want
to
process
my
pictures
before
I
send
them
out.
So
what
I
do
I
wonder
no
cost
website,
because
I
don't
want
to
pay
for
that.
C
A
lot
and
this
website
can
upload
a
picture
and
process
it
based
on
my
image
processing
logic,
because
I
could
do
a
lots
of
programming
behind
the
screen
and
like
what
your
customers
define
application
logic
inside
those
function.
So
here,
I
I
come
up
with
a
solution
with
I'm
hosting
our
website.
Our
static
website
hosted
by
Huawei
block
storage
that
it
the
website
the
web
page
take
pictures
into
a
OBS
bucket.
It's
kind
of
similar
to
a
stream.
C
So
there's
another
function
that
I've
created
in
function
stage
that
it
will
do
those
image
processing
for
me
and
they
trick
the
table
object.
Upload
events
will
trigger
the
function
to
process
the
image
for
me
so
and
finally,
it
will.
The
processed
image
will
be
saved
into
another
budget
so
that
I
distinguish
between
those
original
and
the
processed
image.
D
C
So
I
have
a
high
some
code
that
deal
with
three
bucket
that
the
upload
image
upload
bucket,
the
output
Park
it
and
the
the
bucket
that
hosts
my
website.
Yeah
I
put
it
here
many
because
I
want
to
say
how
many
times
I
have
been
using
this
web
page.
So
first,
the
events
that
been
dumped
by
OBS
I,
initially
I,
don't
know.
What's
a
structure
of
the
events
so
I
just
dump
it
here.
So
there
later
on,
I
know
how
it
works
and
start
from
the
code.
C
C
C
Afterwards,
I
do
this
transformation,
so
all
those
logic,
Adam,
Twitter's
image,
processing
inside
this
function,
transformed
image
after
it's
been
transformed,
I,
be
I-
will
be
upload.
This
file
to
the
OBS
to
be
shared
later
and
I.
Do
this
counter
increase
and
yeah
so
comes
to
this
transform
image
here.
C
C
C
C
C
It's
not
so
good,
but
at
least
this
is
something
done
by
my
program
and
it's
yours,
very
small
cost
right
yeah.
We
can
work
on
improvement,
that's
exactly
what
I'm
gonna
do
later.
So
initially
we
see
that
the
function
here
they
I
have
a
mutinous
defog
zip.
So
that's
why
I'm
trying
to
here
showing
inside
this
packet
where
I
host
my
function
code
so
say
I
have
to
file
here.
The
first
one
is
the
original
one.
C
Okay
and
the
second
one
is
I,
come
up
later,
who
I
realized
that
if
I'm
yo
leanness
using
my
cell
phone,
the
cell
phone
typically
right
now,
they
have
relatively
bigger
size
of
images.
I
don't
want
to
resize
myself
right.
So
so
I
use
this
program.
The
Python
thing
here
so
I
added
up
this
part
of
things
later
saying:
I'm
checking
the
image
size.
If
you
are
bigger
than
a
certain
amount,
I
want
to
shrink
you
so
that
you
don't
use
a
lot
for
my
computing
resources
and
remember.
Somebody's
function
turns
out
within
five
minutes.
C
C
C
So
this
is
a
big
function.
Big
picture.
Sorry,
it
has
three
mega.
This
is
a
one
that
I
downloaded
from
my
cellphone.
So
this
is
a
picture
I
took
in
when
I
was
only
playing.
It
looks
nice,
but
it's
foggy
it's
a
very
blurry,
so
let's
say
whether
it
can
process
it.
Your
roommate
has
to
eat
sometimes
can
sometimes
cannot.
C
C
Thank
you.
So
there's
a
lot
of
challenges
that
we
encountered,
because
we
want
to
build
this
to
everybody.
We
want
to
be
able
to
host
a
big
customer
and
for
smaller
customer
who
does
not
have
very
good
experience.
We
want
us
to
be
worried,
use
very
easy
to
use,
and
also
there
are
lots
of
big
player
already
in
the
background
in
the
error
that
they
start
to
three
years
before
us.
C
So
when
we
start
our
model,
we
actually
were
able
to
control
this
code
start
within
1.5
seconds,
but
this
is
only
for
the
container
to
start
without
any
of
this
added
on
which
Ian
and
like
a
code,
loading
all
those
kind
of
stuff
and
comes
back
to
the
actual
production
environment.
We
have
so
many
requests
and
we'll
videos
kubernetes
the
actual
performance
test
for
kubernetes
versus
when
they
have
when
we
actually
needs
more
than
two
digits
of
kubernetes
poor.
C
C
We
were
saying
that
kubernetes
is
our
hardware
provider
and
we
actually
may
not
rely
on
their
performance.
So
service
framework
should
be
creative
when
you
want
our
own
stuff,
but
still
we
are
not
reinventing
the
well
we're
using
resource
pool
which
is
providing
by
kubernetes.
We
configure
a
resource
pool
first,
so
think
about
other
services
provider
platform.
We.
Actually
this
is
not
real.
So
for
customer
you
don't
have
those
resources,
you
don't
need
to
pay
for
that,
but
for
us
we
actually
already
paying
for
those.
No
matter
you
use
it
or
not.
C
So
we
start
with
based
on
your
status
data.
We
start
a
whole
bunch
of
resource
pop
behind
the
screen
with
empty
port
with
the
actual
function
library
wrong
in
there,
but
not
with
the
actual
customer
code.
Okay,
so
that
part
save
us
more
than
two
seconds
when
I
say
more
than
two
seconds,
we
say
because
by
kuben
export
a
start
up
time
is
nearly
two
seconds
when
it
is
performed
under
node.
But
if,
if
I'm
they
load,
then
it's
more
than
one
sec,
one
minute
so
and
then.
C
Secondly,
we
do
the
code
injection,
because
we
have
this
resource
pool.
We
needs
a
real
customer
function
to
run.
So
we
do
the
code
injection
by
the
time
we
can
get
the
code.
We
change
the
identity
of
the
kubernetes
port
to
the
customer,
so
you
have
their
real
name
space
and
another
way
we
are
doing
that
so
remember,
JVM
have
not
heard
notoriously
bad
name
of
starting
up
very
slow.
C
What
we
do
here
when
we
use
the
open,
JDK
resources
that
we
have
a
JVM
lab,
they
actually
was
able
to
start
secondary,
not
the
first
one,
the
secondary
JVM.
On
the
same,
we
even
though
they
are
not
sharing
the
same
container,
but
they
are
on
the
same
same
way.
It
can
hearing
the
previous
human
resources
and
you
give
you
a
brand-new
JVM,
but
it
starts
much
much
faster
than
the
original
one.
C
So
second
challenges
security
with
durability.
There's
always
the
case
that
should
I
be
more
secure
or
should
I
provide
more
usability.
Here's
the
causes
so
how
we
internally,
we
use
internally
certified
and
unique
system
CentOS,
so
it
is
over
raising
of
Red
Hat,
but
internally
certified
it
stripped
out.
Many
sins
that
was
deemed
to
be
could
be
harmful,
so
many,
many
of
the
other
third-party
libraries
that
are
not
certified.
We
are
not
allowed
to
build
them
into
the
system,
so
it
turns
out.
There
are
many
standard
one.
C
Why
trying
to
come
to
do
my
demo
I
personally
encountered
those
because
I
kind
of
been
you
to
Python
and
I.
Don't
know
many
of
those
it's
supposed
to
be
standard
because
I
do
not
need
to
download
the
library
and
I
can
use
that
for
just
from
the
scratch.
However,
when
I
try
to
run
that
in
the
crowd,
it
doesn't
work,
it
says
the
board
Allen
was
not
found,
so
we're
saying,
then
some
of
the
standard
library
are
not
there.
C
So
I
have
to
be
invent
for
myself.
So
what's
the
solution
thy
will
be
your
own
customer.
They,
the
pain,
I
solve
this
issue,
okay,
so
other
challenges.
Of
course,
that's
why
I
put
here
number
three
and
I'm
gonna
go
over
this
quick
and
we
have
to
catch
up
for
the
big
players
because
they
are
head
for
three
years:
three
plus
and
we
have
company
with
you
and
we
consistently
broking
us
to
get
some
open
source
code
experience
or
those
kind
of,
and
we
have
like
three
country.
C
Actually,
four
countries
and
the
two
continental
co-workers
working
together.
You
name
it
we
have
it.
Okay,
so
jokes
are
part.
Here
are
some
resources
for
our?
This
is
why
we
cloud,
although
is
the
government
policy,
is
pretty
strict.
You
have
to
use
your
real
name
to
register
your
account.
You
have
a
cloud.
It
says
this
is
a
policy
by
all
the
cloud
provide
in
China.
C
So
I'm,
sorry,
if
you
don't
have
our
Chinese
identity,
cannot
use
it
and
also,
but
this
page
sorry,
this
product,
introduction,
page
I,
think
nothing
yeah
they
stripped
away,
because
this
is
not
in
general
announcement.
Yet
we
are
still
in
the
public
testing
phase,
so
they
strip
it
out,
but
you
can
see
our
funky
examples
in
the
github.
It's
just
called
the
function
stage
so
and
the
demo
it
go.
E
B
C
Thank
you
so
before
we
start,
we
actually
did
a
lot
of
research,
especially
on
those
open
source
projects
such
as
iron,
dot,
IO
fission
and
coulis,
and
open
whisk.
Those
are
lots
of
inspiration,
I
would
say
each
project
has
their
own
benefit.
Many
of
them
like
they
are
simple.
Something
is
just
simple
fact,
however,
the
performance
wise
not
so
idea
I
was
saying
like
the
code
start
has
been
less
than
two
seconds,
so
that's
our
major
concern
and
also
like
because
wow
he
is
able
to
provide
your
enterprise
great
platform.
C
It
has
to
be
able
to
run
in
thousands
more
than
thousand
tens
of
thousands
of
function.
Instance
at
the
same
time,
so
we
cannot
just
user
borrow
everything
from
the
open
source,
because
the
they
are
pretty
I
would
suggest
you
in
the
earliest
states.
They
need
some
improvement,
so
we
borrowed
some
idea
of
the
earth.
Definitely
because
we
use
kubernetes
right.
Kubernetes
is
also
is
kind
of
open
source
and
many
of
those
open
source
for
general
kubernetes
foundation
as
well.
C
So,
thanks
for
all
those
previous
opens
of
Thoroughly
is
provider
that
we
are
not
or
not
from
you,
but
we
do
some
comparison.
We
also
consistently
compare
what
did
we
achieved?
What
we
are
still
like
to
be
hand
between
all
those
like,
even
for
this
now
open
source,
like
Amazon
I
sure,
would
comparison
but
I'm
not
posting
to
here,
because
we
are
still
in
public
testing,
we're
having
general
runs
yet
so
we
I
guess
it
will
come
up
later.
It's
not
for
me
to
say
we
are
bad
on
something
but
I
promise.
C
The
question
is
sorry,
the
question
is:
do
we
take
any
action
to
prevent
our
customer
to
harm
other
customers?
Yes,
we
do.
We
do
put
we're
trying
to
isolate
the
customers
action
by
isolate
their
content
or
actions.
They
are.
They
have
limited
available,
OS
layer
operations
that
can
be
done
there.
That's
why
we
use
our
in-house
certified
CentOS
Linux
system
so
and
we
are
also
still
trying
to.
C
Actually
scan
and
prevent
it
from
happening,
we're
not
saying
we're
done
there
yet,
but
we
do
provide
trying
to
everything
that
we
can,
including
like
control
our
own
OS.
We
are
not
open
this.
This
OS
is
not
borrowed
by
open
source
without
any
internal
certification
or
filtering.
So
also,
we
do
like
a
code
scanning
or
customer
code
scan
just
making
sure
that
they
don't
have
harmful
actions.
Gonna
happen.