►
From YouTube: Webinar: Serverless Operations
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
FaaS functions on Kubernetes are increasingly popular for writing short-lived stateless applications. We’ve often covered the developer productivity advantages, such as the time to create a useful application from scratch without learning a lot about Kubernetes.
In this webinar, we will focus on the operational aspects of serverless applications on Kubernetes. We’ll talk about this in the context of Fission, an open-source FaaS framework used to run Serverless functions on any infrastructure, on-premise or in the cloud.
Operations is largely about reliably managing changes to applications, and making sure they are running optimally.
We’ll cover operational aspects of serverless applications, including:
- Managing deployments using declarative specifications.
-Tracking changes to deployment configuration.
- Using canary deployments to reduce the risk and impact of application changes.
- Managing operational expense: We’ll show how you can make different cost-performance tradeoffs; we’ll discuss what the defaults choices imply, and how to tune these.
- Finally, we’ll demonstrate how to track serverless application performance and issue alerts using Prometheus.
A
Alright,
it
looks
like
our
numbers
have
stabilized
and
we'll
go
ahead
and
get
started.
Welcome
to
today's
CMC
s,
webinar
service
operation,
I'm
taylor,
nard
marketing
manager
at
CNCs
and
I'd
like
to
thank
you
all
for
joining
us
today.
I'd
also
like
to
welcome
today's
presenter,
film,
Vasanti
creator
of
fishin
a.m.
which
is
an
open
source
server
list
framework
he'll
be
talking
about
service
applications
on
kubernetes
today.
A
So
just
a
few
housekeeping
items
before
we
get
started
you
as
an
attendee
are
not
able
to
speak
during
the
webinar,
so
there
is
a
Q&A
box
at
the
bottom
of
your
screen.
So
if
you
have
any
questions
during
the
webinar,
please
feel
free
to
drop
them
in
there
and
then
they
get
to
many
as
we
can
at
the
end.
Also,
the
session
is
being
recorded
and
will
be
sent
out
afterwards,
along
with
the
link
to
the
presentation,
usually
within
about
24
hours.
B
Caitlin
and
hey
everyone
I'm,
some
banana
I
work
at
platform
9
on
the
on
the
vision
framework.
I
created
this
framework
about
a
year
and
change
months
ago,
and
most
of
this
talk
is
not
really
a
specific
deficient.
We've.
We've
thought
a
lot
about
how
server
less
and
fazz
effects
operations
and
how
it
is
affected
by
by
DevOps,
both
technically
and
and
organizationally,
and
we're
going
to
talk
about
that
today.
B
So
really
quick
intro
to
what
fish
in
itself
is,
it's
an
open
source
could
be
Nettie's
native
server,
less
function
or
fast
framework.
If
you're
familiar
with
AWS
lambda,
the
the
idea
is
very
similar,
except
that
it's
open
source
and
you
can
run
it
in
the
kubernetes
cluster,
which
means
you
can
run
it
on
premise
in
the
cloud
wherever
you
can
run
kubernetes
the
main
design
goals
of
this
framework
are
that
it's
easy
to
use,
so
you
don't
have
a
steep
learning
curve.
B
The
goal
is
to
make
developers
very
productive,
and
it's
also
designed
to
be
to
be
in
actual
its
actual
runtime
performance
and,
more
specifically,
its
we've
tried
to
make
tunable
cost
performance
trade-offs,
and
we
will
go
into
this
a
little
more
deeply
later
in
the
presentation,
but
but
first
I'd
like
to
do
you
know,
zoom
out
a
bit
and
and
motivate
serverless
itself.
A
lot
of
the
motivation
for
serverless
has
been
around
developer
productivity.
B
The
the
other
motivation
has
been
has
been
economics
related,
which
is
that
services
that
are
idle.
Ideally,
you
shouldn't
be
paying
for
them,
so
you
you
want
to
pay
just
for
what
you
use,
and
both
these
motivations
mean
that
serverless
is
going
to
occupy
a
really
important
part
of
the
software
stack
and
importantly,
even
though
we're
talking
about
paying
just
for
what
you
use,
there
are
on-premise
benefits
too
and
and
again
we'll
go
into
this
a
little
deeper,
but
generally.
B
So
now
that
we
talked
about
server
less
and
the
developer
benefits
as
well
as
the
cost
benefits,
what
does
it
really
mean
to
take
several
less
apps
into
production?
We
want
all
these
productivity
advantages.
We
think
it's
useful,
both
in
the
public
cloud
and
in
the
data
center,
but
what
we
really
want
is
to
be
able
to
go
faster,
both
safely
and
at
scale
and
and
let's
dive
into
what
that
means,
that
the
general
idea
of
DevOps
has
always
been
that
we
want.
We
want
productivity,
but
we
also
want
safety.
B
B
So
you
know
there
are
various
versions
of
this
pipeline,
but
in
general,
any
organization
has
a
pipeline
through
which
changes
go
from
the
developers
environment
all
the
way
to
the
end-user.
So
a
typical
pipeline
might
have
some
sort
of
pre
mode
checks.
These
may
run
on
the
developers
laptop
or
in
a
CI
system,
and
they
run
before
the
code
affects
anybody
else
other
than
that
developer.
B
Assuming
those
checks
are
passed,
you
go
to
the
next
step,
which
is
building
and
packaging.
To
make
this
thing
ready
for
distribution,
then
it
goes
through
some
more
steps
of
validation
and
approval.
If
there's
a
staging
environment,
it
goes
there
and
this.
This
is
an
environment,
that's
as
close
to
production
as
possible,
but
doesn't
actually
affect
production
users
and
finally,
it
gets
deployed
to
production,
but
that's
not
the
end.
B
B
I'm,
just
gonna
quickly
go
through
these,
but
we'll
dive
into
each
of
them
in
a
lot
of
detail.
They're
really,
the
beginning
of
everything,
is
how
you
specify
your
applications
and
that
that
kind
of
sets
the
stage
for
what
your
tooling
can
and
cannot
do,
and
we're
going
to
argue
that
declarative
configuration
is
really
the
best
way.
It
is
to
specify
applications.
B
We're
gonna
talk
about
live
reload
so
that
you
get
fast
feedback
on
bugs
about
record
replay
for
better
and
debugging
and,
as
we
go
closer
to
the
later
stages,
they're
more
related
to
production
and
operations.
So
we'll
talk
about
reducing
deployment
risk
with
canneries,
we'll
talk
about
monitoring,
metrics
tracing
logs
alerts,
all
that
stuff,
and
finally,
we
will
take
a
little
bit
of
a
tour
into
cost
optimization,
which
is
a
pretty
big
topic.
B
B
So
there's
two
broadly
two
ways
of
specifying
or
configuring
an
application
for
such
that
it's
ready
for
deployment
there's
the
imperative
way,
which
is
essentially
composed
of
you,
know
a
series
of
verbs,
so
you
say
take
this:
do
that
copy
this
file
here
execute
that
file
there.
You
know
open
this
port
there
and
so
on,
and
these
are
very
imperative
steps
they're
essentially
like,
if
you
imagine
using
something
like
ansible
or
even
a
bash
script,
that
does
a
deployment.
Those
those
things
are
typically
embedded.
B
By
contrast.
The
other
way
to
do
this
is
to
use
declarative
configurations,
and
here
you
don't
really
say
how
to
do
a
thing.
You
just
say
what
the
end
state
should
be,
so
you
say
ensure
that
this
file
exists
on
this
server
or
in
this
cluster
or
ensure
that
this
resource
exists
over
here,
ensure
that
it's
up
ensure
that
there's
so-and-so
number
of
replicas
ensure
that
it's
healthy
and
things
like
that
and-
and
it
sounds
kind
of
subtle
but
the
what's
enabled
by
declarative
stuff.
B
Is
it's
pretty
different
from
what's
enabled
by
imperative
things,
and
these
the
this
general
difference
between
imperative
and
declarative
applies
to
how
you
how
you
specify,
where
your
source
is,
how
you
package
it,
how
you
configure
it,
how
you
deploy
it
the
whole
pipeline
of
of
deploying
and
specifying
an
application?
So
so
why
declarative?
B
B
Now
that
you've
had
now
that
you
have
a
valid
config,
you
can
do
one-click
deploys
using
the
tool
that
can
understand
that
declarative
configuration
and-
and
this
is
powerful,
because
the
person
doing
the
deploy
doesn't
have
to
know
anything
about
the
internals
of
the
application
we'll
dive
into
this
a
lot
more.
But
the
other
thing
about
declarative
config
is
that
because
it
doesn't
say
how
to
actually
how
to
actually
reach
the
end
state.
B
The
tooling
is
free
to
to
figure
out
how
to
do
that,
and
that
means
it
can
work
both
on
a
clean
slate,
but
also
on
a
somewhat
messy
state
that
may
be
existing.
So,
for
example,
you've
done
you've
installed
version
1
of
your
system
and
you've
applied
it
using
a
declarative
config,
and
you
want
to
do
an
upgrade
and
you've
done.
Some
arbitrary
set
of
changes,
you've
added
some
stuff,
you've
removed
some
stuff,
you've
changed
other
things.
B
Now
the
other
benefits
are
around
how
you
can
how
you
can
actually
use
the
artifact
of
the
configuration
itself.
In
other
words,
your
configs
can
live
in
in
a
git
repo
or
in
whatever
version
control
system
you're
using.
So
you
can
version
everything,
and
this
this
actually
has
benefits
other
than
versioning
itself.
So,
first
of
all,
it
lets
you
collaborate
between
various
teams
that
are
responsible
for
the
config,
so
maybe
developers
are
responsible
for
some
parts
operations
folks
are
responsible
for
others
and
they
can
all
collaborate
in
one
version
control
system.
B
Everyone's
changes
are
visible
to
everyone
else,
and
if
you,
if
you
make
sure
that
your
cluster,
for
example,
is
deployed
automatically
from
this
repository,
then
you
can
also
use
this
repository
as
a
as
a
sort
of
knowledge
base
for
what
your
cluster
state
is
or
should
be.
At
least
if
your
cluster
ever
deviates
from
that
state.
You
can
simply
apply
this
configuration
again
to
the
cluster,
so
you
can
do
all
these
little
automations.
That
map
the
the
repository
to
the
cluster
and
again
we'll
talk
about
that.
B
B
There's
nothing
there's
no
major
loss
here,
because
you
can
always
just
roll
back
to
a
previous
version
and
use
the
automation
that
deploys
from
this
repository
to
make
sure
the
cluster
also
rolls
back
to
the
previous
version.
So
you
version
your
application
and
your
configs,
especially
if
you
version
them
in
the
same
repo,
then
that
makes
rollback
really
easy,
and
this
general
idea
has
been
called
git
ops,
where
you
map
a
cluster
to
a
repo,
and
you
get
all
these
nice
benefits.
B
Another
little
benefit
and
we'll
go
into
this
in
the
demo.
Also,
is
that
once
you've
specified
your
configuration
declaratively
again,
the
tooling
is,
is
free
to
optimize.
How
exactly
that
lies
are
done
so
in
any
kind
of
developer
situation,
where
you
want
really
fast
feedback,
you
can
just
have
tools
that
watch
files
on
your
local
file
system
and
let
you
live
test,
so
you
update
a
functions
code
and
it
updates
immediately
in
a
cluster,
and
it
can
be
an
actual
real
cluster.
B
B
So
I'd
like
to
go
a
little
deeper
into
how
fission
itself
does
declarative.
Config
kubernetes
has
a
concept
called
custom
resources,
and
these
are
essentially
a
sort
of
arbitrary
extension
mechanism
for
the
kubernetes
api
and
it
lets
you
use
the
kubernetes
patterns
on
any
other
resources,
so
fission
resources
such
as
functions,
environments,
which
are
the
containers
that
run
those
functions
and
triggers
which
bind
the
external
events
into
function
locations.
B
All
of
these
things
are
specified
as
kubernetes
custom
resources
and
just
like
any
other
kubernetes
resource
like
a
deployment
or
a
service,
they
can
be
stored
as
yeah
Mille
or
json
files
stored
on
disk
version
controlled
and
so
on.
So
you
get
all
of
these
benefits
by
using
kubernetes
custom
resources.
B
Now
the
thing
about
about
these
yeah
Milland
JSON
files
is
that
nobody
really
wants
to
write
them
from
scratch.
Their
syntax
tends
to
grow.
The
amount
of
learning
you
have
to
do
tends
to
tends
to
only
increase.
Even
if
you
have
good
defaults,
you
sometimes
have
to
fill
out
quite
a
bit
of
boilerplate.
B
Anyone
familiar
with
kubernetes
Yama's
is
familiar
with
this
pane.
You
typically
copy
pasting
a
lot,
but
that's
a
little
error-prone,
so
with
fission.
What
we
did
is
we
we,
we
have
tools
that
create
the
initial
configuration
and
dump
it
in
a
tree,
and
then
all
you
have
to
do
is
kind
of
make
small
edits
to
the
yamen.
Instead
of
writing
it
from
scratch,
and
also
besides
the
custom
resources,
use
the
same
kind
of
declarative
idea
for
specifying
packaging
for
specifying,
where
application
source
code
lives
and
how
it's
packaged
and
uploaded
and
built,
and
so
on.
B
B
You
can
actually
apply
it
and
you
can
do
this
too,
to
any
kubernetes
cluster,
so
whether
it's
mini
cube
on
your
laptop
or
your
production
cluster.
You
use
the
same
command
to
deploy
your
application
and
that's
apply
your
specifications.
So
this
goes
through
all
the
steps.
It
packages
your
source
code,
it
uploads
this
to
the
cluster
depending
on
the
language.
There
may
be
a
build
pipeline
and
it
also
creates
upgrades
or
deletes
as
necessary,
the
kubernetes
resources,
the
functions,
environments
triggers
and
so
on.
B
So
we're
gonna
go
through
each
of
these
patterns
and
just
demo,
all
of
them
in
sequence.
At
the
end,
so
I'd
like
to
talk
a
little
bit
about
live,
reload
live
reload
is
familiar
in
in
some
domains
to
do
developers,
especially
web
development,
and
this
is
a
really
pleasant
kind
of
developer.
Experience,
you're
just
typing
your
source
code
somewhere
in
your
ID,
and
you
hit
the
Save
button
and
you
switch
to
either
a
browser
or
a
test
environment,
and
you
can
just
immediately
test
your
function.
B
B
So
what's
what's
the
point
of
doing
this,
the
the
real
one
of
the
the
ideas
of
DevOps
is
to
accelerate
feedback
loops,
and
it's
it's
consistently
true
at
various
stages
that
the
sooner
you
find
a
bug,
the
cheaper
it
is
to
fix-
and
this
is
quite
nonlinear.
So
if
you
find
a
bug
within
minutes
of
creating
it
of
of
writing
the
code,
then
all
you
have
to
do
is
you
know,
go
back
to
that
same
environment
and
fix
it.
But
if
you
find
it
say
three
months
after
you
wrote
it,
then
you've
probably
deployed
it.
B
It's
affected
users.
You
have
to
have
to
dig
deeper
into
into
why
the
bug
occurred.
Typically,
the
developer
has
sort
of
you
know,
paged
out
the
context
from
their
head,
so
so
really
live.
Reload
is
a
sort
of
big
benefit
in
being
able
to
create
good
quality
code
and
again,
any
any
system
that
has
declarative
configs
is
in
a
good
position
to
to
create
live,
reload,
tooling
and
again
coming
to
fission
the
same.
B
Apply
tools
can
be
used
to
to
do
live
reload,
so
you
just
add
the
watch
option
to
to
the
spec
apply
command
and,
and
this
this
runs
a
continuous
loop
where
it
watches
the
file
system.
It
gets
alerted
whenever
you
save
the
file
in
an
IDE
and
it
pushes
it
to
a
test
cluster
which
again
it
just
uses
your
Nettie's
configuration.
This
process
typically
takes
a
second
or
two.
Of
course.
It
depends
on
how
big
your
your
function
is.
B
How
big
your
changes
are,
how
much
rebuilds
need
to
be
done,
but
typically,
a
few
seconds
is
all
it
takes
and
a
note
that
you're
testing
on
a
real
kubernetes
cluster.
So
if
your
function
talks
to
a
database,
you
can
just
run
a
sort
of
test
instance
of
that
data
based
on
that
cluster
and
and
you
have
a
fairly
high
fidelity
test
going
on
so
again,
fast
feedback
leads
to
higher
quality
code,
continuing
this
of
being
able
to
test
and
debug
things.
B
So
the
idea
is
that
all
the
events
that
can
invoke
a
function-
you
record
them
in
some
form
in
some
sort
of
log
such
that
you
can
read
those
events
back
and
you
can
simulate
them
again
on
the
same
functions
or
on
even
on
different
functions
and
there's
two
broad
benefits
to
this.
You
can
test
so
you
can
replay
either
an
event
that
actually
worked
on
a
previous
version
and
let's
say
you
make
a
new
version.
B
You
want
to
make
sure
that
the
new
version
hasn't
broken
anything
that
used
to
work
in
the
old
version,
so
you
can
replay
a
good
set
of
events
that
used
to
work
on
the
old
version.
Essentially,
this
is
regression
testing
but
made
much
more
painless.
You
can
also
use
it
for
actually
testing
a
bug-fix.
So
if
anyone
broke
a
previous
version-
and
you
think
you
fixed
it-
you
can
verify
this
by
by
rerunning
that
event
on
a
newer
version.
So
the
regression
testing
and
bug
fix,
testing
and
also
a
really
cool
usage
is
debugging.
B
So
you
can,
if
you
find,
let's
say
that
any
event
broke
something
in
production,
but
you
you
can
almost
never
take
a
debugger
into
your
production
environment
and
attach
it
because
it
will
it'll
break
your
actual
production
environment.
But
you
can
take
a
recording
of
what
happened
in
the
production
environment
and
debug
it
in
in
a
separate
test
environment.
B
B
Things
that
that
make
this
idea
good
for
fast
one
is
that
fast
functions
are
typically
stateless,
so
they
don't
have
any
in
memory
state
they're
designed
to
be
able
to
to
be
able
to
be
terminated
and
restarted
at
any
point
they
kind
of
expect
to
be
short-lived,
and
because
of
that,
we
don't
have
to
replay
every
event
since
the
beginning
of
time.
The
state
of
function
is
pretty
much
determined
by
the
event
that
invokes
it
and
possibly
the
other
services
that
it
talks
to,
and
the
other
thing
is
that
phase
functions
are
short-lived.
B
So
if
you're
debugging
strategy
involved
going
into
an
instance
of
a
function
after
something
bad
has
happened
and
poking
around
at
some
state
and
trying
to
figure
out
what
went
wrong
in
the
function
that
doesn't
work
well,
because
the
faz
framework
may
have
killed
that
function
already.
So
there
may
be
nothing
left
to
inspect
so
record
replay
really
helps
you
with
stateless
things
that
may
that
may
not
be
alive
by
the
time
you
get
around
to
to
trying
to
debug
the
problem.
B
Broadly,
there's
two
use
cases
here:
a
developer
in
their
own
tight
change
and
test
cycle
can
use
record
replay
that
just
makes
testing
easier.
So
you
can,
for
example,
manually,
do
some
sort
of
interaction
with
your
function
with
recording
switched
on,
and
then
you
can
replay
that
all
of
the
events
that
occurred
there-
and
this
is
essentially
an
automated
way
to
to
do
testing,
and
the
other
way
is
that
operations
teams
can
enable
recording
on
some
sort
of
subset
of
production
traffic.
B
For
example,
you
can
record
only
the
requests
that
cause
errors
and
then
that
log
can
be
given
to
the
development
team
as
part
of
a
bug
report
and
the
dev
team
can
simply
take
that
log
dump
it
into
their
dev
cluster
and
reproduce
the
problem
and
debug
it.
They
can
even
verify
their
change
on
the
same
events
and
then
ship
that
change
back
to
the
operations
team.
B
So
up
until
now,
a
lot
of
these
features
have
been
mostly
dev,
optimizations,
making
devs
productive,
trying
to
increase
quality
by
reporting
bugs
faster,
but
this
last
idea
essentially
improves
collaboration
between
ops
and
devs
and
and
it
lets
ups
teams
give
far
more
precise
bug
reports
to
two
devs.
So
imagine
getting
paged
with
a
link
that
when
you
click
it
takes
it
straight
into
a
debugger
that
that's
already
loaded
with
the
event
that
broke
your
function
and
all
you
have
to
do
is
step
through
it.
B
So
in
fission
we
have
a
built
in
record
replay
system.
It
can
store
HTTP
requests
for
now
it's
mostly
based
on
HTTP.
It
can
also
also
store
the
responses,
and
then
you
can
do
filtering
based
on
that.
So
you
can
say
you
know,
find
me
the
recordings
that
failed
or
find
means
recordings
that
succeeded
or
find
me
the
recordings
that
took
a
long
time
so
replay
the
slow
requests
and
you
can
replay
on
demand
and
lets.
B
So,
even
after
all,
qualification,
it
is
a
good
practice
to
do
instead
of
shipping
a
new
version
to
100%
of
your
users,
to
just
take
it
to
deploy
it
to
the
cluster,
but
only
send
a
fraction
of
users
to
that
new
version,
and
often
these
can
be
some
sort
of
dog
food
users
or
users
who
have
signed
up
for
beta
and
so
on.
And
then
you
monitor
whether
your
API,
your
services,
are
succeeding
or
are
healthy
with
that
with
that
new
version.
B
So
if
that's
happening,
you
can
gradually
increase
that
percentage,
and
if
it's
not
happening,
you
can
roll
back.
So
just
just
to
explain
more
concretely,
if
you
have
a
version,
v1,
you've
tested
and
qualified
v2,
the
idea
is
to
deploy
both
v1
and
v2,
but
only
send
some
fraction
of
users
to
v2.
So
that's
the
idea
of
canneries
and
you
you
have
to
monitor
whether
v2
is
succeeding.
So
that
brings
us
to
to
the
next
idea,
which
is
that
if
you
have
to
monitor
whether
a
version
is
succeeding,
there's
there's
some
potential
for
automation
there.
B
So,
especially
in
a
fast
system
because
we're
running
specific
functions,
the
framework
knows
whether
that
function
succeeded
or
failed.
Usually,
so
we
can
define
some
sort
of
threshold
for
errors
and
say
that
if
function
is
erroring
out
a
lot
more
than
it
used
to,
then
we
should
not
proceed
with
the
candidate
deployment.
And,
conversely,
if
it's
doing
well,
then
we
should
keep
gradually
increasing
the
traffic
that
goes
into
that
function.
So
you
can
actually
automate
that
whole
loop
of
of
increasing
the
the
traffic
to
v2
and
fission
has
automated
kennedys
built
into
the
framework
you.
B
Even
though
it's
deployed-
and
at
some
point
you
define
you,
say:
okay,
let's
send
ten
percent
of
traffic
to
to
the
new
version.
So
now,
95
percent
is
going
to
the
old
version
and
ten
put
aside.
90
percent
is
going
to
the
old
one
and
ten
percent
to
the
new
one,
and
the
system
is
monitoring
whether
this
is
succeeding.
B
All
of
these
all
of
the
parts
of
this
picture
and
figure
able
so
the
the
width
of
that
step
is
the
interval
of
the
canary
deployment.
The
height
of
the
step
is
the
increment
that
occurs
at
each
step.
So
essentially,
you
can
define
how
fast
your
rollout
occurs
and
it
can
be
over
the
course
of
minutes
or
even
the
course
of
several
days
it.
B
Various
any
kind
of
production
environment
has
a
need
for
people
to
be
able
to
understand
the
system's
performance
where
it's
going
wrong,
and
this
is
a
this-
is
a
really
deep
topic
monitoring.
In
general,
we
can
talk
about
logs
metrics
alerts,
tracing
observability
in
general,
but
I'd
like
to
focus
on
just
two
or
three
things
and
how
they
can
help
service.
The
first
one
is:
is
log
aggregation.
This
one
is
essential
to
several
assistants
because
functions
are
short-lived.
B
So
if
a
function
just
logs
to
some
sort
of
local
storage,
that
storage
is
likely
to
have
disappeared
by
the
time
a
developer
or
an
Operations
person
tries
to
look
at
those
logs.
So
you
need
to
take
these
logs
and
ship
them
somewhere.
There's
good
open
source
tools
for
this
flu
and
D
is
really
common
in
the
kubernetes
world.
B
It
has
integrations
that
that
can
ship
logs
to
almost
every
commonly
used
log
aggregation
system
elastic,
is
a
popular
open
source
stack
for
for
shipping
logs
in
to
and
fission
integrates
with
flu
and
D
and
configure
set
such
that
flu.
India
is
shipping
out
your
logs
from
from
your
function,
so
you
don't
really
have
to
worry
about
the
function
going
away.
You
can
always
access
its
logs.
B
B
We
already
talked
about
log
aggregation,
efficient
tracks,
various
things
about
a
function
since,
since
it
routes
requests
into
your
function,
it
knows
how
long
that
request
took.
It
knows
whether
that
request,
succeeded
or
failed.
So
your
function
itself
does
not
have
to
write
any
kind
of
Prometheus
integration.
It's
built
into
fish
in
itself
and
if
you
install
prometheus
in
the
same
cluster
as
vision,
it
finds
the
cluster
it
finds
the
fish
and
deployment
using
annotations
and
it
just
stores
all
these
metrics.
So
you
can
keep
track
of
your
function.
B
I'm
gonna
dive
a
little
bit
into
the
topic
of
cost
optimization.
The
general
idea
is
that
almost
every
system
has
cost
performance
trade-offs
in
the
public
cloud.
Several
assistants,
like
lambda,
let
you
pay
exactly
for
what
you
use
typically
build
in
some
sort
of
sub
second
intervals,
but
you
pay
a
premium
for
this
elasticity.
The
trade-offs
get
worse
as
your
usage
gets
higher.
B
B
In
general.
You
always
want
to
carefully
configure
your
resource
limits
of
whatever
you're
running
in
a
cluster,
whether
it's
public
private
and
that
allows
order
scaling
to
be
more
efficient.
So
schedulers
can
actually
plan
placement
more
correctly
when
your
resource
limits
are
are
accurate,
and
if
you
have
auto
scaling
on
the
cluster
itself,
then
you
can
actually
grow
the
cluster
and
the
utilization
within
the
cluster
as
demand
grows,
and
this
lets
you
actually
map
your
operational
expenses
to
resource
usage.
B
I
want
to
dive
a
little
bit
deeper
into
one
specific
cost,
optimization,
which
is
really
important
in
the
survey
less
world,
and
that's
the
that's,
the
one
that
makes
idle
services
free.
So
if
a
service
has
zero
usage,
let's
say
every
weekend
or
or
if
it's
used
very
intermittently,
then
we
don't
want
to
be
paying
us
even
a
small,
constant
fee
for
that
service.
B
B
So
this
this
is
the
goal
start
problem
in
fission.
We
solve
this
cold
start
problem
with
a
pool
of
containers
that
are
that
are
already
running
on
the
cluster.
There's
lots
of
configurable
and
tunable
bits
here,
I'm
just
going
to
point
out
the
documentation
in
the
end,
but
essentially
there
is
there's
a
lot
of
tuning
that
can
be
done
here
and
you
can.
B
You
can
move
between
points
on
the
cost
performance
trade-off,
so
you
can
have
functions
that
don't
use
the
pool
at
all
and
have
a
high
latency
to
start
up
or
you
can
use
functions
that
are
always
running
where
you
do
pay
a
certain
constant
cost.
Even
if
the
function
is
idle
but
there's
very
low
latency.
B
Interestingly,
because
you're
using
an
open
source,
stack,
kubernetes
and
fission,
you're,
not
subject
to
the
lambda
a
pricing
model.
So
if
you're,
if
your
workload
doesn't
fit
the
the
use,
the
users
that
lambda
optimizes
for
so,
for
example,
if
you
have
a
batch
workload
or
you're
using
your
functions
for
almost
all
day
or
even
several
hours
a
day,
you
can
use
the
cheapest
possible
instance
that
works
for
you.
So
you
can
use
a
reserved
instances
if
you're
using
your
functions
a
lot
and
you
can
use
spart
instances.
B
More
generally,
fishing
lets
you
configure
CPU
and
memory
resource
usage
on
your
functions.
You
can
also
also
to
various
art
of
scaling
parameters
such
as
when
you
actually
trigger
increase
in
the
number
of
replicas,
and
when
you
trigger
reduction,
you
can
also
trigger
the
start
and
end
of
those.
So
if
you
set
minimum
scale
to
2-0
your,
you
actually
have
idle
equals
free.
If
you
set
it
more
than
zero,
you
you've
guaranteed
a
low
latency
for
your
api's
okay.
B
So
we
talked
about
a
lot
of
things:
I'd
like
to
demo,
some
of
them
and
I'm
going
to
quickly
do
a
little
hello
world
talk
about
declarative,
configs
again
and
show
how
they
work
in
fission.
In
particular,
I'm
gonna
do
a
quick
little
live
reload
demo
I'm,
going
to
show
you
a
small
record
replay
demo
and
then
we'll
kick
off
a
cannery
deployment
on
a
cluster.
So,
let's
see
here
we
go
okay,
so
the
first
one
is,
let's
just
start
with
hello:
world.
B
B
Here's
a
simple
hello
world
JavaScript
function,
it's
written
in
modern
JavaScript
using
async
await,
and
it
receives
a
context
which
contains
the
event
that
actually
caused
that
function
to
be
invoked
and
it
can
return
any
status
that
that
gets
converted
to
the
HTTP
status
then
can
return
a
body
and
there's
other
ways
to
return.
But
this
is
a
simple
hello
world.
So
so
we
use
the
simplest
way.
B
We
create
efficient
environment,
so
efficient
environment
is
essentially
the
runtime
container.
That
knows
how
to
run
that
function.
All
of
the
components
in
fission
are
language
agnostic,
except
the
environments
which,
each
of
which
is
specific
to
a
certain
language.
So
this
one
failed
because
I
had
already
pre-created
it
and
its
parts
are
have
been
running
for
a
while
on
the
kubernetes
cluster.
These
parts
are
idle;
they
they
haven't
loaded
the
function
yet
they're,
just
waiting
for
for
a
function
to
be
loaded,
so
they're
already
running
they're
deployed
they
have
an
IP
address.
B
B
Route,
this
essentially
says
when
you
do
a
get
request
on
the
/hello
path
to
the
fission
router
service.
You
invoke
the
function,
hello,
so
let's
actually
run
the
function
and
in
the
function
a
few
times,
the
first
time
it
takes
about
half
a
second,
the
second
time
it
takes
about
150
milliseconds-
and
this
is
in
the
statistical
test.
B
There's
some
variability
here,
but
typically
the
the
first
one
is
where
you
incur
the
code
start
overhead
and
subsequent
requests
are
just
hitting
a
cached
function
and
the
difference
between
these
two
is
your
cold
start
overhead,
there's
better
ways
to
measure
this
using
Prometheus,
but
this
is
just
hello
world.
Okay,
so
if
you
notice
here,
we
created
functions
using
using
an
imperative
model.
We
said,
create
this
function,
upload
this
source
code
and
in
fact,
when,
if
the
function
had
already
existed,
that
that
would
have
failed.
So
this
isn't
a
very
automation,
friendly
style.
B
B
Stepping
back
to
this
hello,
world
demo
I
forgot
to
tell
you
that
the
part
is
now
actually
labeled
with
that
function
name,
so
you
can
use
usual
kubernetes
tools
to
find
that
part.
If
you
have
any
kind
of
external
log
aggregation
system
like
say,
stack,
driver
on
google
or
at
AQL
and
sun
on
amazon,
you
can
you
can
search
for
this
function
by
label
and
you
will
find
your
loss.
Fusion
also
builds
in
a
little
log
database
optionally.
B
B
Do
the
same
thing,
except
that
we
passed
a
spec
option,
so
this
does
not
affect
the
cluster
in
any
way,
but
it's
the
same
command
that
you
ran
to
create
your
function.
This
time
the
function
is
in
go
so
so
I
can
show
you
how
builds
work.
This
is
a
yeah.
You
don't
really
need
to
understand
it.
Unless,
for
example,
you
want
to
change
execution
and
order
scaling
parameters,
but
most
users
can
can
get
away
without
really
understanding
most
of
that
llamó.
B
So
now
we
we
actually
apply
that
function.
Note
that
we
got
a
little
warning
because
we
did
not
specify
the
environment
in
the
same
specification.
So
we
get
a
little
consistency
warning
saying:
hey
this!
This
is
not
a
good
practice.
If
you
want
your
application
to
be
fully
self-contained,
then
your
environment
should
also
be
specified
in
this.
B
In
this
configuration
and
if
you're
doing
something
more
production-ready,
that's
you
probably
want
to
make
sure
you
get
rid
of
all
these
warnings
by
by
doing
by
doing
what
they
suggest
here
we
see
the
build,
has
actually
succeeded,
so
we
can
actually
invoke
that
function.
It's
again.
It's
just
a
hello
world.
All
of
the
demos
in
this
little
demo
today
are
hello
world,
so
yeah
fission
function
test
is
really
the
same
as
curl.
It
just
invokes
the
function
and
gives
you
the
result
back.
B
B
I'm
going
to
kill,
then
I'm
gonna
just
invoke
the
function
manually
again
to
show
that
it
actually
worked
actually
works.
Okay,
so
you
notice
that
I
changed
the
source
code
there
I
did
not
manually
invoke
any
kind
of
build.
Simply
the
live.
Reload
system
noticed
that
the
file
changed
it
did
invoke
the
build
the
build
is
part
of
the
built-in
fishing
go
environment
which
I
had
pre
created
and
already
exists
on
this
on
this
framework.
B
So
I'm,
sorry
on
this
deployment,
so
there's
a
go:
node.js
Python
all
already
set
up
on
this
environment
and
that's
what
allowed
the
function
to
be
built.
So
that's
so.
That's
live
reload,
I'm,
gonna,
I'm
gonna,
move
on
from
here
to
record
replay
again
trying
to
make
sure
we
have
time
for
questions,
let's
see.
B
So
this
is
again
a
really
simple
demo
that
this
time
you
didn't
buy
thon
and
it's
a
function
that
takes
a
query,
parameter
and
says
hello,
so
it
you
can
imagine
it
as
the
target
of
a
form
that
asked
you
your
name,
so
we
create
that
function,
set
up
a
root,
and
now
we
enabled
recording
for
this
function.
So
this
is
just
a
symbol
recorder.
It
records
every
single
event
that
invokes
that
function
so
now
I
provide
an
argument
to
two
that
functions.
B
Saying
name
equals
world
and
that
essentially,
is
hello
world
again,
so
you
can
actually
view
your
recording
here
and
you
can
replay
that
recording.
So
this
is
the
recording.
It
actually
tells
you
that
it
was
a
get.
They
ran
the
function
hi
that
was
two
hundred
okay.
This
is
the
HTTP
trigger
that
caused
the
invocation
of
that
function.
So
everything
is
recorded.
The
response
is
also
recorded,
but
isn't
shown
here,
and
we
can
simply
replay
that
request
by
by
request
ID.
So
this
is
a
simple
demo.
B
The
the
replay
is
replayed
on
the
same
function,
but
you
can
also
just
modify
the
function
and
replay
it,
and
then
you
can
use.
Therefore,
regression
testing
or
debugging,
that
is
there's
also
an
unreleased
bit,
will
be
where
you
can
simply
pass
debug
and
to
drop
you
in
it'll
it'll
output,
a
debugger
configuration
for
an
IDE,
and
you
can
just
open
your
ID
and
start
connecting
to
a
remote
debugger
and
debug
your
your
function,
which
is
already
in
the
environment
that
that
caused
your
error.
B
So
again
we
have
a
fission
cluster
kubernetes
cluster
nodejs
is
set
up
and
we
have
two
functions.
So
that's
version
one
and
this
version
two.
They
don't
do
anything
significantly
different
other
than
tell
you
what
version
they
are
so
so
this
is
the
interesting
part.
We
create
a
route
which,
to
the
outside
world,
looks
just
like
any
other
route.
I'll
get
request
to
a
certain
URL,
except
that
we
define
two
functions,
not
one
and
we
give
them
weights.
B
B
So
here
we
can
see,
the
traffic
starts
coming
in
the
red
function
is
v1
and
the
blue
line
is
v2
since
we're
short
on
time,
I'm
not
gonna,
wait
for
the
entire
cannery
to
actually
complete,
but
just
to
quickly
show
you
what's
possible
there
is,
you
can
create
a
configuration
you
can
give
it
the
two
versions,
your
current
version
and
your
last
version.
You
can
specify
what
route
to
actually
update
and
you
can
specify
the
speed
at
which
that
that
switchover
should
happen.
B
B
There's
a
cut
over
point
at
which
they're
close
to
50%
that
keeps
progressing
because
both
functions,
because
because
v2
is
successful
here
and
when
that,
when
the
intervals
over
v2
has
a
hundred
percent
of
traffic,
so
this
is
still
going
on
in
the
new
one,
but
I'd
like
to
quickly
switch
back
to
a
few
links
that
that
you
can
use
to
follow
up
I'm
not
going
to
go
too
deep
into
fish
in
itself.
But
there's
functions.
Environments
triggers
I
already
spoke
about
this
a
bit
we
can.
B
A
Awesome
thanks
for
the
great
presentation,
let's
try
and
get
to
a
few
questions
within
the
next
five
minutes,
and
then,
if
we
don't
get
to
any
we'll
make
sure
everyone
gets
their
answer
after
the
fact.
Yes,
so
how
does
Beck
handle
ingress
annotations
with
respect
to
fission
function
such
as
north-south
via
an
API
gateway,
yeah.
B
So
so
this
is
kind
of
almost
any
kubernetes
resource
can
be,
can
be
specified,
declaratively
right.
The
ingress
answer
is
a
little
bit
longer.
So
one
thing
we
we
did
with
our
HTTP
triggers
is
we
have
a
little
flag
that
that
you
can
just
set
to
true
and
any
HTTP
trigger
with
that
flag
set
to
true,
creates
an
ingress
resource
automatically.
So
there's
a
fission
controller
running
that's
watching
these
resources
and
creates
an
increase
resource.
B
B
So
today
they
are
not
well
you
can
you
can
set
up
again.
This
is
the
same
question.
The
the
Redis
setup
can
be
installed
with
persistence
or
without
it
you
can
also
dump
them
from
the
CLI.
And
then
you
know
you
can
have
custom
tooling
that
that
works
with
those
recordings.
So
you
can
create
a
test
from
it
load
it
back
into
another
cluster
and
replay
them,
but
today
the
answer
is,
you
can
just
set
up
greatest
persistence.
B
Thanks
for
asking
that
the
fission
environments
have
actually
have
two
containers
that
that
are
part
of
them.
There's
the
runtime
and
the
build
time-
and
you
saw
this
a
little
bit
with
go
where
we,
where
we
do
builds.
So
this
any
function
has
two
archives
associated
with
the
source
and
the
deployment
and
the
source
can
specify
requirements
like
in
Python.
B
You
can
have
a
requirements
or
txt
no
Jess,
you
can
have
packaged
or
JSON,
and
each
language
has
some
sort
of
equivalent
and
the
builder
will
take
care
of
actually
fetching
those
creating
an
artifact
from
that.
Storing
it.
I
can
point
a
documentation
for
for
how
to
actually
use
this
stuff
and
they're
examples
in
on
github
under
the
examples
directory
for
for
both
Python
node
and
go
so
so
yeah
dependencies
can
be
automatically
pulled.
B
The
other
way
is
to
actually
make
custom
environment,
so
you
can
store
them
in
your
container,
and
this
tends
to
be
a
kind
of
Finnish
use
case
like
if
you
have
some
sort
of
binary
that
isn't
present
in
in
in
the
in
the
repositories
of
those
languages.
But
yeah
dependencies
are
well
supported
through
the
builders.
B
A
B
A
All
right,
so
we
are
at
the
top
of
the
hour
now.
So
thank
you.
Everyone
for
joining
us
today
just
a
reminder
that
the
recording
and
slides
will
be
online
within
24
hours
and
thank
you
so
much
for
a
great
presentation
and
we
look
forward
to
seeing
everyone
at
a
future
of
CMC
s.
Webinar
have
a
great
day
the
one
thank.