►
From YouTube: Pontoon: An Enterprise Grade Serverless Framework Using Kubernetes – As Used in VMware
Description
Pontoon: An Enterprise Grade Serverless Framework Using Kubernetes – As Used in VMware Cloud Services [I] - Kumar Gaurav & Mageshwaran Rajendran, VMware
In VMware Cloud services, we perform both batch and real-time computations based on periodic schedules and on-demand events, using our in-house developed serverless framework called Pontoon. This provides better utilization of resources and enables our service developers to write serverless functions with simple declarations.
Kubernetes provides Jobs and Deployments as design constructs to handle such needs, while other frameworks like IronIO Functions, Fabric8, et al aim to solve end-to-end use case . However, we had to extend on top of Kubernetes Jobs & Deployments to define the packaging and I/O interactions of the function, implement a priority queue for execution, and provide declarative retry policy while ensuring high availability. A developer 'writes' a function supporting common EAI patterns for start time parameterized variables, and defines it's packaging and scheduling using a yaml file. The framework then packages it as a Container alongwith an 'observer' container in a pod, 'registers' it with the scheduler while ensuring choice of 'warm' vs on-demand requisite replicas of the pod, and then through a 'Scalar' manages the execution and life cycle of job, while logging and tracing failures/success.
This framework is in use over months in VMware Cloud services and we are now open sourcing it.
About Kumar Gaurav
Kumar Gaurav is working on the first set of services under VMware Cloud Services umbrella, a SaaS offering. He is a veteran in VMware, having built many cloud management products over 9 years and holds dozens of US patents, and few academic publications in Container space. He is the host and organizer of Kubernetes meetup Bangalore chapter. He works out of VMware Bangalore office, as Director R&D.
About Mageshwaran Rajendran
Mageshwaran Rajendran is a lead designer and co-architect of Cost Insight- one of the service under VMware Cloud Services SaaS offering. He has earlier built big data based batch & real-time data pipelines handling TB’s data for financial institution and distributed applications handling high concurrency with specific focus on code quality and long-term maintainability. He works out of VMware Bangalore office, as Staff Engineer.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
A
This
may
be
a
bit
of
repeat
if
you
have
attended
our
earlier
well
today,
we
had
two
plus
one
three
sessions
already
gone
by
on
server
less
in
the
same
room,
so
it
may
be
bit
of
an
overdose.
We
have
one
more
coming
after
my
talk,
but
in
very
quick
to
recap:
serverless
enables
running
your
back-end
logic
without
you
managing
this
servers,
and
it
also
gives
you
instantaneous
scalability,
because
most
of
the
things
are
event-driven.
Of
course,
it
puts
some
constraints
that
you
have
to
think
about.
Especially
the
process
should
have
a
particular
finish
time.
A
It
should
have.
It
should
be
ephemeral.
It
should
also
be
stateless
because,
once
the
execution
is
done,
you
will
lose
everything
which
is
maintained
there
and
good
practice
try
to
have
minimal
shares
if
at
all,
no
shares
across
your
execution,
states
and
typical
use
cases
for
scheduling,
tasks,
dynamic
and
burstable
workloads
message,
driven
applications.
A
By
the,
if
you're
thinking
that
cyber
this
is
a
new
paradigm,
it's
actually
not.
It
has
been
well
there
for
quite
some
time
it
just
like
we
are
catching
up
at
catching
up,
add
on
to
it
as
a
trend
and
with
trend.
There
is
lot
of
community
enthusiasm
and
hence
good
and
proper
tools
which
you
can
use
for.
Writing
your
applications,
and
we
hope
that
it's
not
yet
another
hype
cycle,
but
rather
this
is
something
which
is
there
to
stay,
because
we
are
me
as
it
company
for
the
work
that
I
do.
A
The
major
benefit
is
because
of
the
economics
you
will
see
you
should
you
should
be
able
to
fairly
reduce
your
overall
cloud
expend
because
you
are
going
to
pay
only
when
you
have
a
real
usage
of
the
hardware
underlying
it
and
the
projections
are
looking
pretty
good.
Most
of
the
workloads
will
move
towards
this
direction
sooner
or
later,
and
hence
we
have
all
the
big
names,
all
the
cloud
vendors
coming
up
with
a
native
implementation
of
server
less.
A
So,
with
this
brief
upon
our
agenda,
we'll
start
with
the
needs,
then
we'll
translate
the
requirements
and
then
I
will
hand
over
to
my
colleague
who
will
go
over
the
pontoon
architecture
whew
and
the
last
key
takeaways
we'll
also
have
a
live
demo.
So
what
is
our
need
here?
So
the
two
of
us
mean
you
can
call
me
cagey
by
my
initials
and
my
colleague
magesh.
We
work
in
the
VMware
cloud
services.
It
says
it
is
multi
cloud
management
solution
which
combines
the
capabilities
of
application,
orchestration
monitoring
cost
inside
and
network
inside.
A
We
shape
our
binaries
in
docker
container
packaging
for
all
the
goodness
that
it
comes
with,
and
we
also
use
kubernetes
heavily
as
an
orchestration
layer.
We
do
run
both
on
AWS
cloud
as
well
as
vSphere
private
cloud.
That
means
sorry.
We
can't
use
AWS
lambda
few
months
ago.
We
realized
that
there
is
a
need
for
us
to
use
a
framework
to
run
both
schedule:
job
on-demand
jobs,
as
well
as
some
of
the
bursting
applications.
A
We
did
evaluate
a
few
of
the
popular
ones
for
us
to
see
if
we
can
readily
consume
them,
but
we
have
to
finally
build
up
roundup,
which,
which
is
what
we
call
as
first
pontoon.
It
was
fairly
easy
to
build.
We
were
able
to
use
it
as
a
pattern
on
top
of
kubernetes
jobs,
and
it
took
hardly
2
to
3
engineering
effort.
I
mean
one
person,
engineering
effort
for
two
to
three
months
to.
A
And
before
we
proceed
further,
I
will
like
you
to
keep
an
eye
upon
why
we
are
building
it
because
we
primarily
build
it
for
self
consumption.
So
we
are
solving
it
for
self.
It
may
not
cover
all
the
use
cases
which
we
will
highlight
and
important
disclaimer
and
like
Google
Android
ad,
we
don't
get
paid
for
solving
technical
problem,
but
only
business
problem.
So
that's
a
cap
on
engineering
investments
we
can
make.
But,
yes,
we
are
pretty
happy
with
the
end
result
which
we
have
so
changing
gears.
A
Now,
let's,
let's
look
into
the
requirements
which
we
came
up
with,
so
we
have
two
sets
of
requirement.
One
is
the
set
of
all
functions,
sorry,
the
serverless
functions,
and
that
is
the
server
list
framework
itself.
So
we
came
up
with
these
six
requirements.
The
first
one
says
a
process
function
is
a
process
which
is
capable
of
running
business
logic
in
response
to
an
event.
It
should
be
finite,
so
you
you,
you
can
preempt
on
on
time
outs.
It
should
be,
of
course,
stateless.
A
That
is
anything
it
has
to
person
should
be
in
shared
persistence,
storage
and
good
practice.
It
should
have
shared
nothing
architecture
between
executions.
We
don't
impose
any
language
constraints
on
containers,
but
we
know
that
we
will
be
fairly
using
JVM
Python
and
maybe
go
at
it
at
some
point
in
time.
So
container
was
a
default
choice,
whereas
runtime
of
our
functions
is
not
exactly
in
millisecond.
It
is
built
to
cater
to
an
order
of
few
seconds
to
a
few
minutes,
and
hence
container
was
absolutely
fine
for
us
and
though
our
function
has
to
be
idempotent.
A
Second,
set
of
requirements
quickly
getting
through
is
on
the
survivalist
framework
itself.
So
since
we
are
heavily
built
upon
kubernetes,
we
wanted
it
to
be
a
sort
of
facade
on
the
top
of
it.
We
should
be
able
to
persist
the
logs
and
also
do
root
cause
analysis
when
needed
function
should
be
able
to
access
other
kubernetes
resources.
This
is
a
very,
very
important
requirement
for
us,
because
when
we
very
designing
these
functions,
we
knew
that
they
are
going
to
run
for
quite
some
time.
A
They
will
interact
using
kubernetes
network
constructs
with
other
live
services
available,
and
hence
it
should.
It
should
be
able
to
access
those
resources.
We
also
have
a
configurable
way
to
say
what
should
be
the
retry
limit
for
a
function
and
how
many
such
parallel
invocations
should
run.
We
can't,
of
course,
assume
infinite
parallelism
because
it,
after
all
cost
money.
We
do
limit
our
resources
on
AWS
using
reserved
instances,
and
hence
we
know
that
we
have
to
limit
the
parallelism
to
a
great
extent.
A
It
should
be
highly
available
and
the
execution
of
function
should
be
visible
in.
In
today's
terms,
we
call
it
observe
availability.
We
should
guarantee
at
least
one
execution
of
each
function.
Exactly
once
is
desired,
but
may
not
eventually
happen.
There
can
be
cases
where
two
invocations
will
also
be
fired
and,
lastly,
in
developer
friendliness,
so
ability
to
register
function
with
an
API
and
also,
let
me
call
out
the
non
goals
for
us.
We
haven't
seen
a
need
for
us
to
do
sort
of
chaining
or
workflows
of
functions.
A
If
I
translate
these
requirements
into
a
bit
of
pictorial
diagram,
it
will
look
like
this.
We
have
a
concurrent
job
store
and
also
along
with
that
disk,
back
cue
implementation.
Let's
call
it
a
paretic
implemented.
We
also
have
a
pod
scheduler,
which
keeps
a
track
upon
how
many
open
jobs
are
there
at
an
at
a
moment
and
hence
how
many
pods
should
be
there
in
the
system
to
serviced
them.
A
Then
we
need
to
have
an
orchestrated,
which
is
the
jobscheduler.
Linamar
was
partially.
This
is
job
scheduler,
which
takes
input,
requests
through
a
pis
and
does
the
following
important
tasks
for
adding
jobs
to
the
queue
managing
the
jobs
when
they
are
done,
or
even
in
case
of
error,
again
put
them
back
into
the
queue
based
upon
some
policy.
Then
we
have
on
this
on
this
side
the
pods
themselves.
So
we
have
put
sidecar
container
pattern.
That
is
the
init
container,
which
helps
the
business
logic
worker
container
to
do
the
job.
A
This
init
container
consumes
the
jobs
with
a
set
of
priority
queue,
lengths
from
the
concurrent
store
and
keeps
on
updating
the
progress
of
the
job,
so
whenever
it
picks
a
job,
it
should
update
that
this
particular
container
ID
is
taking
up
this
job
and
on
successful
completion.
It
should
mention
what
is
the
exit
code
so
that
it
doesn't
get
retried
or
if
it
has
to
be
retried,
then
what
is
the
error
code
for
that?
It
has
to
maintain
all
of
it
and
then,
lastly,
our
business
logic
worker.
A
Few
questions
that
we
should
be
thinking
at
this
moment
is
how
do
we
package
these
functions?
So
there
is
a
full
CI
CD
pipeline,
which
is
expected
to
do
the
part
definition
right
from
the
goal
right
from
the
point
when
a
developer
says
that
this
is
a
function,
and
this
is
the
runtime
packaging
of
the
docker
container
around
it.
A
And,
of
course,
there
is
respect
for
our
developers
how
they
have
been
to
publish
a
particular
container
and
before
we
ventured
onto
and
building
pontoon,
we
did
comprehensive
analysis
upon
what's
existing
there
and
out
in
the
market.
So
we
look
forward
to
a
few
of
the
dimensions
like
native
cup
kubernetes
integration,
access
to
kubernetes
cluster
resources,
what
type
of
packaging
our
language
bindings
does
this
technologies
support,
event,
monitoring,
persistence,
logs
and
so
on,
and
we
evaluated
four
of
them
are
in
I/o,
fabricate
efficient
and
open
risk.
A
If
you
skip
to
all
of
this,
but
just
need
read
the
last
lines
for
the
conspired.
We
thought
that
we
have
a
good
handle
with
fabricate
function,
but
this
was
up
approximately
about
the
time
when
fabricate
was
also
sandbox.
So
Red
Hat
said
that
if
they
are
no
longer
going
to
support
it,
but
rather
move
towards
open
risk
or
even
Qube
less
so
at
this
moment
we
decided
that
let's
invest
in
our
one.
B
So,
after
all
the
evaluation,
we
decided
to
go
ahead
with
pontoon.
So
what
is
it?
It's
a
a
tide.
Holo
structures
basically
helps
to
float
in
water.
So
what
are
the
major
characteristics
of
a
pontoon?
Is
it's
lightweight
it
built
for
a
purpose
and
it's
versatile?
So
what
do
we
mean
by
like
that?
So
the
whole
point
on
infrastructure
can
be
deployed
in
your
cluster
using
a
single
deployment
file.
B
It's
cube
apply
for
a
deployment
deployment,
Yama
file,
nothing
else,
and
it's
built
considering
the
scenario
that
you
already
have
a
cube,
Nettie's
cluster,
which
was
our
case.
That
was
productive
productionize
for
more
than
a
year
and
we
quickly
needed
a
solution
and
it
it
works.
So
it
helps
to
float
the
functions
around.
B
So
we
had
lot
of
white
boring
while
designing
pontoon,
and
we
took
few
major
choices,
the
first
being,
even
though,
to
support
polygons,
we
standardized,
we
decided
to
use
containers,
we
said
pontoon
will
not
directly
interact
with
Docherty
directly
right,
so
we
don't
want
to
handle
deigned
or
doctor
inside
docker
or
dock
or
on
top
of
docker.
We
do
not
want
to
handle
how
to
clean
up
the
image
once
it's
been
executed
or
if
you
would
under
docker
with
then
given
a
new
name.
B
How
does
your
monitoring
tools
understand
it
if
you
wanna
debug
how
you
can
get
inside
a
container
dynamically?
So
we
said
we
are
not
going
to
deal
with
all
those
things.
Rather,
we
will
go
with
Cuban.
It
is
very
communities.
Job
and
deployment
looks
like
solving
all
the
problem
for
us,
so
Cuban
it
is
job,
so
it
provides
two
awesome
abilities.
B
It's
very
easy
for
us
to
say.
Function
can
scale
to
a
great
extent
like
event
scale,
but
the
real
problem
comes
with
respect
to
what
your
function
is
actually
doing.
If
your
function
is
depending
on
few
databases
to
retrieve
data,
you
are
limited
based
on
the
concurrency
that
your
database
can
hold,
so
it
became
for
us
very
what
it
became.
It's
very
important
for
you
to
limit
how
many
functions
can
be
executed
parallely.
We
cannot
don't
play
that
aspect.
B
So
what
what
community's
job
doesn't
do?
Our
let's
user
to
do,
is
how
to
actually
do
the
message
passing
if
your
parts
are
running
five.
Concurrently,
it's
going
to
run
like
hundred
times
what
are
the
corresponding
inputs
that
you
have
to
pass
through
the
pod,
so
that
is
left
to
the
users
and
ponton
uses
that
effectively.
B
The
pontoon
is
written
in
Java,
it's
not
in
golang,
and
we
do
not
aim
for
a
pure
play
function
as
a
service.
If
you
are
thinking
of
like
and
and
security
multi-tenancy,
those
things
are
not
in,
we
hugely
depend
on
cuban,
it
is
cluster
for
auth
and
so
on
so
forth.
So
let's
see
how
pontoon
works
right,
so
it
has
three
major
components.
The
first
one
is
the
pontoon
API
set
were
so.
This
is
the
only
state
full
component
in
the
whole
architecture
and
it's
built
using
Zenon
micro
service
framework.
So
it's
highly
available.
B
B
Okay,
it
primarily
holds
meta
data,
so
how
to
register
a
function.
It's
a
test
API
where
you
define
what
your
function
is
like
a
docker
image
and
you
register
it,
then
how
to
trigger
a
function
or
vendors,
a
function
triggers
whenever
an
event
is
been
thrown
so
event
can
be
registered
by
firing
a
post
API.
So
the
event
goes
and
sits
in
the
event
queue.
This
is
a
priority
queue
and
it's
built
on
top
of
Zenon
itself,
so
the
priority
is
applicable
only
within
a
type
of
function.
B
So,
if
you
have
three
different
functions,
priorities
are
effective
only
within
a
function
not
across
functions.
So
the
next
component
is
this
scalar,
which
keeps
on
monitoring
the
event
queue
so
by
looking
at
the
event
queue
it
understands
what
are
the
pending
events
that
it
has
to
process.
It
also
looks
at
the
capabilities,
API
server,
to
figure
out
the
status
of
the
jobs
and
deployments
that
it
has
previously
created.
So
now
it
has
the
view
of
what
is
pending
and
what
are
the
available
slots
by
looking
at
what
is
already
executing.
It
can
detect
that.
B
Okay,
the
function
is
configured
for
fie
parallelism,
whereas
the
jobs
are
already
done
in
with
two
concurrency,
which
means
when
it's
creating
a
new
definition.
It
has
to
run
only
with
three
concurrency,
so
it
has
a
basic
that
logic
and
it
creates
a
job
definition
dynamically
and
it
interacts
with
the
API
server
to
create
the
job
definitions.
Now,
when
a
job
is
deployed,
you
can
see
it
not
only
has
the
function
reported,
but
it
also
has
a
sidecar
watcher.
So
the
watcher
keeps
on
watching
the
reporter
to
say.
B
Okay,
you
are
supposed
to
execute
for
five
minutes,
but
you
are
going
beyond
the
five
minutes.
So
let
me
time
it
out,
so
it
does
a
mini
lifecycle,
management
of
the
function,
and
it
also
speaks
to
the
API
server
to
get
the
event.
So
when
pods
are
executing
parallely,
you
need
someone
to
ensure
I
event
is
been
picked
by
only
one
person
at
a
time
basically
acquire
a
lock,
and
if
your
function
goes
crazy,
you
have
the
release
a
lock
so
that
it
can
be
retried,
so
the
those
jobs
are
big
by
watcher.
B
B
I
am
done
or
failed
to
indicate
success
or
failure,
so
the
every
other
logic
has
been
wrapped
inside
the
sidecar
watcher.
So
now
we
have
a
full
end-to-end
flow
executing,
so
the
same
thing
is
applicable
for
the
warm
notes,
with
a
difference
being
instead
of
job.
We
now
create
deployment
and
instead
of
a
normal
container,
we
need
to
have
a
warm
container,
which
has
some
more
logic
to
simulate
the
completions,
that's
provided
by
cubing
at
this
job
and
we
again
use
a
cube
annuity
scale-up
to
determine
the
parallel
concurrency.
B
Okay,
so
let's
see
what
are
the
other
capabilities
that's
available
in
Ponto,
so
you
can
define
functions
and
group
multiple
functions
in
a
single
app.
We
have
the
ability
to
pass
static
configurations
both
at
an
app
level
and
at
a
function
level.
Those
configurations
can
be
received
as
a
header
or
as
an
environment
variable.
So
you
have
a
ability
to
configure
retry
number
of
parallelism
that
you
want
to
achieve.
You
also
have
able
to
define
a
cron
to
create
these
events
automatically.
B
So
if
you
want
to
create
your
function
can
be
invoked
only
by
eye
on
the
event
and
events
can
be
created
by
setting
up
a
cron,
see
every
five
minutes
create
an
event
of
this
particular
type
and
it
integrates
perfectly
with
open
tracing.
So
if
incoming
event
has
a
trace
ID,
it
retains
it
and
passes
to
the
function.
If
it
doesn't,
it
creates
the
Nutri
city
that
is
applicable
when
we
have
cron
kind
of
scenarios.
B
So
one
of
the
most
important
reason
for
us
to
build
this
from
ground
up
was
the
operational
aspects
of
it,
so
it
has
to
integrate
with
your
existing
login
solutions.
You
if
you
have
already
a
Cuban
Attis
deployment
cluster
in
production,
you
have
your
own
login
solutions.
You
have
your
own
monitoring
solutions
like
your
own
page
of
duty,
so
on
so
forth
right.
So
it
has
to
work
well
with
everything.
Since
we
have
build
everything
on
top
of
key
when
it
is
and
we
don't
dynamically
create,
any
containers
rather
depend
on
Cuban.
B
It
is
construct
all
of
those
simply
works,
and
we
had
an
use
case
where
these
things
should
run
on
a
private
cloud
by
deploying
cuban
it
is
on
private
cloud.
Testing
simply
works,
so
one
other
important
aspect
is
when
we
are
writing
a
function.
How
do
you
enable
it
to
test
right,
so
it
becomes
a
limiting
factor
in
a
lot
of
frameworks.
So
with
pontoon,
it's
very
simple
Larry
you
can
bring
a
passing
simple
HTTP
mock
server
and
all
you
need
is
a
three
api
box
like
get
input,
communicate,
success
or
failure.
B
B
B
Okay,
so
yeah,
we
came
great
in
duplicate
events.
We
created
an
event
to
find
all
the
prime
numbers
between
1
and
thousand,
so
we
fired
it
against
demo
and
prime
Waddesdon.
One
prime
demo
is
our
app
that
we
have
created
earlier
and
prime
is
the
function.
That's
been
registered
already
in
second
docker
container
that
we
have
registered.
B
So
yeah,
so
we
registered
a
function
where
you
have
given
a
docker
image
of
a
particular
version
and
you
clear
that
event
just
by
passing
a
payload.
Now
what
happens
is
your
function
would
have
been
started
and
your
function
is
saying:
I
am
going
to
read
the
input
as
a
long
array
and
I'm
going
to
create
I'm
going
to
compute
the
prime
numbers.
So
when
you
execute
your
function,
you
get
an
event
ID,
which
you
can
query
against.
So.
B
Okay,
so
you
can
see
all
the
functions
that
we
have
created
and
you
can
also
see
the
response
body,
so
we
asked
you
to
find
prime
numbers
between
1
and
thousands,
and
it
has
found
something's
and
you've
got
the
response
back
going
back.
The
most
important
aspect
is
yes:
whatever
log
monitoring
solution
you
had
earlier,
it
works,
so
you
will
be
able
to
see
the
logs
using
whatever
solution
you
already
had.
If
you
had
few
dashboards
that
you
have
built
on
top
of
used.
A
B
B
So,
finally,
our
experience
or
before
you
picking
some
open
source
framework,
just
look
at
your
requirements
and
see
whether
it
fits
exactly
or
not
since
errorless
is
in
very
earlier
phase.
There
are
so
many
projects
and
lot
of
them
are
getting
better
day
by
day
and
back
in
time.
We
didn't
find
one
suitable
one
and
we
had
to
build
something
on
our
own
and
it
was
very
easy
given
I
discuss
all
the
heavy
lifting.
B
So
when
using
service,
we
have
to
watch
out
for
scalability
issues,
not
with
respect
to
the
actual
function,
but
with
respect
to
all
if
all
of
its
dependencies.
So
even
though
there
is
no
server,
you
still
have
to
do
lot
of
operations.
You
are
observing
dream
becomes
more
and
more
tougher,
if
not
none,
but
by
adopting
Cerberus
what
you
can
observe
this.
Your
product
becomes
more
and
more
robust
and
it
starts
transition
into
fellow
factor
that,
if
you
are
all
familiar
with
it,
so
what
we
saw
is
by
using
pontoon.