►
Description
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects - Learn more at https://kubecon.io
Lesson Learnt Running an Indian Bank on Kubernetes - Akshay Mathur, Xebia
In this presentation, Xebia team will share their experience creating a modern platform based on Kubernetes. This platform is the backbone of the bank's digital transformation journey. It is helping them become API and data driven organization. We will cover following: Why Kubernetes is the right foundational technology for running the bank? Set up and best practices for creating Kubernetes cluster. What all CNCF projects we used to build the modern platform? How we manage configuration at scale for different environments? How we do release management using Helm? . How we helped bank to develop the right mindset? Best practices for distributed tracing. Best practices we use for cluster management. Best practices we use to test applications in Kubernetes. How we take care of storage concerns in our platform?
https://sched.co/YWJE
A
So
I
did
you
allows
a
good
lunch
like
the
food
yeah,
so
I
guess
we
have
to
thank
you
Khan
for
that.
So
this
session
is
about
we
running.
You
know
we
working
with
Bank.
You
know
an
Indian
Bank,
a
sort
of
running
kubernetes
introduction.
We
thought
there
was
enough
sort
of
interesting
learnings
from
it
that
might
make
for
a
good
talk.
A
A
Unfortunately,
couldn't
make
it
personal
emergency,
so
yeah
it'll
just
be
me
which
so
he
was
doing
the
presentation.
A
PowerPoint
is
not
my
primary
skill
and
then
he
had
to
bow
out
early
in
the
morning.
So
this
is
just
what
I
cobbled
together
last
week.
So
I
did
a
version
of
this
in
bangalore
over
the
weekend,
just
to
kind
of
get
some
feedback,
and
then
I
thought
we'd
end
up
with
us
supremely
you
know
superior
version,
courtesy
Shekar
Gulati,
but
yeah
unfortunately
couldn't
make
it.
He
really
wanted
to
be
here
thing.
A
He
worked
on
it
all
of
yesterday,
but
there
we
go.
So
this
is
the
not
as
improved
version.
Let's
put
it
that
way,
so
I
guess,
who
are
we
and
also
Shekar,
is
the
CTO
at
CBO
in
Gurgaon
how
many
people
from
out
of
town
a
few
of
you
are
you
from
Bangalore
Mumbai
yeah,
because
it
took
me
about
the
same
amount
of
time
to
come
here
right
from
Gurgaon.
So,
like
I
said,
checker
is
a
CTO
I
kind
of
do
some
CN
CF
related.
A
You
know:
I
Drive,
CN,
CF,
sort
of
activities
within
zbr
trying
to
get
more
traction.
So
what
is
CBI
I?
Guess
we
do
consulting
and
IT
services.
I
promise
you.
This
is
not
a
sales
pitch,
I
kind
of
want
to
set
some
context.
You
know
why
am
I
here
so
CBO
does
consulting
in
IITs.
We
are
not
in
infra
because
we're
not
TCS,
we
do
nice
stuff.
Some
of
you
have
been
to
ZB.
Obviously
we
have
fairly
good
DevOps
and
cloud
practice.
A
Lots
of
great
cloud
experience
not
as
great
at
you
know,
on-premise
infrastructure
experience,
that's
not
our
sort
of
core
skill.
We
mostly
have
application
developers
we're
great
at
software
architecture.
A
you
know.
Digital
transformation
is
sort
of
a
better
bread-and-butter,
but
you
know
we
have
customers.
We
do
these
things
for
customers,
I
am
I
too
loud
or
something
or
am.
I
is
this
better,
just
okay,
great,
so
digital
transformation,
like
I,
said
you
know
whatever.
That
means
you
know
we
can
do
architecture,
but
we
kind
of
depend
on
the
on
our
customers.
A
It
teams
to
do
the
you
know,
sort
of
the
rest
for
us
and
the
reason
I
bring
this
up
is
how
many
people
here
are
running.
Self-Managed
kubernetes,
you
know
maybe
on
Prem
with
their
own
infrastructure.
So
I
am
probably
envious
of
you
guys.
You
know
if
everything
is
going
smoothly,
is
it
going
smoothly
because
you
probably
have
great
sort
of
infrastructure
platform
teams,
whatever?
How
many
people
do
you
have
in
your
IT
platform
teams?
You
know
five,
ten
twenty.
A
A
All
right:
okay,
we
don't
really!
You
know
we
infra
is
not
a
core
skill.
We
don't
want
to
compete
with
TCS
right.
So
now
what
happens?
If
so,
the
point
is,
we
are
not
a
product
organization,
we
are
not
a
start-up.
That
kind
of
indicates.
We
get
a
certain
level
of
level
of
application
developers.
Linux
kernel
contributors
are
not
going
to
come
work
at
CBI
as
much
as
we
like
might
want.
You
know
want
to
have
them,
we
don't
compete
with
Paytm.
We
I,
don't
think
we
can
compete
with
big
basket.
A
You
know
in
India
you
know,
and
then
you
know
you
have
all
of
your
so
internationally.
You
know
if
you
are
uncloudy
they're,
all
these
organizations
running
openshift
on
Prem.
There
are
a
lot
of
people
on
GCP
on
any
case
now
a
case
is
really
picking
up
and
Ani
case.
You
know
on
on
AWS,
for
we
don't
have
a
lot
of
on
Prem
scale.
You
know
we
don't
have
our
own
data
centers
either
right,
which
means
if
we
set
up
a
platform.
Covenant
is
base
platform.
Essentially,
I
have
to
get
application
developers.
A
People
I
really
have
people,
I
trust
and
then
upskill
them.
We
have
to
kind
of
learn
that
right,
I
think
we
tried
to
get
a
really
nice
smart
infringe
in
here
and
he
ended
up
joining
Amazon,
so
yeah.
That
tells
you
a
lot
about
that.
So
why
all
of
this?
So
the
reason
is
about
a
couple
of
years
back,
you
know,
like
I,
said
we
sort
of
tied
up
with
one
of
our
customers,
a
sort
of
leading
Indian
Bank.
A
They
wanted
to
sort
of
go
through
a
transformation
exercise
and
they
came
to
us
for
sort
of.
You
know
you
know
some
inputs,
you
know
how
do
they
build
sort
of
an
API
platform
which
is
very
data-driven
Yin?
There's
a
data
Lake
lots
of
data
data
which
cannot
go
on
the
cloud
it
has
to
you
know
it's
Bank
RBI
doesn't
allow
you
to
take
stuff
to
the
to
the
cloud
so
that
data
has
to
stay
within
the
bank's
data
centers,
but
they
want.
A
You
know
a
PSA
one
micro
services
and
they
have
to
be
polyglot
because
you
know
we
are
not
the
only
vendor
they
have
dotnet,
they
may
have
Python.
Whatever
so
we
sort
of
talked
it
over
with
them,
there
was
a
whole
back-and-forth
and
we
finally
ended
up
recommending
kubernetes
to
them,
which
sounds
great.
You
know,
technically,
it
was
a
right
choice
to
make,
but
somebody
has
to
operate
disk
open.
It
is
right,
it's
not
on
the
cloud.
A
Maybe
you
can
get
a
managed
service.
This
is
where
banks
traditionally
are.
You
know
some
of
them
are
much
further
along
now.
You
know
your
societe
Generale's
and
your
JP
MCMC
is
a
bank.
You
know,
but
you
know
your
goldman
sachs
kind
of
people.
They
are
fired
of
the
game,
but
then
they're
driven
internationally,
but
traditionally
banks
would
have
is
ESB.
Is
you
know
websphere?
You
know
IBM
servers,
how
many
people
have
heard
of
all
of
these
things.
A
You
know
not
everything
used
to
run
in
containers
containers
are
fairly
new,
so
the
younger
folks
probably
don't
remember
that
and
then
you'd
have
enterprise
systems
from
IBM
Oracle
Microsoft.
But
the
thing
is
there's
no,
there
usually
won't
be
a
lot
of
clear
infrastructure.
You
know,
automation
or
automation
is
too
low
that
infrastructure
orchestration.
A
You
know
in
terms
of
light
your
unit,
annexes
or
your
you
know:
you'd
they'll
have
VMware,
but
maybe
not
everything
will
be
API
driven
a
lot
of
stuff
will
be
manual
because
it's
also
sort
of
dictated
by
where
the
IT
teams
are
so
there
will
not
always
be
a
clear,
centralized
view.
I
cannot
go
to
a
bank
and
say
give
me
50
compute
instances,
and
you
know
so
many
buckets
of
object.
A
Storage,
I
can't
go
to
the
IT
team
and
say
that
you
have
to
really
work
with
them
to
get
the
kind
of
compute
and
storage
that
you
want,
so
they
they
would
traditionally
have
long
release
cycles
and
everything
will
be
heavily
regulated.
You
can't
do
your
get
ups
or
continuous
deployment
because
releases
are
regulated.
You
can
only
release
so
many
times
per
month
in
certain
windows
and
and
they'll
have
some
cloud
adoption.
You
know.
A
So
the
question
is:
why
did
we
pick
kubernetes
right?
If
you
have
all
these
operational
challenges?
As
well
right,
you
want
to
sort
of
go
so
here's
the
thing.
You
cannot,
if
you
any
enterprise,
you're,
not
going
to
digitally,
transform
your
IT
the
next
day,
you're
not
going
to
wake
up
today
and
the
CIO
says
tomorrow
we
will
be
completely
elastic.
That
does
not
happen
right,
so
you
have
to
kind
of
find
inroads,
which
is
what
they
wanted
to
do
and
what
we
were
trying
to
help
them
with.
A
So
we
pick
kubernetes
because
micro
services
or
services,
let's
drop
the
micro,
they
run
great
on
cloud.
You
know
you
have
auto
scaling
groups
in
your
AWS.
You
know
you
have
container
services
etcetera
on
on
Prem.
What
is
the
infrastructure
abstraction
that
you
might
want
to
use
and
there
have
been
a
few
over
the
years?
You
know
this
is
not
a
new
problem,
but
we
could
not
really
go
in
and
transform
the
entire
IT
operations.
A
So
what
we
had
to
do
is
to
run
our
services
because
remember
they
came
to
us
for
functional
application
development,
but
we
needed
an
infrastructure
abstraction.
So
that's
how
we
go
out
into
the
infrastructure
game
to
start
with,
because
they
did
not
have
other
people
or
internally
who
were
yet
at
that
sort
of
level
of
infrastructure
automation.
A
So
we
pick
kubernetes
medic
because
it
gives
us
consistent,
infra
obstructions
right,
it
has
API
is
for
provisioning.
You
know
your
workloads,
everything
is
API,
driven
not
for
storage
for
for
scaling,
it's
all
API
driven,
which
is
essentially
what
we
wanted,
because
that
makes
our
sort
of
development
experience
much
much
smoother.
It
gives
you
a
framework
for
essentially
building
polyglot
resilient
elastic
services,
contain
orchestration,
etc,
and
then
in
future
you
know
the
big
question
was:
do
we
do
it
on
Prem
go
on
cloud?
A
We
can't
go
on
cloud
yet
so
like
well,
just
run
on
kubernetes,
and
then
tomorrow,
if
you
make
up
your
mind
or
the
strategy
shift,
just
lift
ensure
of
the
services
to
clear
it'll.
Just
work.
Excuse
me,
plus
there
was
a
great
community
support.
That
was
the
major
driver
right
and
there's
high
cost
savings
because
it's
it's
more
efficient.
Instead
of
getting
you
know,
whatever
six
apps
hours
and
huge
boxes,
we
can
share
infrastructure.
Now
so
that's
all
well
and
good.
You
all
already
know
this
right.
A
The
big
question
is:
why
don't
we
use
open
shift,
how
many
people
are
using
open
ship?
A
few
of
you
right?
It's
it's
a
great
solution
right.
Why
didn't
we
just
go
with
that?
It's
managed
kubernetes
essentially,
and
the
reason
is
again.
We
are
not
in
a
place
to
transform
the
IT
operations
of
our
customers
right.
A
They
have
not
really
heard
of
kubernetes.
They
had
not,
certainly,
two
years
back,
we
were
selling
it
to
them,
which
means
meant
we
had
to
start
somewhere
and
open
ship
I
mean
for
those
of
you
who
are
running
in
it
is
an
investment.
It's
a
great
investment
if
you
plan
to
take
it
long
term,
but
if
you're
unsure
about
this
crappy,
you
know
company
that
you
haven't
really
heard
of
that's
us
who
are
proposing
this
newfangled
thing.
A
It's
a
big
investment,
it
sounds
like
a
risky
investment,
so
we
went
back
and
forth
and
then
you
know,
obviously
there
are
other
platforms
in
the
mix.
You
know
privately
Cloud
Foundry
was
still.
It
was
a
kind
of
the
platform
of
choice
back
then,
for
for
the
ecosystem
in
which
we
operate,
which
is
sort
of
the
Java
ecosystem.
A
You
couldn't
run
really
run
dotnet
on
it
a
case
engine
was
out,
you
could
run
it
on
Azure
stack,
obviously,
but
it's
it's
non-managed
again
and
then
you
know
you
had
IBM,
which
no
cloud
private,
which
kind
of
doesn't
exist
in
its
current
state
internally,
that
was
Cloud
Foundry
again
and
PKS
had
just
come
out.
So
we
had
a
lot
of
back-and-forth
PKS.
You
know
the
reason
we
didn't,
because
they
didn't
really
push
for
it.
I
think
for
us
as
well
them
going
with
a
managed
service
would
have
made
sense.
A
You
know,
IBM
is
pretty
good
at
that,
but
PKS
licensing,
adding
then
was
per
part
or
something
which
made
no
sense.
You
know
you
couldn't
really
scale
that
out
and
open
shift
had
its
community
version,
which
we
could
have
gone
with
and
migrated
to
OCP
down
the
line.
But
you
know
it's:
it's
a
community
open-source
version
in
a
financial
institution.
You
want
something
that
at
least
heavily
supported
by
the
community.
So
you
know,
open-source
communities
was
was
that
we
had
had
enough
sort
of
experience.
This
was
not
our
first
experience
with
kubernetes.
A
We
had
had
enough
people,
you
know
sort
of
managing
open
share
of
managing
cloud
kubernetes
servers
for
about.
You
know
GK
for
a
bunch
of
our
customers,
but
this
was
essentially
us
having
to
manage
the
kubernetes
platform
itself,
so
this
infra,
which
we
don't
get
into
and
don't
want
to
get
into
and
then
there's
the
platform
itself,
which
ideally
is
this
better
I
mean?
Oh,
is
this
too
loud?
Okay
step
back,
so
where
was
I
right,
so
community
kubernetes
made
to
be
like
I
said
we
have
application
developers.
A
We
didn't
really
want
to
deal
with
the
kubernetes
layer.
We
wanted
to
submit
pods
and
deployments,
but
they're
like
no.
You
have
to
run
this
platform
as
well,
and
then
this
is
one
division.
You
still
have
to
go
in
and
sell
it
to
the
other
stakeholders
within
the
bank,
because
you
cannot
run
forward
of
your
IT
teams
right,
they're,
the
ones
giving
you
the
infrastructure,
so
we
kind
of
detailed
out
the
components
we
had
and
if
you
look
at
this,
it's
all
open
source
right
once
we
could
not
go
OCP.
A
We
essentially
picked
up
all
the
we
tried
to
get
support
for
a
couple
of
pieces
for
docker
etcetera,
but
the
people
we
talked
to
for
the
registry
basically
said
well,
you
can
buy
a
kubernetes
managed
cluster
from
and
then
you
can
run
the
registry
on
top
of
it.
I
already
have
a
Chester.
I,
don't
need
another
one.
So
this
is,
you
know
we
needed
a
gateway
right,
something
going
risk.
So
that's
where
we
ended
up
with
is
to
you
because
again
it's
a
it's
a
enterprise
right.
They
don't
just
want
knots,
knots
out
traffic.
A
Everybody
knows
not
south
east
west
right,
not
south,
is
traffic
coming
into
your
cluster
east
west
is
traffic
within
sort
of
your
data
center
or
your
cluster.
They
wanted
governance
on
the
east-west
traffic
as
well,
so
a
service
mesh
made
sense
and
we'd
been
talking
to
a
few.
You
know,
we've
been
talking
to
Google,
etc,
and
they're,
like
is
too,
is
now
mature
enough
that
you
can
really
use
it
right
and
then
we
had
an
SSO
provider.
So
a
lot
of
this,
though,
is
most
kind
of
modeled
on
the
OCP
stack.
A
It
just
seemed
to
make
sense.
The
intent
I
think
was
always
to
go
to
OCP,
but
you
know
have
a
maker
start
and
really
prove
that
this
is
a
viable
option
as
well.
So
you
have
you
know
you
can
run
a
Redis
cluster
on
it.
You
need
object,
storage,
you
can
run
mini.
Oh,
we
are.
We
are
doing
that
now
you
can
run
your
services.
A
How
many
people
using
Zipkin
distributed
dressing
yeah
anybody
using
Jaeger,
you
know
thinks
Lipkin
is
crap
and
old
right,
I
thought
so,
but
you
know
we
have
we're
from
a
Java
background
net
flix
OSS
spring
cloud
etcetera.
We
were
fairly
comfortable
with
with
Zipkin
with
running
it.
So
then
we
and
I
hadn't
really
looked
at
yeah
girl.
Deeply
enough.
A
Containerization
wasn't
really
mainstream
I
know
because
I
heard
those
conversations
you
know,
certainly
in
India,
and
a
lot
of
big
enterprises
weren't
really
moving
to
containers.
Yet
it's
it's.
It's
a
big
rip,
lat
forming
effort.
This
is
what
we
were
trying
to
do
here,
and
this
is
the
same
same
is
true
for
most
organizations.
This
is
about
two
years
back.
A
You
know
it's
not
that
long
back
OpenShift
was,
you
know,
redhead
wasn't
IBM
yet
so
I
said
this
statement
last
week,
I
said
ready
at
was
still
right
at
and
then
away
nice
gentleman
from
IBM
came
up
and
corrected.
Me
said
it's
still
red
hat,
which
I
know
you
know
it
isn't
its
own
individual
entity,
yeah
by
da
Vinci
froze
on
v3
core
OS,
wasn't
really
integrated
into
open
shifty-eyed.
It
was
its
own
entity.
You
could
get
core
OS,
you
can't
get
core
OS.
A
Now
you
have
to
go
with
Frederick
or
OS,
or
people
are
saying
yes
flatcar.
Whatever
history
was
just
stabilized,
I
think
evasion
when
I
just
come
out.
Half
of
you
know
at
least
half
of
the
current
C&C
of
projects.
I,
don't
think
they
were
even
an
incubation
and
cube
ADM
wasn't
yet
the
default
in
store.
Any
people
are
using
cube
IDM
to
do
all
of
their
sort
of
production
provisioning.
A
A
Today,
the
you
know
just
in
the
last
two
years,
so
much
has
changed
right.
Everybody
here
understands
what
a
container
is.
Everybody
appreciates
you
need
a
container
Orchestrator
kubernetes
is
the
clear
winner.
It
was
still
the
clear
winner
back
then,
though
yeah
and
then
you
have
operators
now
the
whole
new
world.
You
automate
everything
there's
should
be
no
manual
intervention
needed,
you
submit
your
API
resource
and
your
CR
D
gets
read
and
it
applies
the
cluster
state.
A
How
easy
is
it
to
convince
enterprises
to
adopt
something
completely
new
and
that
to
open-source
community
driven
stuff?
How
easy
is
it?
How
did
he
convince
them?
That's
the
real
reason
we
are
here.
You
know
honestly,
I
think
so
so
a
lot
of
effort.
You
know
we
had
to
go
in
and
do
workshops
so
convince
the
people
we
were
working
with
then
convince
the
people
they
had
to
work
with
then
convince
the
infratry
then
talk
to
the
security
and
compliance
teams,
and
these
guys
you
know
if
something
goes
wrong.
A
I'll
be
I
will
come
and
put
them
in
jail,
or
also
they
say
you
know
fit
banks,
so
they
have
to
be
really
convinced
and
we
had
to
take
a
bunch
of
exceptions.
Nonetheless,
you
know
because
it's
just
the
way
kubernetes
works.
You
can
only
do
so
much
if
you
were
on
open
shift.
You
know
you
have
some
backing
with
SELinux
or
a
palma
or
whatever,
but
community
communities
and,
like
I
said
we
were
not
really
linux
experts.
A
We
built
that
expertise
up,
so
you
know
we
had
to
convince
them
about
each
of
the
core
components,
and
then
we
did
a
lot
of
estimation.
You
know
capacity,
planning,
etc,
because
really
what
they
wanted
was
something
fairly
ambitious,
something
that
scaled
a
lot,
which
means
you
have
to
provision
all
this
compute
upfront.
They
have
to
have
purchase
orders.
They
have
to
buy
all
this
infra
or
at
least
provision
it
on
the
VMS.
But
then
vm
servers
are
limited
by
the
capacity
you
have
to
buy
a
new
vm
servers
as
well.
A
You
know,
VMware
servers
would
be
so
nice
to
run
on
the
cloud
and
just
scale
up
whenever
you
felt
like
it,
which
means
we
don't.
We
can't
do
no
daughter
scaling.
For
instance,
you
know
just
auto
scaling.
Quadrotor
scaling
is
a
limit
okay,
so
the
journey
was
I.
Don't
know
if
you
guys
can
see
this
well
enough,
so
the
confession
I
have
to
make
is
I.
Did
this
slide,
while
I
was
sitting
here
right
before
the
sessions?
But
but
you
know
this
started
around
July
2018
they
came
to
us.
We
had
a
lot
of
discussions.
A
Yeah
I,
don't
have
a
laser
pointer.
Sorry,
so
you
know
I'll
just
read
it
out
and
then
around
September
we'd
finally
started
pitching
communities
to
them.
We
talked
to
a
lot
of
people.
Then
we
had
to
detail
out
each
component
give
comparison
matrixes
matrices,
you
know,
like
I,
said
PKS,
why
not
open
shift
or
PKS
or
sure
or
whatever?
Why?
Why
kovin
it
is,
and
then
we
had
to
get
architecture,
approvals,
right,
governance
approvals,
and
so
I
will
say
this.
For
the
most
part,
everybody
was
really
thrilled
to
have
that
transformation
happening.
A
It's
just
that
it.
It
takes
a
little
bit
of
due
diligence.
It
takes
some
effort.
You
know
somebody
has
to
come
in
and
be
a
little
bit
pushy,
but
everybody's
thrilled
with
new
technology
in
any
enterprise.
Their
big
problem
is
there's
so
much
inertia.
They
really
want
to
break
that
and
really
move
fast,
but
it's
it's
always
a
challenge.
To
do
that.
A
Once
we
were
past
that
we
started
kicking
off
now
we
had
to
give
detailed,
you
know,
sort
of
hardware
specifications
right
and
that
that
means
everything
right
down
to
which
ports
have
to
be
opened
where
on
which
fire
was
because
this
is
heavily
regulated.
So
you
have
to
prepare
all
of
that
up
front.
A
Ok,
how
many
people
can
predict
where
a
pod
is
going
to
be
scheduled
across
eight
nodes
and
unless
you
put
labels
on
it
or
whatever?
Can
you
predict
it?
However,
you
have
to
give
the
oit
team
and
architecture
diagram
saying
this
server
is
where
this
application
is
going
to
run.
This
server
is
where
this
application
is
going
to
run
and
then
I
have
to
go
in
and
convince
them.
I
have
no
idea
where
it's
going
to
be
scheduled.
A
So
they
don't
like
that.
Cubelet
or
docker
Daymond
has
to
run
as
root.
They
don't
want
anything
to
run
as
root
right.
So
a
lot
of
discussion
around
that,
but
we
finally
went
to
production
in
April,
two
thousand
nineteen,
so
not
too
bad.
It
got
a
little
bit
hairy
at
the
end
and
mind
you.
This
is
not
just
kubernetes
right.
We
were
actually
doing
application
development
for
them.
The
reason
I
stress
all
of
this
is
our
platform.
A
So
we
learned
that
cloud
and
on
frame
are
very
different
beast
just
because
we
got
kubernetes
pods
running
on
AWS
and
on
gke
in
just
because
we
got
containers
running
on
open
ship
does
not
mean
we
were
in
any
way
qualified
to
establish
our
own
kubernetes
platform.
Honestly,
we
didn't
know
what
we
were
getting
into,
and
that
was
the
best
thing
possible
because
he
just
went
ahead
and
did
it
right.
It's
a
hummingbird
story.
A
We
did
not
know
storage
when
we
started.
I
can
be
fairly
upfront
about
it
because
the
the
problem
is,
you
know,
I
want
to
just
deal
with.
You
know,
I'll,
give
a
persistent
volume
claim
and
I'll
get
a
persistent
volume.
However,
the
underlying
storage,
you
know,
there's
no
orchestration
there.
These
guys
are
used
to
attaching
disks
manually
or
okay
over
the
network.
A
They
had
NFS
and
they
themselves
didn't
recommend
are
using
it
because
they
like
we've,
never
quite
certified
it
for
production,
so
storage,
you
know,
I
have
no
idea
what
I
Ops
was
when
I
started.
I
had
to
learn
all
of
these
things
same
with
my
team.
Networking
I
knew
a
little
bit,
but
then
I
had
to
kind
of
explain
it
a
little
bit
to
my
application
developers
in
the
java
application
developers,
but
that
was
slightly
easier
but
again
I'm,
not
an
IT
guy.
A
You
know,
my
knowledge
of
networking
is
very
different
from
what
the
operations
team
knows
so
yeah
and
then
again
we
had
to.
We
had
to
actually
understand
what
exactly
is
a
container.
You
know
so
I
had
to
start
actually
go
back
in
and
sort
of,
because
I
had
to
explain
this
stuff
right.
What
is
the
C
group?
What
is
a
namespace?
This
is
what
a
container
is.
This
is
how
it
runs
this.
Is
our
community
schedules
it.
So
we
had
to
learn
and
then
you
have
to
run
this
yourself.
A
A
The
thing
is:
enterprise,
IT
teams,
don't
like
it.
When
you
try
to
circumvent
trying
to
circumvent
them,
but
try
to
run
ahead
of
them
right,
they
have
their
own
standards
and
is
the
meant
to
safeguard
security.
They
meant
to
safeguard
the
jobs
as
well.
You
know
whatever
you're
doing
well
risky
to
them,
so
our
goals
obviously
were
to
install
and
operate
communities
using
only
community
tools
right
manually
on
VMs
from
scratch:
real
Williams,
no
core
OS
there
and
then
obviously
get
the
DevOps
into
place.
A
Sed
pipelines
figure
out
governance
strategies
so
that
I
don't
have
a
big
DevOps
team.
This
was
my
team.
By
the
way
you
know
it
when
I
say
I
actually
mean
I.
You
know
I
actually
leading
the
initiative,
which
means
we
have
to
bring
in
as
much
automation
and
governance
as
is
possible
and
sort
of
redeveloped
from
scratch.
Whatever
open
ship
gives
you
out
of
the
box
and
then
the
sort
of
govern
is
to
you
in
a
flexible
enough
mana
to
govern
Israel.
A
Okay,
I
will
take
two
moments.
Is
that
fine
and
then
I'll
close
right?
So
we,
the
learnings,
were
run
all
processes
as
docker
containers.
You
know
no
open
Internet,
so
you
actually
have
to
get
all
the
images
into
the
enterprise
docker
registry.
We
used
cube
spray
to
set
up
on
rel
VMs.
Anybody
use
cube
spray.
No,
so
it's
essentially
an
opinionated
wrapper
around
cube.
Idiom
gives
you
a
lot
more
tooling
by
it's
slightly
slow,
not
a
big
problem
for
us
but
use
ansible
playbooks.
A
It
sets
up
the
Calico
overlay
networks,
etcetera
and
then
we
set
up
some
admission.
Control
policies
and
Hellen
charts
were
a
big
big.
Big
of
you
know
offering,
because
all
of
those
components
I
showed
were
installed
through
an
charts
in
case
the
chart
didn't
exist.
We
went
in
and
wrote
our
own,
and
this
is
sort
of
the
size.
It's
not
great
for
cloud.
You
know
into
it.
For
instance,
last
week
we're
running
1200
nodes
on
AWS.
We
have
about
50
across
various
clusters.
A
You
know
there's
an
experimentalist
out
to
test
out
upgrades
and
then
there's
an
on
prod,
prod
and
ER,
and
then
some
of
the
dashboards
we
did.
You
know
I
really
like
we've
scoped
I'm,
not
sure
that
it
just
looks
just
so
pretty
and
I'm
not
sure
I've
ever
figured
out.
How
read
the
information
we
just
looked
so
pretty,
but
prometheus
is
a
God
Zen.
You
know
it
can
monitor,
is
to
your
performance.
It
can
actually
monitor
our
SSO
server
and
you
know
sort
of
your
java
memory
utilization.
Everything
you'll
be
tired.
A
Each
of
the
services
to
essentially
get
from
it
just
to
scrape
their
data
so
very,
very
useful
to
track
memory
issues
and
then
you
know,
custom,
auto
scaling.
We
have
something
called
backends
for
front
ends,
which
is
essentially
Zul
gateways
from
the
spring
cloud
stack.
They
don't
a
not
memory
intensive
there
I
owe
in
tensive,
so
we
had
to
plug
in
a
steer
to
that
sort
of
scale
per
connection.
And
then
you
know
timeouts
retry
as
rate-limiting.
A
All
of
that
stuff
should
I
close
or
do
I
have
500
minutes
or
okay
was
hoping
for
some
idiotic
operation
problem
areas
like
I,
said,
storage
orchestration
was
a
pain.
Ultimately
we
ended
up
with
and
if,
as
we
tried
out,
cluster
anybody's
tried
out
lustre,
everybody
seems
to
hate
it
nowadays.
You
know
I've
talked
to
a
few
people,
it
again
we're
not
storage,
X
but
sort
of
operationalizing.
It
proved
complicated
and
was
still
backed
by
NFS.
A
Ultimately,
we
ended
up
with
NFS
it's
working
out
fine,
but
now
I'm
kind
of
looking
at
cash
and
rope
rope.
Basically
doing
the
socket
storage,
orchestration
from
kubernetes
itself,
so
looks
more
promising.
Now
helm,
helm
with
tiller
was
super
painful.
Thankfully
he'll
and
three
is
out
now,
but
it
would
it
kept
blowing
up
initially
until
we
kind
of
figured
out
how
to
massage
it
to
run
nicely.
A
You
can't
really
do
a
lot
of
dynamic
configuration
with
it.
It
keeps
going
up
but
yeah,
and
then
you
know
our
prince
is
tio.
So
it's
do
configuration
give
us
the
most
problems,
especially
with
TLS.
We
had
a
lot
of
problems
with
sort
of
connection
pooling
at
the
gateway
level
and
then
with
the
certificates
we
did
not.
You
know
we
can't
use
sort
manager,
because
this
is
all
closed
on
environment.
You
have
to
have
a
purchase
order
for
certificates
as
well,
so
stuff
kind
of
work
stuff.
A
Sometimes,
if
you
has
a
lot
of
configuration
for
a
lot
of
special
use
cases-
and
sometimes
some
of
them
just
don't
be-
and
traffic
stops
coming
to
your
cluster,
so
a
lot
of
that,
so
what
we
learned
was,
you
know
don't
run,
is
kubernetes
adoption
in
front
of
sort
of
traditional
IT
teams.
You
have
to
work
hand
in
hand,
and
so
we
learned
from
a
bunch
of
so
you
know,
did
restart
the
CDs
node
h,
CD
would
crash.
A
Okay,
let's
see
one
end
yeah.
This
is
sort
of
you
know,
okay,
so
nine
three
nines
that
no
comment,
I
guess
but
I'm
hopeful
we
will
get
to
three
nines
I.
Think
you'll
agree.
It's
kind
of
a
challenge,
but
you
know
we
want
to
kind
of
the
expected
peak
load
is
5,000
transactions
per.
Second,
we
have
enough
in
front,
so
the
challenge
is
more
about.
If
things
go
wrong,
then
we
have
to
recover
really
fast.
A
Fine
should
I
close
because
nobody's
coming
to
push
me
off
the
stage,
not
sure.
What's,
okay,
you
guys
want
me
to
continue.
Do
let
me
know
if
I
have
to
run
off.
You
know
if
the
next
speaker
is
waiting
for
me:
yeah,
okay,
you
all
ready
all
right,
so
I'll,
step
off
I
kind
of
wanted
to
show
this
up,
so
we
have
a
Gradle
build
and
that
essentially
handles
the
entire
build
lifecycle.
A
Jenkins
just
calls
a
braided
riddle
build
so
because
we
we
can't
afford
a
lot
of
overhead
grady
bootstraps
every
service
and
that
has
all
of
the
operations
stuff
inbuilt
already.
So
essentially,
you
just
write
the
code,
business
logic
code
and
you
know
sort
of
your
environment
specific
configuration
and
it
seems
to
work
well
enough.
Typically,
you
know,
like
I,
said,
there's
only
like
three
or
four
people,
so
you
know
you
have
a
your
you
build
it.
You
run
it
mentality.
You
have
to
then
help
out
with
production,
which
is
fine
startups.
A
Do
that
all
the
time
right
and
insecurity
is
sort
of
cube.
Benched
occupancy
is
benchmarks
all
of
that
stuff.
Traditional
scans
image
hardening.
We
had
to
actually
go
in
and
do
all
of
this
manually
right,
no
open
sure,
and
then
we
do
sort
of
Quarterly's
do
these
activities
quarterly.
So
I
guess
in
conclusion,
you
know
just
want
you
know.
The
reason
for
this
talk
is
yes,
it
is
possible
to
you
know,
sort
of
adopt
kubernetes
across
your
enterprise,
your
traditional
enterprise,
even
running
on
prim
with
the
community
version.
That's
a
lot
of
great
support.