►
From YouTube: Experiences From Running Istio in a K8s Production Environment - Line Moseng & Johnny Horvi
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Experiences From Running Istio in a K8s Production Environment - Line Moseng, Norwegian Labour and Welfare Administration & Johnny Horvi, NAV
The Norwegian Welfare Administration has gone from on-prem only Kubernestes clusters to also run production applications in the public cloud. On this journey, we have taken Istio into our open source platform. In this presentation, we will talk about our experiences from running Istio in a multi-cloud production environment consisting of clusters on-premise and Google Kubernetes Engine. We will focus on how we use Istio to control the traffic between services, Mutual TLS, Istio RBAC, describe the challenges we have had on our way and how we solved them.
To learn more click here: https://sched.co/FuKx
A
A
And,
like
I
said
we're
from
the
welfare
and
labour
administration,
which
we
call
nob
for
short.
Now
is
the
governmental
directorates,
who
are
responsible
for
implementing
the
Norwegian
welfare
model
and
just
to
give
you
an
example
of
what
we
mean
by
the
welfare
model,
I'm
going
to
give
you
a
quick
introduction.
So
we
say
that
enough.
We
will
follow
you
from
cradle
to
grave.
A
By
that
we
mean
that
if
you
have
a
baby,
we'll
get
benefits
for
that
or
we
will
get
benefits
and
support
if
you
are
unemployed.
If
you
are
have
a
job,
you
are
employed,
but
you
get
sick.
So
you
cannot
work
for
a
while
or
if
you
have
a
disability
so
that
you
cannot
work
at
all
and
when
you
are
old
enough
to
retire
you'll
get
a
pension.
A
And
technically
that
means
that
all
of
these
services,
they're
about
16,
total
that
we
support
from
love
that
in
that
means
that
we
have
a
lot
of
systems.
They
range
from
and
70s
COBOL
system
still
in
production
today,
brand-new
coupling
code.
So
we
had
a
lot
of
systems
in
a
lot
of
different
aspects.
A
The
systems
are
for,
for
example,
applying
for
these
services
and
benefits
their
system
for
processing
the
applications,
systems
for
retrieving
and
storing
information
on
our
citizens
and
so
on
in
the
Norway.
Now
is
a
huge
deal.
We
distribute
about
a
third
of
the
national
budget
each
year,
and
we
also
see
that,
in
a
period
of
about
five
years
time
about
95
percent
of
the
Norwegian
population
are
using
some
of
nobs
services.
A
That
means
that
we
do
have
a
lot
of
sensitive
personal
data
on
the
Norwegian
population.
It
also
means
that
our
application
development
teams
enough
have
to
be
able
to
maintain
and
debug
their
applications
in
an
efficient
manner.
They
also
have
to
reemployment
implement
new
ones
when
there's
new
regulations
in
laws
and
stuff
like
that.
So
let's
talk
about
the
platform
team,
which
Johnny
and
I
are
a
part
of.
A
We
saw
the
need
for
giving
the
application
development
teams
a
platform
that
made
them
suited
for
and
that
made
them
suited
for
work
efficiently,
be
autonomous
and
could
respond
to
changes
in
regulation
and
loss
in
a
quick
way.
So
one
and
a
half
year
ago
we
built
a
platform
called
nice
which
are
Cuban.
It
is
based.
A
A
So
we
added
tools
like
building
blocks
in
the
system
so
that
they
could
configure
these
to
their
needs
when
they
needed
them,
and
all
of
this
is
to
optimize
for
autonomously
development
in
speed
and
continuous
delivery,
and
this
is
what
we,
what
is
the
goal
of
our
platform?
Nice,
the
application
platform,
and
it's
also
our
guidance
when
we
face
designs,
design
decisions
in
the
platform.
B
B
In
addition,
we
provide
storage
to
the
developers.
We
use
a
project
called
rook
that
sits
on
top
of
SEF
and
provides
block,
storage
and
object.
Storage
for
those
application
who
needs
needs
that
we
also
have
a
centralized
vault
instance
that
the
development
teams
they
put
application
secrets
and
during
deployment
we
make
these
secrets
are
available
for
actual
application.
B
We
have
a
centralized
logging
solution,
so
the
applications
simply
write
to
stand
out
and
stand
there.
And
then
we
have
fluent
e
running
as
a
diamond
set
in
the
cluster
transporting
the
logs
into
elasticsearch,
and
then
we
have
key
Bamba
as
a
front-end
for
developers
to
view
the
logs
on
to
do
self
dashboards.
B
Similarly,
for
metrics
we
have
Prometheus
running
inside
the
cluster,
so
the
application
provide
us
with
endpoint
that
that
has
metrics,
so
this
is
scraped
and
put
into
Prometheus
and
the
application
developers
set
up
their
own
dashboards
in
graph
form
where
they
have
their
monitoring.
So
the
idea
here
is
that
we
ask
the
platform
team.
We
provide
a
set
of
base
metrics
for
these
dashboards
that
has
to
do
with
infrastructure,
and
then
the
application
developers
instrument
their
own
applications
to
add
the
business,
but
our
domain
specific
metrics.
B
B
B
B
B
B
So
what
we
wanted
out
of
still
is
basically
to
secure
the
crypt,
the
communication
end-to-end
with
strong
identities
for
each
of
these
applications
and
basically
being
able
to
provide
the
mutual
TLS
as
a
service
to
the
development
tips,
so
they
don't
have
to
deal
with
certificates
and
trusts
or
some
rotation
and
everything
like
that.
In
addition,
there,
the
possibility
to
enforce
access
policies
between
each
of
these
applications
to
say
that
this
application
here
can
only
be
communicated
by
these.
B
This
set
of
applications
or
only
the
ingress
controller,
so
that
is
the
basic
driver
for
a
further
thing
is
to.
In
addition,
we
wanted
to
enhance
the
observability
as
we
believe
that,
in
order
for
things
to
go
really
fast
for
fast
iteration
cycles,
then
you
need
to
increase
and
invest
in
the
actual
of
observability
and
having
the
ISTE
of
proxies
sitting
beside
the
application
processing.
B
All
the
traffic
allows
for
some
really
great
insight
both
for
tracing
and
seeing
what
applications
talks
to
what
other
applications,
and
also
also
what
happens
with
each
call
are
things
starting
to
fail.
We
see
latencies
both
of
the
applications
ever
and
the
platform
level,
so
that
is
our
basic
motivation
for
other
things.
Do.
A
I'm
going
to
talk
about
how
we
are
journey
with
this
dough
and
how
we
started
and
stuff
like
that,
and
that
takes
us
back
to
January
this
year
in
Norway
in
January
it
was
very
cold.
It
was
tons
of
snow
people
are
actually
going
skiing
in
the
city
centre
to
get
to
work,
and
this
is
a
picture
of
Johnny's
dog
in
front
of
his
house.
As
you
can
see,
there's
a
lot
of
snow
and
this
with
this
we
had
started
and
decided
to
test
sto
because
we
heard
it
on
Cube
cone
the
year
before.
A
A
A
But
we
quickly
found
that
a
lot
of
the
stuff
that
we
thought
was
going
to
work
did
not
work.
For
example,
egress
TCP
did
not
work
and
we
used
a
lot
of
time
and
resources
to
try
to
get
the
other
stuff
that
was
supposed
to
work
to
work,
and
we
felt
like
this
black
box,
where
we
tweaked
a
little
on
the
CR
DS
to
just
see
what
happened
when
we
applied
them.
A
So
we
didn't.
There
was
a
lot
of
stuff
that
we
didn't
understand
and
we
used
a
lot
of
time
to
try
to
understand
it
and
to
try
to
make
it
work
for
our
small
use
case,
but
all
in
all,
they
didn't
feel
that
ready
to
use
in
production.
We
did
not
feel
like
this
was
something
that
we
can
put
onto
our
working
production
clusters.
A
A
They
had
been
involved
in
developing
central
components
in
is
Joe
and
when
we
explained
our
concerns
and
issues
from
the
last
time,
they
assured
us
that
first
of
all,
it
was
well.
He
still
had
became
much
more
stable
in
release
year
0.8
and
that
a
lot
of
the
issues
we
had
were
solved
so
with
them
on
slack
and
slack
cause.
A
A
So
what
we
did
we
decided
to
test
out
Google
cloud
platform.
We
created
clusters
there
and
we
installed
its
Joe
this
time
we
installed
it
with
mutual
TLS,
which
we
did
not
do
the
first
time
and
in
the
in
like
June
this
year
we
had
is
Joe
running
in
production,
with
this
small
app
and
it
was
stuff
did
work.
A
We
found
that
a
lot
of
change
since
the
0.5
release,
but
stuff
did
make
sense,
because,
even
though
we
had
to
relearn
a
lot
of
the
CR
DS
and
concepts
when
we
got
things
to
work,
of
course,
it
made
more
sense.
So
it
felt
like
something
that
we
could
actually
put
into
use
in
production
and
exposed
to
all
of
our
application
development
teams
and
with
this
it's
time
to
celebrate.
So
this
is
our
team
office
and
the
day
that
we
put
this
into
production.
A
A
B
Alright,
so
now
we're
going
to
take
a
look
at
how
we
actually
ended
up
implementing
these
features
and
how
we
expose
this
to
the
application
developers
so
again
way.
Let's
imagine
that
you
have
released
your
application.
It's
pushed
to
a
docker
image.
Your
dock
dock
Remus
is
put
dr.
a
post
array,
and
you
want
to
get
it
to
run
on
mice
like
this.
B
B
B
So
when
you
have
your
noisy
mo,
all
the
development
teams
have
cube
CTL
access
and
their
pipelines
as
well,
so
they
use
it.
They'll
apply
this
noisy
mo.
The
application
is
the
application
resource
is
created
inside
the
cluster
inside
the
cluster.
We
have
an
operator
that
receives
events
about
this
resource
kind
and
what
that
operator
does
is
that
it
takes
the
information
from
application
llamo
and
it
creates
all
the
different
key
varieties
and
still
resources.
B
B
So
if
we
take
a
closer
look
at
the
application,
we
see
you
can
notice
the
team
label.
This
is
the
name
of
the
team
that
is
responsible
for
the
application,
and
this
is
important
because
inside
the
cluster
we
have
another
operator
that
ensures
that
only
this
team
has
access
to
to
administer
that
application.
Since
we
have
a
multi-tenancy
environment
with
a
lot
of
different
development
teams
and
a
lot
of
application,
it's
important
that
the
access
control
is
correct.
B
Then
we
have
the
default
information
that
you
need
to
get
a
working
deployment,
how
to
scale
the
application
and
what
kind
of
probes
that
the
application
has?
Typically
liveness
and
readiness,
probes,
metrics,
pre,
stop
hoops
and
so
on,
and
those
endpoints
are
what
we
call
the
platform
endpoints
that
doesn't
really
have
anything
to
do
with
the
application
domain
itself.
It's
just
services
for
the
platform,
then
they
define
what
ingresses
that
their
application
should
be
reached
at.
So
for
that
we
create
a
ver
still
virtual
service.
B
B
So
and
here
we
have
plus
the
access
policy-
and
this
is
where
the
application
called
up
in
this
case-
define
what
other
applications
can
talk
to
and
communicate
with
this
application.
So,
based
on
this
information
here,
we
create
a
cavalettis
Network
policy
and
the
East
EO
are
back
key
resources,
namely
the
service
role
and
service
role
binding
these
resources.
Look
like
this.
First,
a
network
policy.
Here
you
first
select
the
workload
that
you're
targeting.
B
B
Here
we
have
the
service
role.
This
role
just
says
that
you're
able
to
talk
to
application
called
app,
and
then
we
use
this
role
in
the
service
role
binding.
We
say
that
the
service
role
is
bound
to
the
service
account
of
the
consuming
application.
So
this
way
on
layer,
7
all
applicate
application
with
or
only
this
user,
meaning
the
application
can
talk
to
the
name
called
the
alright.
B
These
two
are
back
also
the
network
policy
and
port
security
policy
that
is
also
inside
the
inside
leg
history.
So
these
here
are
mechanisms
that
we
are
able
to
provide
as
a
service
to
the
development
teams
without
them
having
to
care
about
the
different
details
around
that
all
right.
So
now
we're
going
to
take
a
quick
look
at
some
of
the
challenges
that
we
met
to
describe
this.
We
start
with
just
this
example.
B
So
to
solve
this,
there
are
two
proposed
solutions
by
the
community.
So
to
explain
one
of
them.
We
start
again
with
the
application
and
we
see
that
it
has
a
binary
and
OS.
Other
applications
are
still
happily
talking
to
it,
but
the
cubelet
is
not
so
what
you
can
do
is
you
can
bundle
curl
inside
the
application
container
image,
and
then
you
can
use
exact
type
of
aliveness
redness
problem
and
everything
will
work
fine.
So
that
is
one
solution,
the
drawback
being
that
you
have
to
have
a
curl
inside
the
application
image.
B
So
you
can
say
that
the
platform
services
are
served
on
port
9000,
no
one
in
this
case,
and
then
you
can
tell
is
still
too
lot
and
force
mutual
TOS
on
that
specific
port,
and
that
will
work
fine
drawback
being
here
at
all
of
at
least
all
of
our
applications
had
to
change
into
this
moment.
So
what
we
ended
up
doing
is
basically
this,
but
sewing
it
on
the
platform
layer.
B
B
It's
not
something
we
consider
a
sole
problem,
but
it's
the
best
that
we
have
for
now.
Ideally
you
would
you
don't
want
to
have
the
overhead
we're
having
an
extra
and
why
so
there
might
be
a
solution
where
II
can
imagine
that
this
invoice
configuration
in
the
nice
car
is
somehow
moved
into
these
two
approxi.
So
if
anyone
has
any
thoughts
or
ideas
around
this,
please
come
talk
to
us.
After
of
this.
A
First
of
all,
since
we
started
this
and
there
was
really
cold
in
Norway-
a
lot
bunch
of
snow
and
there
was
a
very
hot
summer
in
June,
we
learned
that
our
experience
with
Easter
is
quite
correlating
to
the
weather
in
Norway
and
since
winter
is
coming,
we
are
fearing.
We
are
afraid
that
there
will
be
dark
still
times
ahead
of
us,
but
hopefully
not.
A
A
A
A
We
also
learned
that,
while
our
team
has
a
lot
of
freedom
to
use
new,
cutting-edge
technology,
when
we
see
that
that
will
suit
our
needs
solve
the
issues
or
problems
we
have,
we
saw
that
when
we-
actually
you
do
this
sometimes
with
alpha
products,
it
can
mean
that
you
cut
yourself
that
bleeding
edge
actually
means
that
it
is.
It
is
blood,
so
that
was
quite
new
to
us,
because
when
we
started
with
East
EO
in
January,
we
we
thought
that
this
was
more
used
and
more
stable
than
that.
A
That
means
that
we
will
still
run
a
lot
of
services
on-premise
for
a
little
good.
While
so
we
will
look
into
having
sto
installed
on
our
on
premise
clusters
as
well,
both
for
the
security
reasons
and
in
order
to
make
the
environments
more
trans
more
equal,
so
that
the
migration
from
on-prem
to
public
cloud
providers
will
be
easier
for
us
and
for
the
application
teams.
A
A
Take
a
look
at
the
multi
cloud
feature
of
a
statistical
to
see
that
whether
mesh
expanding
over
several
clusters
will
make
sense
for
us
to
use,
and
we
also
know
that
there's
a
lot
of
experience
around
people
solving
stuff
in
a
different
way.
So
we
learned
a
couple
months
ago
that,
for
example,
Google
cloud
is
providing
SEO
as
a
service.
So
we
look
into
that
and
see
if
we
can
learn
something
from
them
and
see
if
they
have
solved
some
of
our
issues
in
another
way
and
will
that
make
sense
for
us
to
use
so.
A
That's
about
what
we
had
time
for
check
us
out
on
Twitter.
If
you
want,
we
have
our
own
a
nice
IO
account
for
the
platform
and
if
you
have
some
experiences
or
thoughts
or
questions
about
the
things
that
we
had
talked
about
have
talked
about,
please
come
seek
us
out
after
the
talk.
Thank
you
so
much
for
listening.