►
From YouTube: Istio, We Have a Problem! Understanding and Fixing Bugs with a Service-Mesh - David Gageot, Google
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Istio, We Have a Problem! Understanding and Fixing Bugs with a Service-Mesh - David Gageot, Google
Istio, we have a problem! We've just deployed a shiny new set of micro-services and it behaves in a strange manner. Hard to say why with so many moving parts... Let's leverage the newly installed service mesh to understand what we've deployed, find the root problem, fix it with a bandaid and then do a proper, non trivial, blue-green deployment of a v2. That will involve the Service Graph, Prometheus monitoring, Grafana Dashboards, Traffic mirroring, all orchestrated in a simple manner by Istio.
https://sched.co/MPfG
A
All
right
well
good
morning
welcome
everybody.
My
name
is
Randy
avinasi
from
rxm,
and
this
talk
is
particularly
timely
as
more
and
more
organizations
are
moving
to
micro
service,
architectures
cloud
native
systems,
debugging
those
systems,
especially
the
more
mature
ones,
that
interact
with
lots
of
other
systems,
are,
you
know,
kind
of
a
challenging
problem,
and
you
know
sto
brings
some
very
interesting
solutions.
B
That's
quite
a
room:
okay,
so
we're
going
to
talk
about
is
tio
and
we're
going
to
talk
about
how
it
can
help
you
understand,
but
you've
deployed
and
maybe
fix
them.
So
today,
I'm
gonna
be
a
web
developer.
I'm
gonna
deploy
a
web
application
on
kubernetes
and
it's
based
on
micro
services
way
too
many
micro
services
compared
to
the
application.
B
B
Okay,
all
of
you,
you
create
bags.
So
this
is
my
small
web
application
I'm
going
to
deploy
and
given
a
test
by
the
way
for
my
demonstrations,
everybody
everything
will
be
deployed
on
my
machine
I'm
using
docker
for
Mac
that
has
support
for
Cuban
RS,
so
everything
is
going
to
be
deployed
locally
and
I'm,
going
to
show
you
my
application.
So
it's
just
a
quiz
you're
shown
two
hexagons.
B
There
are
two
products
from
the
Google
cloud
platform
and
there's
one
description
and
you
have
to
choose
which
one
is
the
correct
one
and
so
I'm
gonna
deploy
it
so
usually
you'd
build
docker
images
and
you
apply
some
llamó
to
kubernetes,
but
I've
got
a
tool
that
I'm
gonna
use
for
my
demonstration.
You
don't
have
to
think
too
much
about
the
tool.
It's
just
that
with
one
command
I'm
going
to
deploy
my
application
to
a
running
cluster
okay.
So
we
should
see
the
application
coming
up
very
soon,
yeah.
B
So
there's
layers
here,
some
people
said
they
didn't
write
any
bug.
Well,
like
I,
don't
believe
you,
so
it
should
come
up
quite
quickly
with
it.
Okay,
something
already
is
gonna,
be
adding
right.
Okay,
so
I've
got
a
bag
okay.
So
this
is
my
application
whoo
and
it's
showing
hexagons
and
sometimes
there's
a
beragon,
because
I
told
you
I
write
bags,
okay,
so
there's
a
bag
and
I
can
take
a
look
at
what's
going
on,
usually
I
would
open,
Chrome
and
see
that
one
of
the
URL
is
not
working
properly.
B
So
sometimes
it's
sending
me
some
JSON.
Sometimes
it's
a
500
error,
okay,
so
not
cool.
So
what
can
we
do?
First?
We
should
acknowledge
that
it's
Co
we
have
a
problem.
Can
you
help
me
fix
it,
because
it
would
be
nice
if
the
application
was
deploying
on
a
platform
that
gives
me
full
visibility
on
what
I've
actually
deployed
you
see,
it
would
give
me
the
list
of
endpoints
that
are
called
on
my
application
on
all
the
services.
I
could
even
maybe
see
distributed
traces.
B
It
would
be
very
helpful
if
I
had
that
the
problem
is
kubernetes
doesn't
give
me
that
you
know
that
right.
If
you
install
vania
qls,
you
don't
get
that
kind
of
visibility.
You've
got
the
list
of
the
services
you've
deployed.
You
can
see
the
logs,
you
can
see
maybe
the
status
of
each
service,
but
you
don't
really
see
the
big
picture.
So
this
is
where
SEO
comes
in,
just
to
make
sure
that
we
are
on
the
same
page
I'm,
going
to
show
you
how
SEO
works
very
briefly.
B
So,
basically,
this
is
my
application
deployed
on
a
QRS
cluster.
Every
service
runs
inside
a
pod
equivalents
pod
and
basically,
when
my
users
connect,
they
connect
through
an
ingress
and
then
the
top
two
one
service
that
talks
to
another
talks
to
another
and
then
way
back,
okay
and
somewhere
in
that
system.
There's
a
problem
so
enters
seo.
Seo
is
a
bunch
of
additional
containers
that
will
extend
the
platform
that
that
is
very
important.
It's
not
just
that.
B
It's
good
run
on
Cavanaugh's,
it's
going
to
extend
kubernetes
and
it's
going
to
extend
queries
to
bring
you
lots
of
features
that
you
are
going
to
bring
visibility
on
what
you've
deployed
and
that
are
going
to
help.
You
understand
the
bugs
you've
deployed
and
how
does
it
work
on
each
part
inside
each
pod?
B
There's
going
to
be
a
small
and
voi
proxy
who's
going
to
be
injected
by
a
steel,
and
this
proxy
will
capture
traffic
coming
in
and
out
of
the
containers,
who
knows
envoi
quite
a
few
people
now,
okay,
so
for
those
alone,
no,
it's
like
a
cheap
proxy
or
nginx.
It's
an
HTTP
reverse
proxy
that
is
able
to
capture
traffic
and
maybe
modify
your
query
or
modify
the
response
and
also
what's
very
important.
It's
able
to
capture
a
lot
of
information
that
it
can
then
send
to
other
components,
a
VCO.
B
To
give
you
the
developer,
a
lots
of
visibility.
You
then
be
able
to
observe
what
you've
deployed.
Let
me
show
that
to
you,
so
just
to
be
sure
that
you
all
understand
each
invoice
will
capture
all
the
traffic
and
it's
going
to
bring
a
lot
of
features
to
you.
So
let
me
start
by
connecting
to
one
of
the
additional
components
that
is
running,
which
is
called
a
key
ally
I'm,
going
to
connect
to
clearly
through
port
forwarding
and
I'm
gonna,
create
some
forced
traffic
to
my
website
to
make
sure
that
it's
activated
okay.
B
I
can
see
the
ingress
and
then
each
service
that
compose
my
application
and
I
can
see
which
one
is
behaving
correctly
or
not.
I
consider
the
traffic
somewhere
in
this
component
doesn't
behave
correctly
because
the
rest
of
the
chain
is
all
green,
but
then
the
response
are
sometimes
in
error.
So
it's
very
very
convenient
to
see
that
kind
of
information
lots
of
companies
adopt
easier.
Just
for
that
just
to
see
what
they've
actually
deployed.
B
B
I'm
gonna
send
some
traffic
just
to
be
sure,
and
this
is
the
kind
of
dashboards
that
you
have
by
default
when
you
install
Sto,
it's
a
dashboard
that
gives
you
some
kind
of
health
indicator
for
every
of
your
services,
so
I
can
see
that
basically,
the
UI
does
not
behave
correctly,
but
I
know
why
it's
because
it's
querying
the
game
microservice
that
is
not
behaving
correctly.
I
can
zoom
in
I
can
really
start
to
understand.
What's
going
on
in
my
application,
I
can
really
see
that
some
queries
are
not
behaving
correctly.
B
B
Here
we
are
so
you
can
see
that
SEO
as
the
notion
of
the
HTTP
protocol.
So
it's
able
to
understand
that
some
micro
services
are
sending
errors
to
the
user,
so
I
can
see
in
green
when
it's
behaving
correctly
and
I
can
see
in
yellow
when
it's
not
behaving
correctly.
Okay,
so
that's
very
convenient.
That's
really
kind
of
of
the
variability
that
we
need
to
understand
what's
going
on
aji
HTTP
level,
but
it
also
works
with
G
RPC
and
that's
very
convenient.
B
What's
better
than
that
is
thanks
to
VCO
I
can
have
distributed
Tracey's
almost
for
free.
Let
me
show
that
to
you,
I'm
gonna
connect
to
not
this
one
and
and
with
jagger
I
can
so
some
of
the
logs
that
were
captured.
For
example.
Here
you
will
see
two
kinds
of
patterns
you
will
see
one
that
says
ten
spans
and
one
that
says:
48
spines,
there's
an
error.
So
let's
drill
on
this
one,
this
one
shows
the
interactions
with
all
my
services
from
the
users.
B
Okay,
so
the
user
connects
to
the
ingress
that
talks
to
the
UI
that
tooks
to
the
game
micro
service
that
talks
to
the
ref
micro
service
multiple
times.
In
fact,
the
first
time
I
can
zoom
in
and
see
that
it's
querying
for
the
slash
count,
a
URL.
In
fact,
I
know
the
application.
The
game
is
supposed
to
ask
the
referential
how
many
X
seconds
there
is,
and
then
it's
supposed
to
take
one
random
another
one
random.
So
first
query
is
to
get
the
kind
of
excellence.
B
Second
query
is
to
check
the
first
exit
gun
and
then
again
the
first.
Oh,
that's
strange
and
then
again
the
first
oh
there's
button
here.
So
something
in
my
code
makes
it
impossible
to
get
to
random
hexagons,
see
it's
always
going
for
the
same
URL.
So
each
time
it's
getting
the
same,
X
alone
and
again
with
only
one
exactly
one:
it's
not
as
furnace
again
with
two
hexagons
right,
so
I
can
see.
I
can
understand.
What's
going
on,
I
can
drill
down
to
another
interaction.
This
time,
I
can
see
the
difference.
C
B
Is
this
explains
why
it
works
sometimes
and
why
it
doesn't
work
the
other
time?
Okay,
so
you
see
you
get
this
distributed,
Tracey's
almost
with
no
change
in
your
code
base.
It
works
with
any
language,
it's
just
added
by
the
platform
from
the
outside.
As
soon
as
you
deploy
your
application
on
ETO
and
you
make
very
small
modification
to
your
sorbet,
so
your
code
base
to
propagate
the
open
tracing
headers
you
get
tracing
for
free,
you
don't
have
to
unbearable
library.
B
It
works
with
any
language,
so
I
think
I
can
I
kind
of
understand
what
the
problem
where
the
program
comes
from
and
I
can
maybe
show
you.
So
this
is
one
very
convenient
comment
that
I
put
in
my
code
base
and
you
should
do
the
same
for
all
the
bugs.
You
create
right
put
a
comment.
First,
then,
the
baguette
okay.
Do
that
you
thank
me.
Okay,
so
I'm
not
going
to
spend
too
much
time
fixing
the
bug.
I'm
gonna
fix
it.
B
It's
somewhere
here,
I'm
gonna,
just
remove
those
nasty
lines
and
I'm
gonna,
just
rebuild
all
my
components
to
make
sure
and
it
builds
and
I
fixed
the
bug,
and
so
basically,
I
can
just
ship
it
right.
It
works
it
compiles
I
mean
I
removed
lines.
I
cannot
create
a
new
bug.
By
removing
lies,
can
I
oh
okay:
I
can
okay,
so
what
I
could
do?
Also
I
could
use
SEO
for
another
thing.
That
is
good
at
it's
good.
B
At
helping
me
deploy
my
application
in
a
more
controlled
manner,
so
I've
got
a
fix,
but
my
users
they're
still
seeing
the
Bagon,
so
I'm
gonna
deploy
my
fix
in
a
more
controlled
manner.
I'm
gonna,
deploy
it
with
a
canary
deployment.
Who
knows
what
I
can
already
plumb?
It
is
quite
a
lot
of
people.
Those
of
you
know
what
it
is.
How
how
how
are
you
is
to
to
implement
in
your
codebase
quite
hard,
sometimes
yeah.
B
It
can
be
very
hard
and
you
have
to
like
do
it
for
any
language,
every
language
with
this
Co,
it's
way
more
way
easier.
So
what
are
we
going
to
do?
Is
we're
going
to
deploy
the
fixed
version
of
our
game
and
we're
going
to
make
sure
that
all
users
they
go
through
all
the
v1
micro
services?
So
they
will
see
the
bug,
but
only
one
user,
David
based
on
HTTP
errors
or
cookies,
is
going
to
go
through
a
different
set
of
micro
services.
B
I'm
gonna
make
sure
that
when
the
UI
tooks
to
the
game
and
it's
David
who's
asking
it's
going
to
talk
to
the
fixed
version
of
the
game-
that's
very
powerful.
It's
done
at
the
end,
vallevan
it's
going
to
change
the
destination
of
your
micro
service,
just
depending
on
HTTP
headers.
Let
me
show
how
to
configure
that.
So,
basically,
I
just
have
to
add
an
additional
deployment
to
my
criminalize
deployment
to
deploy
a
fixed
version
of
my
game
and
I've
got
to
configure
Sto
so
sto
as
I
said,
you
extends
the
cumulus
platform.
B
So
it's
going
to
add
some
additional
types
of
resources
that
I
can
use
to
configure.
My
application
and
virtual
service
is
one
of
those
additional
resource
types
with
virtual
service
I'm
able
to
change
the
flow
of
the
queries
from
the
users,
depending,
for
example,
on
HTTP
headers.
So
I
can
say
that
when
one
micro
service
is
talking
to
the
game
micro
service,
SEO
is
going
to
look
at
the
HTTP
errors
and
if
the
user
is
exactly
David,
it's
going
to
change
the
destination
to
another
version
of
the
same
micro
service.
B
B
Okay,
so
I'm
going
to
deploy
it,
it's
just
reconfiguring
the
whole
application.
The
application
is
going
to
continue
to
respond
to
my
users,
but
it's
gonna
change
slightly
in
its
behavior.
So
right
now,
my
users.
They
see
that
sometimes
it
works.
Sometimes
it
fails.
Okay,
if
I
do
it
with
a
curl.
Sometimes
it
fails.
Sometimes
it
works.
If
I
change,
the
header
and
I
say:
okay,
David
is
the
going
to
query
that
page.
It's
gonna
work.
All
the
time
yeah
seems
like
a
fixed
bug.
B
I
can
also
I've
got
a
nice
extension
on
Chrome
that
I
can
use
to
set
a
request
from
a
request
header,
so
I'm
going
to
say
that
user
is
always
David
and
now
my
application
is
always
behaving
correctly.
Cool
I
can
really
have
internal
users
test.
The
application
or
I
can
have
I,
can
have
a
group
of
external
users
test
the
application
before
I,
deploy
it
to
all
my
users.
So
what
does
Chuck
Nori's
says?
B
And
he
says
okay,
if
it
works
for
David,
it's
gonna
work
for
everybody.
Right
I
mean
all
the
users.
They
are
the
same
I
mean
sometimes
they
use
different
pages,
but
is
the
same
right
right?
No,
okay!
So
what
we
could
do
we
could
maybe
try
to
identify
if
the
application
is
going
to
behave
correctly
in
front
of
all
the
users,
but
there's
a
nice
way
to
do
it
with
traffic
mirroring
with
traffic
mirroring.
We
are
able
to
deploy
two
versions
of
the
same
application.
B
One
the
v1
is
going
to
answer
the
users,
the
so
the
Ghana
cedi
bug,
but
also
the
queries
will
be
sent
to
the
fixed
version
of
the
game
so
that
we
see
here
if,
if
it
behaves
correctly
in
front
of
all
the
users,
so
it's
just
that
the
UI
is
going
to
send
the
same
query
to
the
view,
one
version
of
the
game
and
to
the
fixed
version
of
the
game.
The
green
line
is
going
to
be
a
sink
so
that
I
don't
block
my
users.
It's
just
that.
B
I
can
then
open
my
dashboard
and
see
if
the
fixed
version
would
behave
correctly
in
front
of
all
the
users.
This
is
the
kind
of
questions
that
I
can
answer.
So
how
do
I
do
that?
I
just
have
to
reconfigure
my
digital
service,
so
I
remove
everything
that
I
did
before
and
I'm
just
going
to
change
that
part
I'm
going
to
say
that
if
one
service
is
talking
to
the
hexagon
game
service,
I'm
gonna
send
it
to
the
v1,
but
also
I'm
gonna,
mirror
all
the
queries
to
the
fixed
version.
B
B
So
we
can
really
ups.
We
can
really
take
a
look
at
how
it
behaves,
so
it
should
come
up
very
quickly
yeah.
So
you
can
see
that
here.
This
is
the
v1
version
of
my
service,
serious
errors.
Here
this
is
the
fixed
version,
getting
queries
from
actual
users
and
it's
behaving
correctly.
All
the
time
and
the
latency
is
good
and
I
can
zoom
in
and
see
that
it's
behaving
correctly.
I
can
take
a
look
at
the
logs.
Everything
like
I
can
do.
B
My
users
won't
see
the
difference
they're
still
using
the
broken
version,
I'll
say:
okay,
oops,
so
that's
very
convenient
lots
of
team
lots
of
the
teams
are
I
talked
to
they.
Don't
do
that
kind
of
thing
that
basically
try
to
capture
all
the
traffic
for
one
day
and
then
they
try
to
replay
the
traffic
on
their
CI
CD
to
see
if
actual
users
would
break
the
system.
This
is
a
much
easier
way
to
do
it.
It
requires
more
resources
on
your
production
platform,
but
it's
it's
easier
to
set
up.
Okay
works
for
all
users.
B
Let's
deploy
it
like
press
the
button
and
boom
all
the
users
will
be
switched.
We
can
do
better
than
that.
We
can
have
a
Bluegreen
deployment.
So
Bluegreen
deployment
is
just
another
pattern.
You
can
use
to
deploy
your
fixed
application
this
time,
we're
going
to
say
that
80%
of
the
users
are
going
to
talk
to
the
v1
version
and
20%
of
the
users
are
going
to
talk
to
the
fixed
version
and
we're
gonna
then
change
that
way
that
weights,
depending
on
how
well
the
application,
behaves.
Okay,
you
want
to
do
that.
B
Just
a
change
in
the
llamo,
okay,
I
hope
you
all
like
yam
all.
Do
you?
Yes,
because
you're
gonna
have
to
and
those
are
all
files.
They
are
quite
small
compared
to
what
you
can
come
up
with
with
SEO,
but
this
is
quite
interesting
to
see
that
there
are
small
but
yet
polar
forms.
So,
basically,
you
can
say
that
all
the
traffic
going
to
the
hexagon
game,
100%
is
going
to
go
to
the
v1
and
0%
would
go
to
the
fixed
version.
B
B
So
yeah
errors,
so
I
can
even
see
it
better
with
the
with
this.
Sometimes
it's
okay
sometimes
doesn't
so
I'm
gonna
change
the
weights.
I'm
gonna
change
the
ways
if
I
remember
what
where
they
are.
Okay,
I'm
gonna
change.
The
way
it's
like
80/20
and
I'm
gonna
run
that
to
make
sure
that
it's
reconfigured,
so
usually
I
would
like
use.
My
dashboard
or
I
couldn't
even
have
like
I
could
even
use
the
ECU
API
to
write
some
kind
of
boat,
some
kind
of,
but
sorry
that
monitors
the
application
and
then
change
the
weights.
B
D
B
Come
on
yeah,
so
I
can
finally
play
my
game,
which
is
very
nice
game.
Okay,
I,
don't
think
I,
don't
know.
If
you
agree.
Okay,
we're
live.
We
fix
the
bug.
Okay.
What
did
we
do?
We
use
SEO
to
get
full
visibility
on
what
we've
actually
deployed.
We
can
really
see
all
components
we
deployed
how
each
other
is
talking
to
the
other
ones.
We
could
really
see
all
the
end
points
that
were
cold
and
we
could
see
distributed
traces
without
even
touching
our
code
base
and,
of
course
we
deploy
them.
B
We
deployed
a
new
application
in
a
more
control
manner
with
maximum
confidence.
So
you
don't
have
to
go
all
the
way
through
a
canary
blue-green
mirror.
You
can
do
your
own
things,
but
it's
very
convenient
to
be
able
to
use
those
tools
to
understand
what's
going
on
and
to
deploy
your
application
in
a
more
controlled
manner.
Oh
say:
ok,
so
SEO
can
do
much
more
for
you
than
just
that.
This
is
really
an
intro
to
show
you
that
developers
can
really
leverage
eesti.
Oh
it's
not
just
operations
as
developer.
You
can
really
leverage
the
CEO.
B
You
can
test
it
on
your
machine.
If
you
install
it,
you
can
get
much
more
than
that.
You
can
really
add
some
more
resilience
to
your
micro
services
because
as
soon
as
we
deploy
many
micro
services,
nothing
happens
in
memory
anymore,
it's
all
connected
by
the
network,
so
it's
very
useful
to
understand
what's
going
on,
but
it's
also
very
useful
to
have
some
kind
of
additional
mechanism
brought
by
the
platform
such
as
retries.
B
If
an
app
micro
service
is
talking
to
you
another
one,
ECU
can
retry
many
times
in
case
of
an
error
and
then
after
few
retries
it
will
stop.
You
can
also
inject
secret
breakers,
so
this
is
the
kind
of
things
that
you'd
usually
unbated,
to
into
your
codebase
right.
But
now
you
can
just
remove
that
from
your
codebase
and
let
the
platform
do
it
for
you,
you
can
configure
which
service
is
going
to
have
a
circuit
breaker.
You
can
configure
a
lot
of
things.
B
You
can
add
some
more
security
as
soon
as
you
install
it
you,
your
micro
services,
can
on
HTTPS
instead
of
HTTP,
it's
all
going
to
be
taken
care
of
by
the
platform,
so
really
I
encourage
you
to
go
to
more
SEO
talks
today
to
understand
more
and
I
encourage
you
to
test
it.
As
I
told
you
I
just
installed
it
on
docker
for
Mac
and
it
works
great.
You
just
need
a
lot
of
memory
and
yeah
just
for
the
short
story.
B
The
other
day
I
did
the
same
demonstrations
with
only
two
gigs
of
memory,
and
it
still
wouldn't
start
with
two
gigs
of
memory.
Okay,
so
you
just
you
press
the
buttons
and
it
doesn't
work,
so
it
needs
a
bit
more
than
that,
but
it's
very
convenient
to
state
it
test
it
locally.
So
that's
about
it
for
me.
Thank
you
very
much
and
I'm
open
to
a
lot
of
questions.
E
F
B
So
the
question
is:
what's
the
impact
in
terms
of
performance?
So
that's
a
very
good
question,
because
there
is
an
impact
as
soon
as
you
intercept
the
traffic
and
inspect
it
or
modify
it.
You
add
some
overhead
and
if
you're
the
call
stack
is
very
deep.
You
had
an
average
on
each
micro
service,
so
it
can
add
up
so
there's
a
lot
of
work
that
has
been
done
on
e
Co
for
the
last
few
realities
to
make
that
overhead
be
much
lower.
B
So
we
can
see
lot
of
users
who
have
adopted
SEO
very
early
on
now
find
out
that
it's
very
it's
quite
good.
Now
it's
quite
good
balance
between
overhead
and
features,
so
you
should
really
try
it
out
and
also
that's
very
important
that
you
don't
have
to
install
all
the
features
from
SEO
from
the
beginning.
The
idea
is
that
you
should
have
a
problem.
You
should
try
to
fix
it
and
you
should
try
to
find
the
features
in
easier
that
you
really
need
it.
B
You
don't
have
to
install
like
key
le
and
that
ball
and
prometheus
and
TLS.
You
don't
have
to
install
everything
at
once.
Just
install
it
see
if
the
performance
is
good,
for
you
be
sure
that
the
engineering
team
they
are
trying
to
take
it
even
lower,
but
I
think
that
the
balance
is
quite
good
already.
G
H
Okay,
I
have
a
two-part
question.
First
part
is
if
you're
using
the
traffic
mirroring.
Yes,
your
linear
call
stack.
You
would
effectively
double
the
load
on
your
micro
service.
One
step
behind.
Is
that
correct
it
is.
It
is
exactly
that
here.
Is
there
a
solution
to
that?
It's
the
first
part,
and
the
second
part
is
what
you
do
if
you
have
requests
with
side
effects
as
well
or
get
requests
requests,
which
actually
does
something
yeah
yeah,
so
so.
B
Let's,
let
me
come
I,
don't
know
which
slide
it
was
something
like
maybe
that
come
on
just
we're
gonna
make
it
so
yeah
traffic.
Knowing
so
yes,
it's
gonna
double
the
need
for
resources
after
the
UI.
In
my
case,
for
everything
except
the
UI,
and
in
my
demo
all
the
traffic
from
the
fixed
version
of
the
game
goes
back
to
the
view.
One
version
of
the
ref-
maybe
don't
do
that,
it's
just
that
my
application
is
read-only,
so
there's
no
side
effect.
B
So
if
I
send
the
same
required
to
database,
it's
gonna
be
okay
in
real
life.
I
might
have
to
deploy
a
mock
here
or
might
have
to
deploy
a
full
string
of
six
services.
So
there's
not
there's
no
easy
way
to
do
that,
but
this
is
why
it's
it's
it's
still
complicated
to
set
up
real
traffic,
mirroring
I.
Think
the
the
need
for
additional
resources
is
not
is
not
so
important.
B
C
I
B
B
Extension
for
vs
code
or
IntelliJ
that
you
can
use
to
debug
your
applications
that
are
deployed
on
QRS
cluster.
It's
basically
quite
simple.
You
just
have
to
find
the
right
word
where
your
container
is
running
and
open
the
right
port
and
cut
code
could
do
that,
for
you,
sto
I,
don't
think
there's
such
features
do
that.
B
That's
a
good
question
and
I'm
I
wonder
if,
in
the
letters
version
of
each
do,
if
it's
not
going
to
so
in
a
recent
Russians,
the
ingress
was
replaced
by
the
Gateway,
the
Gateway,
so
so
that
it's
it's
not
really
the
same,
and
you
should
maybe
have
an
ingress
in
front
of
your
cluster
and
then
use
the
Gateway
to
do
easier
things.
So
at
the
beginning,
it
was
very
much
the
same
and
you
had
like
wanting
grass
controller,
with
least
your
features
but
liking
all
the
other
features
you
expect
from
nginx
another.
B
F
Yep,
sorry
yeah,
so
we
we
run
a
word
Club,
which
is
highly
regulated,
and
one
of
the
requirements
is
that
data
in
transit
is
to
be
encrypted
and
it
of
course
it's
a
lot
of
work.
We
have
to
deploy
our
server
certificates
and
manage
it
and
set
up
the
encryption
standards
all
of
the
time.
So
is
it
still
a
good
drop-in
replacement
with
using
using
mutual
TLS,
where
we
could
just
skip
all
of
that
or
are
there
any
security
risks
that
we
would
be
exposed
to?
If
we
were
to
do
that
so.
B
One
of
the
main
features
of
issue
is
that
you
can
really
use
it
to
do
the
mutual
TLS
for
you,
so
you
can
like
delegate
it
to
the
platform
and
you
don't
have
to
implement
it
in
every
language.
Okay,
and
especially
like
all
their
components,
you
have
to
go
back
to
them
and
try
to
sort
out
how
to
to
do
some
HTTP.
So
you
can
really
do
better,
get
it
to
you
and
there's
a
couple
of
knobs.
B
You
can
turn
to
make
it
work
the
way
you
want
you
can
make
it
by
namespace
or
you
can
make
it
for
a
given
set
of
pots.
So
you
can
really
try
it
and
maybe
use
your
current
set
up
and
maybe
try
it
and
see
if
it's
good
enough
for
you
I'm,
pretty
sure
it's
gonna
be
good
enough
for
you.
You
can
really
control
the
keys
extra.
Oh
yeah.
Should
we
try
it?
It's
gonna
be
much
easier
to
maintain.
B
J
B
Is
it
your
replacement
for
nginx?
It
could
be,
but
usually
you'd
have
an
ingress
in
front
of
your
cluster.
It
could
be
an
edge
X,
ingress
or
as
a
proxy
ingress
or
trafficking
grass.
So
you
would
keep
your
own
grass
I
I
guess,
but
it
can
do
some
of
the
things
that
ingress
can
do
and
also
it
doesn't
run
as
a
pod,
each
one's
as
pods.
So
if
you
take
a
look
at
everything
that
eesti
Owens
is
installing
it's
basically
that
ok
here
I've
got
the
full
set
of
features
installed.
B
It's
called
the
demo
profile
and
it's
installing
an
ingress
gateway.
It
seems
to
lean
the
sidecar
stretching,
etc.
So
it
runs
in
inputs.
So
you
need
a
not
so
little
cluster
at
the
beginning,
but
you
can
make
it
run
locally.
So
if
you
have
Dhaka
desktop
or
mini
cube,
there
is
a
set-up
guide
for
you
to
install
eesti.
Oh,
you
should
really
try.
It's
not
complicated,
give
it
a
try
and
you
like
it.
Thank
you
very
much.