►
From YouTube: Istio Environments Meeting 2021-09-29
Description
Istio Environments Meeting 2021-09-29
A
All
right
good
morning,
folks,
I
think
we
should
be
good
to
get
started
pop
right
into
it.
So
the
first
item
on
the
agenda.
This
is
something
we
discussed,
I
think
two
or
three
weeks
ago.
It's
the
proposal
for
prioritized
leader
election.
So
basically,
for
those
of
you
who
aren't
aware,
we
want
the
default
revision
to
handle
all
of
the
things
that
are
guarded
by
leader
election.
A
I'm
the
main
discrepancy
last
time
was
whether
we
needed
to
change
the
actual
kubernetes
leader
election
library
that
we
rely
on
to
make
this
happen,
and
I'm
pretty
convinced
that
we
actually
do
after
working
on
it
for
since
the
last
meeting
so
yeah
this
is.
The
proposal
is
pretty
much
in
good
shape
if
we
could
get
formal
approval
on
it.
That
would
be
great
and
we
also
have
a
a
pr.
That's
that's
up
and
ready
for
review,
so
yeah,
just
kind
of
request
for
comments,
requests
for
feedback.
A
A
So
the
only
change
now
is
that
when
you
do
that
same
thing
to
change
the
default,
it
will
also
make
it
so
that
it
handles
like
ingress
status,
controller
updates
and
gateway
status,
controller
updates
and
all
the
things
that
only
one
revision
should
be
responsible
for
and
it'll
just
happen
automatically.
So
it's
kind
of
cool
like
if
you
change
the
revision,
but
it's
your
control
tags
at
default.
A
B
Yeah
from
a
user
perspective,
it's
it's
it's.
Basically,
things
will
work
as
expected
in
the
sense
that
the
controllers,
that
matter
I
mean
ingress,
control,
upgrade
and
so
forth,
will
be
controlled
by
the
default
revision,
which
is
you
know
what
you
would
expect
if
you're
doing
a
canary
upgrade.
You
don't
want
canary
to
suddenly
start
picking
up
those
core
functionality.
A
E
A
However,
you
want
to
do
it,
however.
We
decide
that
the
best
way
to
change
the
default
with
helm
is
that'll
just
work
with
the
default
later
election,
and
also
it
works
well
with
older
revisions,
and
it
works
well
when
there
is
no
default
as
well,
so
it
won't
block
one
of
so.
If
you
have
three
revisions,
none
of
them
are
default
for
some
reason,
it'll
still
pick
one
of
them.
One
of
them
will
still
win
the
later
election.
So
nothing.
B
B
B
A
B
A
B
Yeah
we
call
it
mesh
and
because
it's
it's
also
supposed
to
be
a
bit
more
independent
to
to
what
mesh
it
is.
I
mean
it
can
be
easier.
It
can
be
other
people
who
are
using
other
meshes,
and
you
know
easy
to
have
a
consistent
way
for
auto
configuration.
Okay,
let's
take
it
offline.
If
you
wanted,
I
think
we
had
an
agreement,
mostly
okay,.
A
A
Okay,
the
next
item
yeah.
So
this
is
with
respect
to
the
proxy
config
crd.
So
last
time
I
think
we
discussed
it
as
a
working
group
together,
we've
kind
of
settled
that
we
wanted.
If
you
have
a
proxy
config
crd
present,
it
should
just
replace
whatever
is
set
in
the
annotation
and
in
the
existing
mesh
config
proxy
config
stuff.
A
B
F
D
B
If,
if
we
move,
I
mean
there
are
plans
to
move
the
control,
the
proxy
out
of
the
pod,
so
so
it's
you
know
that
in
cni
or
whatever,
I
think
they're
all
kind
of
efforts
to
do
this.
If
that
is
the
case,
then
the
pod
will
be
completely
powerless
to
to
to
mess
with
security
or
with
other
things,
so
the
admin
can
can
enforce
that.
You
know
it
cannot
be
bypassed.
Basically,.
D
B
F
B
D
B
A
A
B
B
C
Well,
the
migration
challenge
is
a
conflict.
What,
if
there's
a
conflict
right
between
the
customer
rituals
in
that
name,
space
or
customer
resource?
That's
selecting
the
workload
and
what?
If
they
are
different
from
the
annotation
so
which
one
is
going
to
win
and
their
workload
might
be
broke
right
because
you
might
be
selecting
a
proxy
config.
That's
not
what
user
wanted
or
the
admin
maybe
selected,
something
that's
conflict
and
that
could
have
broke
the
service.
B
But
then
it's
very
simple:
if
you,
if
you
use
annotation,
we
keep
using
annotation,
we
are
not
removing
it.
There
is
no
no
problem
with
that.
So
if
you
have
a
port
that
is
using
annotations,
there
is
absolutely
no
incentive
to
add
the
proxy
config.
If
you
start
fresh
with
a
new
port,
then
you
just
start
using
proxy
config.
B
B
A
We're
just
promoting
a
tiny,
tiny
subset
of
what
is
configurable
through
the
annotation.
So
if
we
just
completely,
if
we,
if
the
logic
is
like,
if
there's
a
proxy
config
annotation
in
the
namespace,
just
totally
ignore
the
annotation.
Sorry,
if
there's
a
proxy
copic
crd,
just
totally
ignore
the
annotation,
then
how
can
the
user.
A
B
B
B
C
B
D
I
think
it
should
be
reasonable.
One
thing
I
want
to
make
sure
that
we
do
support,
though,
is
like
right
now
with
external
control
plane
you
have
to
configure
a
bunch
of
stuff
like
addresses
and
root
certs
and
whatever
you
should
be
able
to
set
that
at
the
top
level
and
then
not
let
the
lower
levels
override
it.
D
B
B
B
A
B
A
One
one
thing:
that's
still
not
clear
to
me,
so
I
I
see
that
if
there's
an
annotation
existing,
then
we
just
don't
look
at
the
crd.
That
makes
sense
for
back
backwards
compatibility.
So
is
it
going
to
be
the
same
idea
for
the
top
level
mesh
config
default,
convict
like
if
anything
is
set
there?
Then
we
totally
ignore
the
global
proxy
config
crd.
B
B
A
B
A
B
B
C
C
D
The
one
difference
I
would
agree
we
probably
should
align
with
what
they
did.
I
you
should
double
check
what
I
said:
I'm
not
certain
that's
the
case,
but
the
one
difference
here
is
that
then
telemetry
there's
no
need
for,
like
the
control
plane
having
a
hard
override,
whereas
here
I
think
we
do
need
that.
D
If
people
don't
have
any
clue
what
I'm
talking
about
in
external
h2d,
what
we're
doing
is
like
it
doesn't
make
sense
for
the
user
to
control
things
like
discovery
address
or
the
root
certificate
for
xds
or
a
few
other
fields,
because
that's
like
the
control,
plane's
decision,
and
so
those
are
kind
of
hard
coded
at
the
control
plane
level.
Whereas
other
settings
you
know,
can
be
configured
by
the
user.
So
I
think
that's
important
to
keep.
B
John,
it's
not
entirely
clear.
I
mean
that
it's
a
strict
requirement
that
they
cannot
be
overridden.
I
mean
the
requirement
is
that
if
you
create
a
pod,
it
will
work
as
expected,
but
if
you
explicitly
set
discovery
address
to
something
else,
as
you
said,
I
mean
you
can
just
not
inject
or
do
other
things
to
do
to
bypass.
D
A
Beginning
of
the
meeting,
I
think
it
mostly
makes
sense,
I
think
mandar
actually
from
his
comment.
He
he
kind
of
brought
up
the
telemetry
api
as
the
case
that
you
know,
like
they're,
already
doing
a
merge
that
protects
certain
keys
that
shouldn't
be
messed
with.
So
I
I
have
no
idea
what
the
telemetry
api
needs
protected
there,
but
how?
How
are
we
disaligned
with
them?
I'm
not
100
clear
there,
because
I
thought
this.
B
B
A
E
B
E
Okay,
the
reason
why
I'm
I'm
bringing
this
up
is
I'm
wondering
if
there's
more
of
you
know
some
kind
of
directive
that
you
know
all
teams
can
be
kind
of
working
through,
so
that
we
make
sure
we
kind
of
you
know
aren't,
aren't
doing
something
a
little
bit
differently
from
each
other.
So
then
you
know
users
have
a
different
experience
from
you
know,
one
layer
you
know
to
another,
so.
B
A
B
But
but
annotations
are
the
ones
that
we're
supposed
to
deprecate
at
some
point
and
and
kind
of
move
people
to
the
consistent
one.
So
how
about
the
crd
as
defined
is
following
exactly
what
sven
said
and
and
what
we
discussed
with
no
mention
of
annotation
except
one
either.
If
annotation
is
present,
then
you
know
we
are
going
to
it's
going
to
take
over
or
whatever
I
mean
that,
but
not
not
do
merging,
because
that's
not
part
of
the
general
issues
tragedy
because
they
don't
have
annotation
except
this
one.
D
A
It
okay,
that
makes
sense
too
one
thing
I
was
actually
looking
through
some
implementation
stuff
and
one
thing
that
confused
me
was
the
presence
of
a
proxy
config
discovery
service
and
it
looks
like
it
only.
It
only
functions
with
certificate
like
some
value
with
certificates.
I
I
don't
think
I
have
the
context
on
that.
Does
anybody
know
what
that
is?
I.
A
B
Yes,
the
idea
when
we
discuss
this
is
that
since
proxyconfig
can
be
pushed
by
by
xds
server,
we
have
the
ability
at
runtime
to
to
change
those
fields
without
restarting
support.
So,
ideally,
all
the
fields
in
proxy
config
will
be
pushed
dynamically.
So
so
you,
you
will
not
have
to
restart
the
board
ever,
but
implementation
wise,
some
fields
are,
unfortunately,
cannot
be
required
to
start.
So
so
that's
why
we
kind
of
try
to
document
one
or
the
other.
A
B
D
B
A
F
F
The
first
thing
I
brought
up-
and
there
was
a
slight
bit
of
contention
on-
was
that
I
wanted
to
split
out
the
promotion
so
that
multi-cluster
didn't
encompass
like
every
version
and
topology
and
just
kind
of
captured.
Like
you
know,
the
the
you
know,
one
good,
install
method
which
will
be
multi-primary
and
then
just
like
the
core
cross
cluster
service
discovery
feature
rather
than
covering.
F
F
Cool
and
then
the
second
thing
are
just
a
few
lists
of
items
and
docs.
Probably
the
most
commonly
asked
question
is:
how
do
I
control
you
know?
How
do
I
make
a
service
only
talk
inside
of
one
cluster
or
how
do
I
make
it?
So
I'm
only
talking
to
a
service
in
a
specific
other
cluster.
If
I
have
more
than
two
clusters
and
then
we
have
several
ways
to
do
that,
you
know
we
have
this
old
mesh
config
setting,
which
is
just
a
global
setting.
F
You
can
always
just
create
separate
service
resources
that
have
different
names
in
each
cluster.
If
you
really
want
things
to
be
completely
isolated
and
you
have
control
of
that
and
then
we
do
now
have
a
synthetic
label
that
you
can
use
in
your
destination
rule
selector
to
control
this
as
well,
and
so
these
are
all
useful
and
they
all
have
their
own
cases
where
they're
better
than
others,
and
so
I
just
want
to
make
sure
that
people
are
actually
aware
they
exist.
F
B
Comment,
mesh
config
service
settings
is
not
an
option
because
it's
a
you
know,
alpha
api
and
it's
part
of
mesh
config
and
it's
so
if
we
want
to
have
an
option,
we
can
put
it
in
proxy
config,
maybe,
but
even
though
I
don't
know
how
to,
but
maybe
we
should
just
keep
fewer
options.
Really
I
mean
just
a
destination
label
which
is
needed
anyway
and
and
per
service.
Is
you
know,
part
of
what
user
can
do
without
any
yeah.
F
Sounds
good,
so
that's
like
the
biggest
thing
I
want
to
cover
in
docs.
The
next,
like
most
common
mistake,
is
trying
to
have
two
services
of
the
same
name
that
are
very
different,
like
kubernetes
position
on
this
is
that
if
you're
doing
multi-cluster,
then
you
need
your
name
spaces
to
kind
of
be
vaguely
the
same,
and
so
I
want
to
link
to
their
definition
of
sameness
and
kind
of
expand
on
it.
For
you
know
any
cases
that
istio
has
and
then
just
highlighting
troubleshooting
documents,
and
that
should
be
mostly
it.
F
And
then
the
next
thing
would
be
tooling:
the
bug
report
command
right
now
we
don't
really
tell
users
and
some
of
them
get
confused
that
you
need
to
actually
run
it
separately
against
each
cluster.
It
might
be
possible
to
add
support
to
just
make
it
automatically
run
against
all
the
clusters,
but
we
don't
have
that
today
and
then
I
also
want
to
just
add
something
to
it.
F
F
B
B
B
F
Yeah,
it
is
bigger
than
multi-cluster,
but
I
remember
it
was
somebody
mentioned
it
as
like
a
feature
promotion
requirement.
If
you
don't
have
it
for
everything
that
might
be
a
little
looser,
but
I
at
least
want
to
get
some
manual
test
run
with
that
I
can,
or
just
manually
trigger
our
integration
tests
against
the
setup
that
has
the
multiple
versions
installed.
B
F
F
So
just
but
the
thing
that
is
not
well
tested
and
might
have
bugs
is
removing
clusters
from
an
existing
mesh
using
the
cluster
local
controls
and
the
destination
rule
label,
you're
able
to
move
traffic
away
from
a
cluster.
But
right
now
it
there's
no
way
to
fully
remove
a
cluster
from
pilot's
view.
Without
stod
restart,
it
will
try
to
purge
all
the
services.
But
I've
observed
a
couple
cases
where
it
doesn't
do
it
correctly,
and
so
that's
the
my
biggest
concern
as
far
as
a
bug
that
would
block
us
from
promoting.
B
B
F
What
we
ended
up
doing
is
you
know
if
informers
and
and
reachability
to
the
apis.
The
like
remote
api
server
starts
to
fail.
We
have
a
bunch
of
metrics
that
users
can
monitor
to
know
that
they
need
to
start
moving
traffic
away
from
that
as
well.
But
to
tie
those
two
things
together,
it
seems
pretty
dangerous.
B
F
B
B
B
And
you
know
you're
having
an
inconsistent
state
and
again
that
infrastructure
we
have
today
where
we
we
get
the
endpoints.
If
we
had
synchronization
like
what
I
think
nate
or
someone
else
was
working
on
with
endpoint
slice,
synchronization
and
then
persistence,
then
we
could
have
a
consistent
approach,
but
that's
a
new
feature
that
is
probably
not
even
close
to.
F
Okay
and
then
testing
is
pretty
good.
It's
been
pretty
good
for
a
long
time.
There's
a
few
things
that
have
just
got
skipped
that
need
to
be.
You
know,
unskipped
or
we
need
to
find
out
if
they're
real
bugs.
So
I'm
going
to
look
into
those
and
then
the
only
real
feature
that
we
want
to
make
sure
we
support
better
is
hella
service
and
staple
sets.
Staple
sets
are
a
little
weird
because
you
could
end
up
with
name
conflicts
across
clusters
and
right
now
we
do
have
kind
of
partial
support.
F
The
way
it'll
work
is
when
there's
a
name
conflict.
It
will
choose
the
more
local
endpoint,
but
that's
still
not
very
good,
and
I
don't
really
recommend
people
using
it,
but
we
do
have
a
couple
of
users
who
have
tried
out
the
experimental
multi-cluster
headless
support
using
the
dns
proxy
and
dns
auto
allocation.
B
B
Work
because
again
you
you,
when
you
deploy
a
stateful
set
in
a
cluster,
it's
you
know,
it's
presumably
sharding.
You
know
the
data
into
the
three
stateful
sets
and
if
you
I
mean
she
needs
to
be
aware
of
this,
I
mean
it's.
It's
kind
of
something's
application
built
in
is
doing
the
shutting
you
cannot
chart.
B
F
B
F
A
F
So
I've
done
a
few
proof
of
concepts
where
I've
used
the
destination
rule
label.
To,
like
you
know,
do
traffic
shifting
and
find
ways
to
slowly
move
traffic
away
from
a
cluster,
but
there's
nothing
that,
like
truly
drains
long-lived
connections
that
we
have
and
then
once
that's
done,
you
can
delete
the
remote
secret
and
stop
watching
that
other
cluster
and
when
you're
doing
traffic
like
that,
it
seems
to
work
pretty
well.
F
But
I
know
recently
john
sent
me
a
case
where
an
extra
remote
secret
that
was
like
malformed
was
added
and
then
deleted,
and
then
the
endpoints
were
never
cleaned
up
and
I'm
not
sure
if
that
bug
is
reproducible
outside
of
a
case
where
you
know
like.
I
don't
know
if
I
can
reproduce
that
case
where
the
remote
secret
was.
You
know
correct,
because
essentially
what
had
happened
is
there
was
two
remote
secrets
with
different
cluster
names,
but
they
pointed
to
the
same
cluster
when
the
bad
cluster
name,
one
was
removed.
F
Every
endpoint
for
every
cluster
was
wrong.
It
was
all
pointing
to
just
one
cluster,
even
though
there
was
like
four
clusters,
and
I
haven't
been
able
to
replicate
that
yet
and
so
that's
why
I'm
somewhat
concerned
about
the
ability
to
just
delete
a
remote
secret
and
assume
everything
is
going
to
be
cleaned
up
properly.
F
F
B
F
F
B
F
C
C
F
B
C
F
G
D
D
Yep
mine,
too
cool
the
other
thing
was
just
distrulis.
This
is
we
talked
about
this
a
bit
in
the
test
release,
but
just
wanted
to
bring
it
up
here
because
there's
some
relationship,
I
think
we're
planning
to
ramp
up
on
on
this
a
bit,
making
it
more
sported,
adding
better,
tooling
docs,
making
more
test
run
on
it,
etc.
D
We
still
keep
keep
it
as
an
option,
not
the
default
now
yeah.
So
for
now
it
will
be
an
option
opt-in
in
the
future.
I
think
we
want
to
move
in
the
direction
where
it's
a
default,
we'll
always
keep
the
other
one
as
an
option.
I
think,
but
that
would
be
a
wider
discussion.
I
mean
there's
a
lot
of
implications
there
in
terms
of
backwards
compatibility
and
debug.
B
B
We
discuss
in
the
past
of
having
some
flag
that
says,
production
versus
development
mode
and
based
on
that
we
can
have
different
circuits.
For
example,
in
production
we
have
mtls
at
the
default.
We
can
have
distress
as
a
default.
We
can
have.
You
know
the
annotations
that
we
discussed
earlier
ignored
and
so
forth.
I
mean
kind
of
strict
mode
versus
development
model.
However,
you
want
to
call
it.
D
Yeah
yeah
in
terms
of
the
debugging-
that's
probably
the
biggest
blocker
for
us
ever
making
this
default.
But-
and
I
know
we've
been
burned
by
keps
in
the
past,
but
I
believe
that
they're
actually
going
to
have
a
thermal
containers
on
by
default
in
kubernetes,
1.23
and
so
real
users
may
have
it
in
their
clusters
by
say
march,
if
they're
early
adopters,
so
it
may
may
actually
finally
happen.
I
don't
have
a
whole
lot
of
a
lot
of
faith,
but
it
is.
B
D
All
right
yeah-
that's
all
I
want
to
mention
we'll-
have
bigger
discussions
in
the
future
if
we
try
and
make
it
default,
but
for
now
we're
just
trying
to
make
it
better,
perhaps
beta
except
I
realized
that
1.12
is
way
sooner
than
I
thought.
So
I
there's
a
very
minimal
chance
that
that
will
happen
when
not
12,
but
perhaps
when
at
13.