►
From YouTube: kubernetes kops office hours 20190719
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Hello,
everybody:
it
is
Friday
July
19th.
This
is
cops
office
hours.
I,
am
your
moderator,
facilitator,
Justin,
Santa,
Barbara
I
work
at
Google
a
reminder.
This
meeting
is
being
recorded
than
what
we
put
on
the
internet.
I
just
pasted
a
link
to
our
agenda
into
the
chat.
Please
do
put
your
name
name
in
there
or
all
putting
our
name
in
at
the
same
time.
A
Put
our
name
put
your
name
in
there
and
and
also
if
you
have
any
agenda
items,
please
do
pop
them
on
the
bottom
or
the
appropriate
place
in
the
agenda,
and
then
we
can
get
through
them
all,
because
we
do
have
a
lot
of
stuff
on
there
already
I'm,
not
sure
why
don't
you
just
get
started?
The
first
item
on
the
agenda
is
around
that
I
say
that
please
be
mindful
of
our
code
of
conduct
and
be
a
good
person.
A
The
first
up
on
our
agenda
is
about
the
release
status.
This
is
something
where
I
have
failed.
Everyone
on
I
apologize,
but
I
thought
we
could
try
something
more
more
new
where
we
basically
go
through
the
things
promotions.
The
releases
that
we
intend
to
do
and
like
basically
like
do
a
roll
call
on
any
blockers
on
them
and
then
we
sort
of
know
what's
happening
each
time
so
underneath
our
agenda
so
I
don't
know
if
I
should
cast
the
screen
for
this.
A
Maybe
I'll
try
underneath
our
agenda
I
put
a
list
of
what
I
believe
the
releases
are
for
this.
Whatever
this,
we
can
figure
out
what
cadence
it
is.
But
let's
say
this
like
this
two-week
period
or
ideally
today,
in
this
weekend,
I'm
gonna
share
my
screen
just
so.
Everyone
gets
it
on
the
video
that
looks
like
the
right
one.
A
A
It
might
be
too
big.
We
will
see
it's
thinking
about
it
anyway.
Well
that
thinks
about
it.
There
we
go.
A
A
So
we
need
to
make
sure
that,
basically,
in
all
the
releases
that
were
gonna
do
that
the
calico
CV
is
in
there
I
think
thank
you
to
everyone
that
sent
PR
so
that
I
think
most
of
them
have
it,
but
it's
just
like
we
don't
want
to
release
accidentally
without
them
being
in
there.
I
don't
know
if
anyone
else
knows
of
any
blockers
that
would
block
cups,
one
13-0
the
release
and
he
a
little
slower
on
this
one,
because
it
is
a
real
release.
A
A
When
we
do
the
113
zero,
we
then
can
do
the
beta
of
114
0.
We
can
do
the
Alpha
of
115,
zero
and.
A
We
can
also
do
our
first
alpha
of
116,
because
kubernetes
has
published
their
first
alpha
of
116,
so
we
are
in
our
ketchup
period.
We're
gonna
have
two
concurrent
alphas,
which
is
a
little
weird
and
I
will
probably
do
I
guess
I
were
released
cups
116
alpha
from
the
master
branch
rather
than
so.
We
will
do
so
from
at
least
one
Kitty
if
that
makes
sense
to
everyone.
So
we
won't,
because
we
have
two
alphas
we're
not
gonna
like
go
crazy,
crazy
on
the
cherry-picks
okay,
the
a.m.
eyes
are
the
next
on
the
list.
A
So
you
know,
in
our
channels
we
have
like
the
default
a.m.
is
that
we
recommend
there
are
some
in
the
Alpha
Channel
that
I
think
fixed
a
CVE
from
a
couple
of
maybe
a
month
ago,
so
I'm
just
gonna
propose
that
we
put
that
into
the
stable
Channel
I,
don't
know
if
any,
when
I
say
blockers
on
that
all
right.
Otherwise,
I
was
just
gonna,
do
a
refresh
of
the
a.m.
eyes
and
put
them
in
the
Alpha
Channel
I'm,
not
really
aware
of
any
particular
reason
to
do
so
other
than
like.
B
The
channel
update
PR
that
I
have
outstanding,
oh
before
we
before
we
cut
beta
before.
B
A
A
A
I'll
put
on
the
others,
then,
because
it
doesn't
make
sense
to
not
the
so
on.
The
AM
is
I
think
it
sounds
like
we
are
okay,
promoting
from
alpha
to
stable,
which
will
pick
up
that
CVE
I,
think
I
know
some
people
have
been
complaining
or
mentioning
that
it
wasn't.
There
I'll
do
an
ami
refresh
to
get
another
one
in
the
pipeline
and
then
kubernetes
versions.
I
guess
the
simple
one
is
for
one
twelve:
nine,
where
its
current
in
the
alpha
channel.
We
should
promote
that
to
stable,
because
there
is
now
a
one
1210.
A
Unless
anyone
knows
of
any
reason
which
we
should
put
in
the
alpha
channel
and
then
I
guess
in
two
weeks
we
can
talk
about
promoting
that
or
whatever
cadence
we
decide
the
I,
and
then
we
would
put
1
13
8
into
1
13
8,
the
latest
113
114
and
115,
into
both
the
Alpha
and
stable
channels,
because
they
are
all
pre-release
versions.
I,
don't
know
if
anyone
has
any
objections
to
any
of
those
bumps
or
know
of
any
problems
with
any
of
the
versions.
A
Once
it's
the
same
thing
for
116,
okay,
I,
don't
know
if
that
is
handy
for
people
but
I
figure.
If
we,
if
we
do
that
every
two
weeks,
then
I
will
know
exactly
what
I'm
supposed
to
be
doing
in
terms
of
releases,
other
people
can
send
PRS
we
can
have.
We
can
use
this
as
sort
of
the
approval
process
by
which
we
get
to
a
more
automated
flow.
Maybe
what
if
people
think
thumbs
up
thumbs
down?
I,
don't
okay,
all
right,
we'll
see
if
I
can
stick
to
it
now.
A
I
need
everyone's
help
to
stick
to
it.
But
let's
try
this
all
right.
Now,
let's
go
into
the
so.
If
I
you
will
try
to
do,
yeah
I
will
try
to
do
all
those
today.
I
think
the
this
one
I
think
is
going
through
the
pipeline
and
I
think
Ryan.
You
just
need
to
rebase
and
we
can
then
get
that
through
the
pipeline.
Yep.
A
C
Sure
yeah
so
I
don't
know
if
this
is
the
right
venue
or
so
feel
free
to
redirect
me
to
issues
or
whatever.
But
I
wanted
to
talk
about
a
quick
issue
that
we
ran
into
and
I
was
curious
if
we
could
either
update
the
docs
or
if
it's
just
a
non-issue
and
you're
doing
something
wrong,
but
we
followed
the
upgrade
from
SCT
to
that
CDPH
process.
That's
outlined
in
the
document.
C
The
migration
document
we're
lovingly
calling
that
process
the
hammer
method,
because
it's
just
pretty
much
hand
resolved
master's
away,
and
we
noticed
that
when
they
come
back
when
the
new
masters
come
back,
they
have
this
like
a
little
reconciliation
period
in
which
they
go
from
all
the
old
masters
are
ready
and
all
the
worker
notes
are
ready,
but
all
the
new
masters
are
not
ready
and
when
the
new
masters
become
ready.
It
marks
all
of
the
nodes
in
the
cluster.
Other
nodes
in
the
cluster
is
not
ready
at
that
point.
C
C
If
anybody
else
had
experienced
that,
and
it
seems
we
when
you're
testing,
it
seemed
to
be
more
common
on
larger
clusters,
so
there
might
be
kind
of
a
race
condition
on
which
that
loop,
that's
nodes,
sync
loop
happens,
which
is
100
seconds,
so
I
just
wanted
to
kind
of
bring
that
up
and
see
if
either
we
could
like
document
that
be
like
this.
This
process
will
cause
a
service.
What
you
know,
cluster
wide
service
outage,
or
you
know,
ways
to
mitigate
it.
C
A
C
A
Once
they
are
not
ready,
that
is,
it's
not
overly
surprising.
They
get
removed
there.
There
is
elsewhere,
a
disruption,
controller
that
that
will,
for
example,
so
if
your
API,
if
you're
a
diverse
API,
tells
you
you
don't
have
any
instances
if
a
dopesick
guy
tells
you
that
one
instance
has
gone
away,
Kerberos
will
delete
the
node.
If
the
a
diverse
API
tells
you
that
I
believe
more
than
two-thirds,
like
all
of
the
nodes
have
gone
away,
it
will
not
blind
kubernetes
will
not
blindly
delete
all
your
notes.
It
doesn't
trust
the
API
at
that
point.
A
I
don't
think
there
is
a
similar
behavior
for
load
balancers
like
it's
pretty
obviously
bad,
to
remove
all
the
nodes,
I
would
say,
but
I
don't
know
of
any
guard
against
that.
So
I,
not
great,
it
makes
sense,
but
yes,
I
think
if
we
can
figure
out,
if
you
want
it
in
the
dark
saying
this
happens,
that's
wonderful.
If
you
want
to
dig
in
that's
great
I
would
definitely
look
at
why
the
nodes
go,
not
ready.
That
seems
like
the
or
like
any
way
to
prevent
them
going
out
ready.
A
D
E
E
Once
the
new
masters
come
into
play,
then
all
the
nodes
go
not
ready
for
a
while,
and
it's
just
this
kind
of
cascading
issue
the
way
we've
gotten
around
it
so
far
as
to
temporary
add
an
ion
policy
to
the
Masters,
denying
them
the
ability
to
remove
instances
from
load
balancers,
and
we
just
have
that
policy
in
place.
While
we
are
rolling
all
the
Masters
and
then
as
soon
as
they
come
back
up
and
everything
has
stabilized,
we
remove
that
policy.
That
is
a
good
fix.
A
It's
not
I
mean
this,
it's
a
little
bit
like
what,
but
it's
it's
actually
probably
I
would
say.
That's
a
nice
fix
to
be
honest.
You
should
I
am
probably
going
to
submit
some
documentation.
Should
we
add
that
as
what
potential
workaround
I
think
that'd
be
a
wonderful
yes,
that
is
a
great
suggestion.
I,
like
this
cube,
like
these
things,
tend
to
be
rabbit
holes
where
you
chase
them
down
and
eventually
you
find
something
and
we
might
not
be
able
to
fix
it.
Where's
the
I
am
workaround
feels
very,
very,
is
targeted
and
I
think.
A
C
Yeah,
we
also
noticed
like
endpoints,
should
also
be
removed
with
any
nodes,
so
endpoints
would
be
effective
with
not
ready
state
for
notes
as
well,
but
we
didn't
actually
see
that
being
enacted
in
the
cluster.
So
maybe
that's
that's
part
of
that
safeguards
stuff.
You
were
talking
about
in
another
controller
or
if
there's
a
I
know.
It's
interesting,
though
interesting
that
is
interesting.
Yeah
I,
it's
possible.
A
D
D
This
one
away
sure
so,
a
couple
weeks
ago,
all
of
ours
were
failing
their
tests
because
of
kubernetes
changed,
and
these
no
longer
allow
certain
node
labels
to
be
applied
by
the
cubelet
on
its
own
node
and
cops
relies
on
that
for
tagging.
The
node
role,
and
there
were
there-
were
discussions
in
the
enhancements
floor.
Request
is
linked
there.
So
I
was
just
curious.
What
our!
What
copses
workaround
for
that?
It's
going
to
be.
Yes,.
A
So
the
underlying
thing
is
there
is
an
attempt
to
prevent
security
issues
where
a
cubelet,
hypothetical
secure
issues
is
not
unknown.
One,
a
cube.
If
you,
if
you
escaped
and
I
get
permission
on
a
node,
you
could
real
able
your
cubelet
as
a
real
able
your
node
as
a
master
or
a
whatever
privileged
labels.
A
You
can
find
in
the
cluster
and
you
could
steer
workloads
to
yourself
and,
in
theory,
steal
the
secrets,
like
there's
a
complicated
node
admission
controller,
which
basically
tries
to
reduce
the
vast
radius
of
a
node
by
by
preventing
by
preventing
any
any
access
from
the
node
to
a
secret.
That
is
not
on
a
pod
mounted
to
that
node.
But
if
the
pod,
if
the
node
can
reel
able
itself,
then
that
sort
of
circumvent
that
at
least
very
narrowly
I
mean
I
feel
like
there's
a
million
ways
around
that.
But
anyway,
that's
what's
behind
us.
A
So
it's
acceptable
to
use
a
controller
to
add
those
same
nodes.
I
do
have
one
in
a
sort
of
working
progress
branch.
It
is
based
on
coop
builder,
so
it
needs
like
once
we
cut
the
114
branch.
I
can
put
it
on
because
it
what
we
have
here
on
116-
that's
not
true,
but
once
we
once
we
get
all
the
releases
done,
then
I
can
I
can
start
that
going.
I
do
think
in
general.
A
So
I
think
if
you,
if
you
have,
if
you
have
uses
for
the
node
role
label,
specifically,
that
enhancement
would
be
helpful
because
I
think
Clayton
is
thinking
that
there
are
no,
that
there
are
no
real
uses
for
it,
and
so,
if
there
are,
then
that
would
be
helpful
to
share
with
him,
because
I
think
he
has
a
perspective.
That
is,
you
know,
from
a
different
environment
than
the
one
note,
but
most
of
us
in
the
kubernetes
community
actually
have
all
right.
Is
that
is
that
and
does
that
answer
what
we
yeah
I?
A
So
and
yes,
our
testing
right
now
is
I,
think
blotter
115,
but
hopefully
we
can
get
that
controller
in
or
figure
out
that
we
effectively
we
want
like.
Maybe
we
should
just
stop
using
the
labels.
I,
don't
think
we
should,
but
we
could
decide
to
stop
using
them
and
deprecated
them
ourselves,
but
that
feels
like
most
of
our
most
of
our
manifests
rely
on
those
labels.
So
it
is
a
much
harder
thing
to
to
do.
A
The
sequence
good
question,
the
sequence
is,
he
node
is
allowed
to
set
its
provider
ID,
which
includes
the
instance
ID.
The
controller
looks
up
the
instance
ID
in
ec2
checks
that,
like
from
there
goes
to
an
instance
group
from
the
instance
group,
goes
to
the
set
of
labels
that
can
be
applied.
It's
actually
a
it's.
A
really
nice
like
this
is
sort
of
getting
similar
to
the
node
authorizer
controller
as
well.
So
it
will
be
much
more
secure.
I
think
will
be.
A
D
A
One
of
the
things
we
can
do
is
if
we
do
it
by
a
cluster
API,
we
might
want
to
have
a
workaround.
We
might
have
a
different
mechanism
there,
maybe
and
if
we
we
might
going
to
do
it
by
the
node
authorizer
as
well
like
relying
on
the
provider.
Id
is
one
mechanism,
but
maybe
we
could
find
another
mechanism
that,
like
securely,
does
something
at
at
handshake
time
but
yeah.
A
G
G
And
the
second
thing
is
that,
a
few
weeks
ago,
we
moved
when
city
manager
and
a
crystal
works
and
I
try
to
understand
how
it
works
under
the
hood,
since
I
didn't
find
many
technical
documentation,
I
say
so.
I
submitted
a
PR
with
the
communication
about
what
I
understood
what
I
said.
I
would
be
glad
anyone
like
you
that
worked
and
that
may
take
a
look
and
give
any
feedback
if
correct
incorrect
in
order
to
to
improve
it
to
and
eventually
merge
it
for
other
people
as
well
to
better
understand
out
the
back
up.
G
A
A
Let's
call
it
two
weeks:
I'm,
just
gonna
call
it
two
weeks
for
now
and
we'll
see
if
we
change
our
mind
for
this
two
weeks
and
that
I
put
a
link
to
your
that
I
will
like
go
and
look
your
PRS
and
thank
you
so
much
for
doing
those
Doc's
I
will
definitely
have
a
look
at
that
and
I'm
sure
they
are
correct,
but
I
I
will
I
will
take
a
look
anyway.
Okay,
I'm,
not
so
good.
Okay,.
A
H
A
Okay,
yes,
I,
don't
know
if
there's
a
we
don't
need
container
on
time
configuration
okay,
that's
a
good
question.
I
feel
like
we
can
try
something
yeah,
that's
a
good
question.
I
will
think
about
that.
One
I
will
comment
on
that.
I,
don't
have
a
good
answer.
I,
don't
think.
One
else
is
a
great
answer
for
how
we
should
do
it.
We
probably
should
we
something
in
general,
I
think
we're
starting
to
see
and
I.
Don't
know
if
this
came
up
on
the
agenda.
A
So
it's
a
pointer
if
you
look
at
that
that
actually
Maps
pretty
directly
to
how
we've
been
doing
component
configuration
like
the
intent
our
component
configuration
is,
is
like
running
with
where
component
configuration
was
three
years
ago
and
it
looks
like
we
have
ended
up
in
the
same
place.
So
that
is
great
news.
The.
A
We
should
probably
do
the
same
thing,
therefore,
for
container
D
I,
don't
think
container
D
will
have
a
configuration
block,
but
we
should
probably
imagine
that
it
had
a
component
config
and
go
from
there
or
maybe
it
does
have
a
component
company
and
see
if
we
can
do
that,
I
know
actually
has
a
configuration
file.
That
is
a
Tamil
file,
but
that
does
imply
a
certain
structure.
So
if
we
try
to
mirror
that
structure,
I
think
we
will
be
in
good
shape,
but
yeah,
hopefully,
lots
has
any
thoughts.
H
H
So
one
proposal,
someone
said,
is
that
it
actually
might
be
in
gr,
PC
and
gr
PC
in
SC
d.
Upstream
actually
has
you
know
they
mentioned
a
fix,
so
I
opened
a
quick
PR
for
sed
manager,
adding
an
additional
version
of
that
CD
that
has
the
G
or
PC
base,
but
I'm
working
on,
seeing
if
I
can
tested
one
of
our
clusters.
A
H
A
J
I
This
is
about
insisting
with
the
conflict
based
option
and
cop
state
store,
because
right
now,
if
they're
different
cop
states
tour
ends
up
for
the
cluster
spec
and
the
rest
goes
from
conflict
base,
so
I
was
wondering
what
would
be
the
best
option
here:
internal
sources
through.
Do
we
take
the
coffee
base?
Already
cop
state
store
the.
B
A
A
That's
sort
of
that's
the
rule
of
thumb,
but
because
we
don't
yet
have
the
CRT
phone,
charlie,
I'm,
pretty
sure.
As
you
point
out,
there
are
lots
of
inconsistencies.
I
think
if
we
follow
that
rule
of
thumb
that
cops
state
store
could
be
a
community
service
and
config
base
should
be
a
storage
area
that
is
shared
with
the
instances
mm-hmm.
Then
we
would
be.
That
is
that's
the
rule
of
thumb.
Yeah.
A
I
G
A
D
C
A
A
Hey
will
I
will
decline
from
commenting
how
about
that
the
we
do
have
this
grid.
If
I
can
find
it
there,
we
go
yeah,
okay,
cool,
so
we
have
yes,
these
all
the
jobs
are
currently
failing
that
are
running
with
with
160,
which
makes
sense.
We
sort
of
know
that
and
yes,
we
need
to
fix
that,
and
it
looks
like
all
the
jobs
that
the
jobs
that
are
pinned
to
a
version
are
okay,
except
for
113,
which
apparently
is
flaky.
So
we
actually
shouldn't
look
at
that
output,
its
volume
so
yeah.
A
A
J
Everyone
else
I'd
say
fifth
of
July
off
sorry,
hey
no
worries,
so
I
have
two
things:
I
want
to
talk
about,
so
this
first
one
is
this
hostname
override,
so
the
short
version
is,
if
you
click
the
issue
and
scroll
all
the
way
to
the
bottom.
J
So
basically,
the
proposal
here
is
to
replace
the
the
way
we
cap,
we
compute
the
names
to
match
how
kubernetes
is
doing
it,
and
so
the
table
there
I
had
in
the
issue
of
the
bottom
is
basically
so,
if
you
didn't
have
the
DHCP
options,
enabled
it's
no
change
to
you,
things
just
work.
If
you
had
the
DHCP
options,
it
starts
working
if
you
are
using
node
authorizer
and
if
you
weren't
it's
the
same
so
because
this
will
also
solve
the
issue
which
you
people
may
have
noticed
like.
A
The
ELB
thing
we
saw
we
took
earlier,
but
I
I'm
in
favor
of
the
nativist
ocean,
override
I,
think
it
makes
some
sense.
I
also
agree
that,
yes,
we
should
defer
the
the
there's
some
duplicated
code
and
I
think
we
can
defer
fixing
that
into
a
later
PR.
But
yes,
this
is,
as
you
say,
there
should
be
no
change
for
anyone,
not
using
DHCP.
There
is
one
extra
ec2
call
or
one
extra
yeah
to
metadata
call
is
one
easy
to
cool,
but
the
ec2
call
is
the
same
one
that
cubelet
does
so.
A
K
Back
to
112,
probably
yeah,
do
you
need
it
and
once
well
we're.
A
J
Three,
so
the
first
one
the
fix
went
in
that
did
a
shallow
copy,
and
so
my
most
recent
fix
that
went
in
whatever
like
two
weeks
ago,
actually
does
a
deep
copy
of
the
map,
but
now
I'm
hitting
it
again
with
completely
different
cultures.
So
I'm
not
sure
if
anybody
is
familiar
with
how
this
weave
works.
Mesh
stuff
works
because
I
don't
even
know
where
it's
getting.
A
J
A
The
yeah,
it's
an
internal
library,
that's
used
by
weave
it's
not
necessarily
particular
widely
used
outside
of
weave
and
in
sed
mender,
for
example.
We
did.
We
used
a
different
one
because
it
has
some
limit.
This
one
is
has
like
some
of
the
surprising
stuff
and
like
it
does
all
sorts
of
like
clever
like
distributed
system
stuff.