►
From YouTube: Kubernetes SIG K8s Infra - 20230215
Description
A
Hi
and
welcome
to
the
kitchen
from
meeting
we
are
February
15.,
so
just
a
reminder
that
this
meeting
is
under
the
cover,
counter,
gonna
be
recorded
and
we'll
publish
on
YouTube
later
so
invite
everyone
in
this
call
to
be
mindful
and
respectful
of
each
order.
So
please
can
I
I
will
try
I'll,
put
the
link
of
the
docs
into
some
chat
for
anyone
new
and
first
thing:
do
we
have
no
ticker.
A
Let
me
see
we
can
go
to
oops
first
order
of
business,
we
have
new
people
in
the
call
that
want
to
introduce
themselves.
A
Okay,
we
have
no
one
okay,
so
we
can
go
directly
to
the
building
review
this
time.
I'm
gonna
to
I'm
gonna,
do
three
things
here,
mostly
two
things
like
Google
gcp
course
and
go
about
the
best
cars,
and
we
can
see
so
I'm
trying
to
see
if
we
can
basically
change
the
way
we
do
building
Repro
between
two
Cloud
providers
of
so
first
change.
We
can
notice
it's
basically
the
fact
that,
thanks
to
Ben,
we
cannot.
We
flip
the
traffic
from
artifact
registry
to
a
screen.
C
Oops
me,
you
can
see
it
pretty
well
in
the
overall
cost
trend
on
the
gcp
billing
page.
Let
me
see
if
I
can
figure
out
how
to
share
that.
A
Okay,
so
science
February
9,
we
have
a
obviously
a
regional
cost
related
because
before
that
date
we
were
observing
well
we're
serving
controller
image
for
IP
for
basically
IP
in
from
non-advis
IPS,
and
now
we
Now
flip
the
cars
to
S3
and
we
see
basically
a
reduced
in
your
cars
and
then
when,
basically,
we
we
both-
we
went
from
200
and
it's
growing
again.
So
we
need
to
monitor
that
closely.
Maybe
we
can
I
think
Buy
in
two
weeks,
which
is
end
of
the
month.
A
We
will
see
the
impact
monthly
and,
if
I
go
over
the
cost
Explorer
for
database
for
AWS,
we
see
basically
signs
that
date
hero.
We
see
basically
cost
increasing,
because
we
now
serve
all
the
blobs
here
for
from
S3
by
default.
Basically
yeah
February,
11
I
think
it's
that
weekend,
yeah
that's
the
weekend.
C
Okay,
yeah
I,
think
never
mind.
I
was
gonna,
say
it's
a
little
bit
confusing
because
it's
still
showing
that
there's
3,
000
or
3.7
mil,
even
though
the
0.7
portion
is
from
the
the
like
extra
credits
from
the
end
of
last
year,
which
are
expired,
but
I
think
the
total
remaining
credits
number
is
still
correct,
based
on
three
mil
minus
the
spin.
So
far,
oh
yeah,
that
that's
just
I
think
it's
a
little
bit
easier
to
see
there.
The
the
green,
the
you
can
see.
C
The
green
bar
has
shrunk,
a
bunch
that
that
green
bar
is
artifact
registry.
A
E
C
Something
like
a
200k
savings
if
we
don't
have
anything
else,
changing
the
trend,
but
that's
still
pretty
far
from
what
we
need.
A
F
Let
me
ask
one
question
here:
so:
last
year
we
did
we,
we
ended
up
doing
some
analysis
of
logs
and
things
like
that
right.
Do
we
want
to
do
one
this
year
and
when
do
we
do
it.
A
F
What
do
you
mean
logs,
looking
through
logs
for
who's,
accessing
from
which
IP
those
kinds
of
things
we
ended
up?
Analyzing
right
to
figure
out
like
who's
pulling
the
images
and
from
where.
A
Oh
yeah,
okay,
the
so
just
to
refresh
what
team
said
like
last
year,
we
spent
basically
CNC
I
I
did
an
exercise
to
identify
where
the
the
cause
traffic
coming
the
most
and
we
realized
that
was
basically
AWS
at
some
point
this
year.
I,
don't
think
we
need
that,
because
now
we
basically
thanks
to
the
logic
in
the
oci
proxy.
We
basically
shift
the
traffic
based
on
the
IPI
traces.
A
C
C
Like
we
have
a
pretty
good
idea,
the
approximate
breakdown
of
like
which
clouds
pull
kubernetes
images
from
the
previous
analysis,
I,
don't
think
that's
going
to
have
massively
changed
so
I'm,
not
sure
that
that's
we'd
have
to
retool
to
read
from
the
new
logs
and
I'm,
not
sure
how
essential
that
is.
If
it
is,
though
we
can,
we
shut
down
some
of
that
stuff
because
it's
actually
reasonably
expensive
to
to
be
constantly
doing
this
large
analysis
right.
C
F
For
probably
asking
that
question
is
you
know
to
see
how
much
like
we
we've
been
trying
to
like
change
our
code
and
we've
been
trying
to
tell
people
to
switch
Registries
and
all
that
right,
like
is
any
of
the
actually
effective
or
not?
Was
the
the
curious.
C
I
think
you
can't
directly
correlate
those
because,
like
say
we
change
tool
like
chaops
to
start
referencing,
the
new
registry.
The
impact
depends
on
when
users
pick
up
the
new
tool
right,
correct
and
that
we
don't
have
that.
We
don't
have
that
granularity
of
information.
Clients,
don't
tell
us
anything
more
than
this
is
containerd1.x
and
I'm
pulling
this
image
from
this
IP.
It
doesn't
tell
us
what
tool
they're
using.
C
F
D
I
know
you
said
it:
yeah
Bob
was
asking
if
it
was
possible
to
get
a
breakdown
of
what
images
were
pulled
mainly
to
look
at
which
version
kubernetes
versions
are
being
actively
pulled.
E
A
C
We
could
we
could
look
at
like
the
we
could.
We
could
identify
certain
images
and
look
at
which
tags
are
most
popular,
it's
theoretically
possible
to
implement
something
that
says
this
is
the
kubernetes
version.
Is
the
most
pulled
I'm,
not
sure
that
it's
gonna
help,
though
like
we
just
want
to
push
all
users
to
move
as
soon
as
possible
across
versions.
F
A
A
G
I
have
one
so
when
do
we
have
money
in
our
budget?
It
would
probably
good
at
it
but
load,
balancing
logs
and
stuff
sitting
in
our
buckets
to
approximate
how
much
of
image
X
was
pulled
by
this
particular
user
agent,
for
example.
So
we
can
kind
of
get
an
idea
of
some
images
of
something
that's
full.
For
example,.
G
No,
we
should
like,
when
we
have
a
based
on
budget
to
gather
analytics
of
like
so.
C
Going
to
tell
you
like
containerdy
1.6
is
the
most
popular
client
or
something,
and
that
that
doesn't
tell
us
what
cluster
like
your
type
of
cluster
you're,
using
or
or
anything
like
that.
We
can
tell
you
what
we
could.
We
can
find
out
something
like
what
cloud
you're,
probably
in,
or
what
container
runtime
version
you're
using.
C
G
You're
running
based.
A
G
I
don't
know
sticking
to
the
first
first
party
report,
I
think
it's
something
that
we
should
do
right.
A
Okay,
assuming
assuming
basically
we
get
that
information
and
we
realize,
for
example,
basically
kubernetes
120-
is
the
most
qbency
cluster
installed
there.
What
we're
gonna
do
about
that,
knowing
that
its
deprecate
is
not
maintained
anymore,
but
we
have.
We
still
have
people
using
communities
120.
What
we're
going
to
do
about
that.
G
A
G
Know
what
you're
going
to
do
with
the
information?
What
I'm
saying
is,
it
would
be
great
if
first
party,
as
it
does
to
host
infrastructure,
are
able
to
release
a
report
says:
oh,
we
want
to
continue
registry,
and
this
is
what
we
were
pulling,
and
these
are
all
the
you
made
these
versions.
We
think
people
are
using
over
a
particular
period.
C
A
I
so
there's
an
issue
related
to
that
somewhere.
We
just
we
just
need
to
keep
the
additional
close
and
when
we
are
ready
to
make
implementation
of
that,
we
can
come
back
to
that.
C
A
C
It
will
give
some
approximation
for
the
moment,
assuming
that
we
don't
wind
up
adding
other
hosts
later.
It's
still
an
expensive
thing
to
be
continuing
to
do.
Presumably
at
some
point
we'll
do
other
useful
things
with
the
AWS
and
yeah
I
think
that
the
expense
is
not
super
great
relative
to
the
like
actual
usefulness
of
it.
C
C
A
Okay,
we
can
resume
the
conversation
by
trying
to
Define.
What's
the
what's
the
use
of
the
those,
basically
all
the
data
we
collect
from
image
pooling
and
we
can
have
a
separate
conversation
about
what
we
want
to
to
do
with
this
information
later.
H
Thank
you,
hello,
I
yeah.
This
is
a
new
section
where
we
thought
we'd
just
quickly
go
over
the
various
projects
and
someone
at
this
I
put
a
link
to
a
I
guess:
I
put
a
link
to
a
very
GitHub
project
board.
You
kanban
thing,
I,
don't
know,
but
yeah
it
seems
to
be
like
yeah,
quite
cool,
actually
quite
easy
to
use.
H
So
from
an
engineering
point
of
view,
I
was
like
quite
pleased
with
that
made
me
feel
like
I
was
a
PM
and
we
are
doing
okay,
I,
think
on
on
moving
everything
over
I
think.
Basically,
everything
is
stood
up.
H
We
have
a
Sandbox
environment,
artifacts,
Dash,
sandbox.case.io
It's,
redirecting
to
some
AWS
buckets
and
those
are
also
fronted
by
cloudfront,
as
sort
of
a
fallback
fronted
by
Cloud
run.
We
have
all
of
that
same
infrastructure
also
stood
up
in
prod.
We
just
the
last
piece
we
have
to
do
is
actually
cut
over
the
artifacts.cates.io
DNS
name
so
that
we
actually
serve
prod
traffic,
the
fear
or
the.
H
We
know
that
we
believe
we
will
take
between
15
and
30
minutes
of
downtime,
because
we
can't
get
a
TLS
certificate
until
we
cut
over
the
DNS.
We
believe
I'm
trying
to
figure
out
if
there's
a
way
around
this,
and
so
once
we
cut
over
the
DNS,
then
it
will
try
to
require
the
certificate
that
will
take
about
15
to
30
minutes,
but
we
will
not
be
serving
in
that
time.
It's
probably
an
ideal
blocker
there
isn't
that
much
currently
on
artifacts.case.io.
H
There
is
a
way
around,
but
anyway
that
is,
that
is
where
we
are
and
so
I.
The
next
step
is
basically
to
cut
over
I'm
trying
to
track
down
internally,
but
there's
a
better
way
and
Muhammad.
Is
it?
Do
you
wanna?
H
H
B
Certificate,
when
you
have
control
of
your
DNS
domain
is
pretty
straightforward.
We
do
it
all
the
time
on
a
quick
stand
up
where
you
use
the
certbot
CLI
and
specify
the
the
domains
you
want
to
use,
and
it
tells
you
the
text
records
and
you
could
submit
those
via
the
case.io
repository
for
I.
Think
we
use
octo
DNS
something
yep
it's
manual,
but
it
it'll
get
us
there
and
you'll
have
to
assert.
You
can
do
it
now
at
the
way.
G
B
Whatever
that's
kind
of
what
I'm
referring
to
is
that
you
do
do
good,
a
wild
concert
when
you
do
that
approach
using
even
just
a
CLI
right.
It's
just
getting
those
TLS
records.
Sorry
text
records
from
cert
manager
in
there.
H
A
I
think
there's
like
a
technical
detail
on
how
we
do
basically
the
flip.
We
can
take
that
on
slack
and
trying
to
figure
out
later,
basically
together
how
we
can
make
that
simple
for
all
we're.
Just
in
there's
like
we
can
have
a
maintenance
window
and
say:
oh
at
father,
kisser
is
down
for
what
one
one
hour
I'm
fine
with
that,
so
I
think
I.
Don't
think!
There's
too
much
to
worry
about
here.
D
I
think
we
kind
of
already
did
with
Ben's
cut
over
so
really
that
was
it
I
think
to
his
thing
saying
he
looked
it.
D
Wouldn't
say,
registry
decades
that
I
was
necessarily
done.
I
still
think,
there's
different
things
that
we
could
try
and
do
to
minimize
costs.
But
the
latest
update,
like
in
terms
of
initiatives
and
things
we're
trying
to
work
towards
registry.case.io.
The
latest
update
has
been
did
a
bunch
of
I,
don't
know
food
and
Magic,
we'll
call
it
that,
but
it
did
work,
reduce
costs,
correct.
C
In
terms
of
reducing
costs,
there's
not
a
lot
left.
The
biggest
one
is
just.
We
have
to
get
people
to
use
it.
We've
offloaded
pretty
much
all
bandwidth
over
to
the
S3
buckets
now.
If
those
become
a
cost
issue,
then
we
may
be
looking
at
doing
other
things,
but
for
now
it
appears
that
it's
not
the
case
and
on
gcp
we
have
dramatically
reduced
the
cost
of
cost
to
run
this
and
I.
C
Don't
think,
there's
a
lot
of
room
for
improving
that
there
is
still
some
work
happening
on
the
registry
to
smooth
Mohammed
has
been
working
on
freezing
case.gcirio
as
part
of
getting
users
to
move
over,
like
new
images
will
not
be
available
on
Kate's
IG
sardale
to
only
be
on
the
registry
and
we're
working
on
two
parallel
approaches
for
that.
C
It
has
shipped
some
PRS
to
bring
up
another
GCR
that
we
use
to
be
able
to
sync
from,
but
we
don't
Point
end
users
at
and
I've
been
working
on
a
tool
to
read
the
contents
from
artifact
registry
and
populate
them
to
S3.
So
we
no
longer
need
to
depend
on
on
GCR
for
that
step,
so
we
should
be
pretty
close
to
immediately
shipping
the
intermediate
GCR
and
we're
also
getting
relatively
close
to
being
able
to
to
eliminate
it
but
running
those
in
parallel,
so
that
so
that
we're
not
blocked
on
this
release.
C
And
that
should
not
pay
off
in
the
immediate
future,
but
it's
an
important
step
towards
ensuring
that
people
really
do
switch
off
of
GCR,
because
they're,
just
the
images
will
cease
to
be
available.
We
started
advertising
registry.case.o
and
125,
but
even
since
then
ever
all
content
has
been
dual
published
so
that
the
Dual
publishing
is
going
to
end
and
we
have
a
fair
bit
of
work
that
has
been
happening
to
ensure
that
ships
and
I
want
to
thank
Muhammad
for
capturing
all
that
in
a
cap
and
driving
for
it.
C
Although
work
to
do
the
more
immediate
infra,
config
only
approach
and
get
that
done
and
for
working
with
stick
release
on
that.
C
I
actually
have
one
for
Muhammad.
Is
there?
Is
there
anything
that
your
blockman
again
I
took
a
quick
poke
at
the
issues.
G
The
registry
was
created
it's
public,
but
that
can
be
fixed
later,
so
we
need
to
merge
the
pr
test
even
further
axles.
It
I
think
that's
a
priority
right
now.
So
that's
going
to
take
a
few
days
to
work
itself
up.
A
Okay,
I
think
it's
just
follow
up
on.
Basically
my
mid
work,
so
we
can
follow
up
on
Slack.
D
Last
I
saw
at
least
when
it
comes
to
the
main
topic
of
building
build
clusters,
Marco
assigned
it
to
himself
five
days
ago
and
I
think
he's
on
the
call.
So
he
can
update
it
better
than
me.
I
Yeah,
okay,
folks
I
have
started
looking
into
this.
We
will
go
forward
with
eks
cluster
and
I
will
I
have
got
access
to
the
account
server
we'll
be
using
and
yeah
I'm.
Looking
into
that,
I
have
two
days
off
today
or
tomorrow
about
as
a
Friday
I
will
be
back
to
take
a
look
into
this
it'll
hope
to
kind
of
stuff
it
by
the
end
of
the
week.
A
D
A
F
Hang
on
I
know
so
when
we
say
that
we're
gonna
do
some
stuff
on
eks
yeah.
Are
we
implicitly
saying
yes
we're
going
to
use
the
same
pattern
we've
been
using
before
using
terraform?
Yes,
thanks,
Marco.
A
I
want
to
refresh
the
question.
I
want
to
give
a
different
answer
for
your
question.
I
think
for
the
moment
we
don't
care
about
tooling,
I!
Think
right
now,
I'm!
Basically,
you
know
we
should
not
care
about
trolling.
We
should
basically
give
room
for
Marco
to
bootstrap
quickly
boost
up
the
AKs
cluster
matching
the
requirement
and
trying
to
plug.
Without
and
later
we
focus
on,
yeah
swelling
yeah.
A
D
A
F
C
The
the
main
cluster
that
we're
using
that's
still
inside
Google
is
actually
from
2016
and
kubernetes
1.4.
We
almost
never
need
to
change
the
cluster.
We
need
to
use
the
kubernetes
API,
so
anything
that
gets
us
to
a
cube.
Config
should
be
fine.
Terraform
is
nice
for
visibility,
but
I
think
not
essential.
Okay,.
E
G
C
Yeah
I
can
follow
up
more
on
that
offline
as
well.
There's
a
process
to
add
an
additional
Cube
config
to
prow
that
we've
done
before
for
Community
generated
gke
clusters.
We
should
be
able
to
do
more
or
less
the
same
thing
there.
You
just
add
it
basically
just
a
config
format
that
has
a
a
name
for
the
cluster
and
then
the
like
Cube
config
used
to
access
it.
It's
pretty
simple.
A
Okay,
can
I
go
for
my
subject:
do
it?
Okay,
so
dear.k.io
is
kind
of
in
pending.
For
for
the
tourism,
the
one
is
like
we'll
need
TLS
certificate
from
fastly
to
ensure
we
can
serve
through,
firstly,
cash
right
now
the
configuration
is
almost
ready,
so
we
now
require
TLS
certificate
and
that's
part
of
the
donation
from
fasting.
So
we
in
the
agreement
we
got
with
fussy.
If
I
remember
we
supposed
to
get
60,
terabyte,
monthly,
plus
five
TLS
certificate,
and
once
we
got
all
of
this,
we
can
move
forward.
A
A
We
don't
give
notice,
we
sleep
and
people
yell
at
us
and
come
with
pitchfork
and
torch
so
I
got.
We
can
also
do
that,
but
I
would
like
to
basically
send
some
a
communication
notice
to
the
community
and
to
Downstream.
D
If
you
want
my
mood
fast
break
things
opinion,
you
should
give
14
days
notice,
give
them
two
weeks.
Otherwise,.
D
C
A
couple
things
on
that,
so
so
one
to
to
the
point
just
made:
yes,
that
is
built
to
Google
internally,
currently
so
it's
nice
to
have,
but
it
doesn't
help
with
the
public
bill
yeah.
So
in
terms
of
speed,
we
don't
necessarily
have
to
rush,
however,
I'm
kind
of
against
giving
a
really
long
notice,
because
I
don't
want
to
set
some
kind
of
precedent
that
we're
being
supportive
of
depending
on
this
sort
of
thing,
deal.cates.io
is
already
a
community
controlled
domain
that
serves
a
redirect
to
some
kind
of
storage.
C
If
you
decide
to
depend
on
where
it
redirects
to
and
resolving
that
directly
or
allow
listing
it
or
whatever
that
is
it,
that's.
That
was
a
bad
call
on
your
part.
We
have
never
ever
said.
You
should
do
that
and,
unlike
the
container
images,
there's
a
much
lower
chance
of
this
breaking
reasonable
production
setups
overnight.
C
C
We
like
there
are,
there
are
repos
that
have
direct
references
to
the
cloud
storage
bucket
and
they
should
not
be
doing
that
and
I'm
sure
it
exists
and
we're
going
to
stop
pushing
things
there
so
that
FYI
needs
to
go
out,
but
I
don't
think
it
should
block
a
timeline
for
when
we
rotate
do.
We
know
how
much
bandwidth
we're
using
currently
have.
Has
anybody
actually
pulled
that
data?
Do
we
know
of
60
terabytes
is
enough.
That
would
be
my
bigger
concern.
We.
C
Okay
but
you're
saying
we're
going
to
roll
it
out
and
it
sounds
like
we
haven't
checked
that
yet
so
I
think
we
need
to
check
that
first.
I,
don't
think
I
have
access
to
that
I'm
happy
to
go,
poke
and
find
someone
that
does
it
sounds
like
the
answer
is
no.
We
haven't
checked
that
and
we
really
should
check
that.
First.
C
E
C
I
will
find
out
I
I,
think
I
know
someone
that
has
access.
Okay.
If
that
information
is
available,
I
think
I
know
someone
has
access
to
the
project,
that's
hosted
in.
A
C
C
C
A
H
I
think
I
think
my
hand's
next,
you
simply
get
more.
Are
we
gonna
put
a
redirect
service
in?
Are
we
going
to
keep
a
redirect
service
in
front
of
DL
in
front
of
these
downloads,
or
are
we
going
to
just
hand
over
to
fasting
because
I
I
totally
agree
like
if
we,
if
we
stand
up
either
another
Porsche,
porch
Porsche
or
the
same
porch
thing,
we
can
like
just
put
fastly
in
the
mix
and
send
them?
However
much
they'll
take
or
we
can
the
thing
and
then
we're
sort
of
stuck
right,
yeah.
A
A
Makes
sense
yeah
we
keep
that,
like
I,
think
I,
my
constant
would
be
basically
I,
think,
is
angenix
and
send
off
to
handle
the
traffic
we're
gonna
flip,
because
other
than
that
we
also
have
a
lot
of
things
not
going
through
that
redirect.
Basically,
we
have
a
lot
of
I
saw
in
the
cake.
I
put
a
lot
of
links
going
directly
to
bucket
I
would
like
to
change
that
to
use
fastly,
so
I
think
that's
my
most
pressing
concern
so
anyway,.
C
It's
a
that's,
a
pretty
irreversible
thing
and
I'm.
Actually,
no.
G
You
can
just
blast
the
bucket
right
empty
the
bucket
and
make
it
useless
again.
A
C
Yeah,
but
also
to
be
clear,
this
is
not
billed
to
our
three
million
gcp
right
now
and
no
one
is
Raising.
It
stuff
is
complicated.
No
one's
raising
a
really
big
stink
about
it
right
now,
it's
more
of
a
like.
It
would
be
nice
to
be
able
to
show
that
we've
reduced
some
other
cost
whenever
we're
asking
for
the
overage
on
the
main
Bill,
and
because
we
want
to
bring
everything
out
of
internal
projects
into
community
control,
but
leaving
it
up
is
not
a
big
problem.
I
mean,
for
example,
GCR
dot.
C
Io
Google
containers
has
also
we
haven't
determined
shutting
that
down
yet,
and
it
still
has
some
usage,
but
since
it's
not
on
the
community
bill,
it
doesn't
it's
not
as
pressing.
C
We
have
less
of
a
hard
cap
when,
when
it's
like,
when
it
stays
entirely
internal
to
like
gke,
as
opposed
to
when
we're
going
external
I,
don't
actually
understand
why
that's
the
case
that's
way
above
my
pay
grade,
but
I
know
it's
the
case.
E
C
A
Yeah,
so
let's
have
this
amazing
conversation
about
this
trying
to
get
that
information
and
make
a
final
decision
in
two
weeks,
but
to
be
clear:
I'm
gonna
go
with
the
notice
because
we
need
to
tell
people
we're
doing
that
flip.
It's
not
about
expectation.
It's
just
about
saying
to
people.
Oh
you're,
gonna,
there's
gonna
be
an
impact
on
your
infrastructure.
D
A
little
bit
and
reorganize
what
initiatives
we
can
kind
of
put
them
on
so
truly
any
negative
feelings
towards
coopermatic.
Having
not
like
started
work
sooner
should
be
directed
directly
at
me.
I
will
fall
on
that
sword.
Second,
off
there's
been
kind
of
a
balancing
act
that
we
unfortunately
have
to
do
between
our
employers
and
the
community.
Hopefully
you
get
what
I
mean
so
for
the
most
part,
this
proposal
is
focused
on
that
balance,
but,
like
I've
I,
said
to
dims
in
a
slack
thread.
D
K
Maybe
I
can
I
can
also
get
give
some
context.
What
what
is
this
proposal
and
why
you
find
several
points
in
it
odd
or
or
strange
so
kilometric
was
approached
by
the
cncf
to
help
create
a
multi-cloud
and
multi-registry
set
up
for
the
kubernetes
infrastructure,
and
we
didn't
have
any
idea
or
I
mean
we
had
some
idea
how
everything
worked,
but
from
our
from
our
colleagues
that
are
in
the
community
and
that
are
active
in
the
community.
K
But
this
was
also
done
in
regards
to
what
Ben
already
mentioned,
that
eventually
everything
should
be
moved
out
of
private
Google
accounts
into
a
potential
multi
multi-cloud
multi-vendor
setup
that
has
a
unified
interface
to
manage
everything,
and
but
everything
I
mean
the
pro
clusters
that
are
that
are
doing
their
thing,
also
creating
the
test,
the
the
the
clusters
for
for
end-to-end
tests
to
run
for
the
upcoming
version,
so
also
the
life
cycle
for
those
clusters
and
anything
that
might
be
coming
up
in
the
future.
K
For
this
sick
or
this
project
needs
some
kubernetes
clusters
to
run.
So
the
idea
behind
this
was
to
create
like
a
a
universal
infrastructure
that
is
manageable,
with
as
many
as
less
resources
as
possible
and
also
is
potentially
not
utilizing,
managed
Services,
because
a
managed,
kubernetes,
Class,
A,
managed
AKs,
rks
and
gke
cluster
is,
in
the
end
more
expensive
than
running
ec2
instances.
K
So
when
we
look
into
the
future
from
now
like
now,
we
we're
talking
about
in
five
years,
the
costs
will
eventually
always
go
up,
and
so
our
approach
was
as
we
we
are
in
an
open
source
project
and
that
basically
solves
this
multi-cluster
approach
and
multi-cloud
approach.
We
proposed
our
own
open
source
tool
for
this.
There
were
some
quick
wins
that
we
we
wanted
to
achieve,
and
this
is
also
where
we
we
talked
to
III
and
we
talked
we
got.
K
We
got
your
input
and
we
got
the
input
of
of
Ben
that
and
we
eventually
said
that
yeah,
let's,
let's,
let's
go
with
the
approach
in
the
beginning
that
everything
runs
as
fast
as
possible
and
we
can
reduce
the
cost
as
fast
as
possible,
but
also-
and
now
comes
the
the
other
business
side.
K
The
cncf
or
basically
gf's
bosses
said
to
us.
We
want
to
have
a
long-term
solution
and
we
want
to
have
a
long-term
solution
that
might
be
also
could
be
managed
by
sres
or
a
p
companies
that
basically
also
donate
help
in
in
terms
of
engineering
capacity
to
maintain
those
infrastructures,
because
we
are,
we
are
facing
a
lot
of
uncertainties
there.
So
nobody,
nobody
knows
in
half
a
year
who
is
managing
the
infrastructure
in
the
end.
K
I
know
the
community
does
a
great
job
in
in
doing
this,
but
I
think
we
can
all
agree
that
no
one
wants
to
be
the
person
who
is
called
on
a
Sunday
evening
at
3
A.M
to
fix
something
that
is
wrong
in
the
infrastructure.
K
And
from
this
perspective,
we
came
up
with
with
the
architecture
that
that
we
proposed
and
I
understand
the
the
yeah.
The
the
feedback
and
I
mean
also
a
Marco
already
foreseen
this,
but
made.
Hopefully
this
gives
some
more
context
onto
the
onto
this
whole
proposal.
Yes,
dims.
F
So
one
of
the
points
I
was
talking
to
somebody
and
they
made
was
like
hey,
we
use
Sono
boy
and
other
things
from
single
render
projects.
So
that
is
not
the
problem
here.
You
know
whether
it
is
a
open
source
project
from
here
or
there,
who's
working
on
the
open
source
project.
That's
not
the
point
like
terraform
is
also
an
open
source
project,
but
the
point
here
is:
where
is
the
pain
currently
and
is
the
pain?
F
What
is
the
problem
that
this
tool
is
trying
to
solve,
and
do
we
actually
have
that
problem
so
that
that
is
the?
That
is
where
we
have
to
look
this
look
at
this
from
and
like
we
do
have
clusters
we
end
up
using
whatever
is
there
on
the
managed
kubernetes
site,
whether
it
is
gke
or
eks?
F
The
thing
is,
we
don't
have
a
problem
of
hey.
We
have
to
build
a
cluster
and
destroy
a
cluster.
You
know
many
times
a
day
or
any
anything
of
that
sort.
Right
like
Ben,
was
saying
we
created
a
cluster
years
ago
and
we
are
still
using
it
in
for
pro.
So
that
is
not
a
problem.
We
are
trying
to
solve
here
right,
so
if
we
want
to
solve
the
problem
of,
you
know
the
CI
and
how
we
use
current,
how
we
currently
do
CI
there
is.
F
There
is
a
big
problem
there
we,
which
we
haven't
even
dug
into,
which
is,
we
have
a
cube
up
script
in
the
cluster
directory,
and
that
is
the
one
that
creates
all
this.
You
know
all
the
Clusters
that
we
use
for
testing
right,
so
that
is
where
we
need
help
to.
F
Essentially
we
there's
two
modes
there
right
like
one
is
pre-summit
jobs
and
post
Summit
jobs.
Pre-Summit
jobs
will
need
to
integrate
the
stuff
in
the
pr.
We
need
to
build
all
the
artifacts
and
then
build
a
cluster
using
those
artifacts
right.
We
don't
have
a
good
answer
for
that
right
now
in
the
community
other
than
the
queue
of
scripts,
and
we
keep
you
reusing
the
coupon
scripts,
because
we
don't
have
anything
else.
Even
the
scalability
jobs
are
heavily
dependent
on
things
that
are
similar
to
that.
F
A
F
Me
finish
that
so
that
is
a
good
problem
to
solve,
and
that
is
a
long-term
problem.
We've
been,
you
know,
hitting
our
head
against
and
the
issue
there
is.
F
We
can
build
these
artifacts
quickly,
using
the
make
file
that
that
we
already
have,
and
we
can
we,
the
problem
is,
we
have
to
put
it
in
an
army
or
whatever.
You
know,
image
that
that
is
there
for
the
for
the
specific
cloud
and
then
you
have
to
deploy
right.
That
is
the
problem
that
we
are
currently
having,
which
is
the
pre-summit
jobs,
it's
very
slow
to
be
able
to
do
that.
It
might
work
for
the
Post
Number
jobs,
but
not
for
the
pre-summit
jobs.
F
So
if
we
want
to
chase
a
problem,
that
is
a
problem
that
I
want
to
go
chase,
which
is
which
will
make
us
do
well
in
the
longer
run,
because
currently
the
cube
of
scripts
doesn't
even
reflect
any
of
the
installers
that
are
out
there
right.
All
of
the
installers
use.
Cubadm
and
the
cube
of
script
doesn't
even
use
cubanium,
just
for
example,
so,
first
we
need
to
identify
the
problem
we
are
trying
to
solve,
and
this
is
not
a
problem.
We
are
trying
to
solve
sorry
I'll,
yield.
K
Yeah,
so
basically,
the
idea
was
also
to
exactly
duck
into
this
so
to
to
create
an
environment.
We
are
quickly
the
Clusters
can
be
tested
in
for
the
different
so
that
any
new
pre-submit
is
is
could
be,
could
be,
could
be
built,
tested
and
run
and
then
basically
reduced
again
to
Ash.
K
The
other
thing
is
why
we
propose
it
as
a
general
platform
is
you
could
basically
manage
everything
in
one
platform
and
I
have
the
commitment
from
our
development
side
that
they
are
also
willing,
and
they
also.
They
also
have
an
idea
how
to
solve
this,
and
this
would
then
yeah.
F
I'm,
all
for
it
Mario
like
if,
if
you
want
to
like
dig
into
that,
I
will
spend
time
with
you
all
and
we
can.
We
can
start
working
on
it
like
really
tomorrow,
like
because
that
has
been
on
on
my
case
for
a
really
long
time,
and
that
is
something
that
I've
been
like
struggling
to
find
a
way
to
like
that
is
a
biggest
risk
that
we
are
carrying
right
now,
because-
and
we
want
to
make
it
work
across
all
the
clouds
right.
F
We
essentially,
we
want
cube
of
scripts
that
will
work
across
the
clouds,
so
we
can
run
the
same
things
using
you
know,
ec2
instances
or
the
Azure,
VMS
and
and
whatnot
go
ahead.
Ben
I.
A
B
Ic
I'll
just
go
really
fast.
One
of
the
things
we're
going
to
need
to
be
able
to
understand
are
kubernetes.
Castings
is
to
have
an
architecture
addressing
the
architecture
for
deploying
it.
The
multi-cluster
PSC
that
I'm
working
on
I'm
going
to
need
to
bring
up
a
primary
Coupe
cost
cluster
that
has
AWS
Iams
to
reach
the
S3
buckets
for.
For
writing
the
data
to
and
in
cooperation
with,
with
coopermatic
understanding,
how
we're
going
to
deploy
not
just
the
to
the
Clusters
that
we
create
on
the
fly.
B
It's
part
of
it's
the
or
the
the
account
that
we
want
to
put
it
in,
because
we
need
to
have
IM
access,
not
the
one
or
for
their
writing,
but
we
need
to
have
access
to
the
billing
and
so
making
sure
that
where
we
put
the
the
the
primary
cluster
has
access
to
the
billing
for
the
consolidation.
B
F
Okay
is
already
integrated
so
that
it
should
not
be
a
problem
right
like
we
can
create
a
e
case
cluster
for
running
the
Cube
cost
and
essentially
plug
in
whatever
you
want
there
hit
IEP.
Oh.
A
A
Just
okay
so
just
to
to
be
clear
about
this,
the
the
one
thing
in
life
we
should.
We
exclude
gcp
from
the
scope
of
this
of
this
POC
slash
diploma
of
cube
class
because
that
might
trigger
costs
on
gcp.
I.
Don't
want
that
at
the
moment,
and
the
second
thing
is
like:
if
you
need
to
have
access
to
the
cons
to
the
global
cost
of
the
AWS
organization,
we
might
need
to
do
something
about
this.
A
A
If
we,
basically,
if
we
need
now,
if,
if
you
need
access
to
the
AWS
organization
billing
data,
we
need
to
do
something.
We
need
to
basically
have
a
separate
configuration
for
this.
A
Okay,
we
have
a
Time
by
one
minute.
We
have
Ricardo
only
one
minute
for
you.
J
Okay,
I
will
be
really
fast,
so
this
is
an
issue
I
have
raised
in
I
guess
two
months
ago.
I
have
this
discussion
already
with
our
known
for
them
on
that.
J
Given
some
good
ideas,
I
just
wanted
to
bring
this
up
again,
because
as
a
project
and
and
for
for
English
in
China
X,
we've
been
trying
to
kind
of
upload
the
usage
that
we
have
from
Pro,
even
for
the
cloud
build
and
bringing
all
of
those
those
things
to
GitHub
and
and
just
using
Pro
infrastructure
for
promotion,
mostly
because
it
would
be
easier
for
us
to
manage
the
branches
and
and
other
things
and
and
the
releases.
J
So
we
don't
have
release
managers
and
all
of
those
people
doing
all
of
the
automations
and
pro
so
I
have
discussed
it
with
the
community
and
I
will
just
like
folks
that
you
take
a
look
into
that
and
if
we
can
maybe
be
the
first
one
with
this
approach.
A
I
I
would
say,
I
think
I'm,
plus
one
or
capability
to
give
basically
to
give
a
subproject
to
use
GitHub
action
to
push
image
to
the
station
creation
string
now,
I
think
there's
like
I,
don't
know
if
the
GitHub
admin
team
need
to
give
the
approval
on
this
I.
A
Think
that's
the
one
thing
like
for
me
in
case
I'm
from
I'm
fine
with
that
we
can
try
to
basically
do
that
for
all
the
staging
Ripple
make
sure
a
each
staging
project
have
the
possibility
to
basically
push
image
from
GitHub
nowadays
also
conversation
with
the
GitHub
admin
team.
We
might
need
the
plus
one
so
try
to
bring
this
to
them
and
see
what's
happening
from
my
side,
kitchen
for
adding
and
plus
one.