►
From YouTube: Code base tour: image builder
Description
Code base tour: image builder and other utilities taken from Kubernetes Release Engineering 20191014
B
A
C
All
right
to
pr's,
when
you
are
probably
referring
to
the
image
builders,
one
Tim,
yes,
okay,
all
right
so
kind
of
the
move
for
this.
So
anyone
who's
aware
of
the
image
builder
pusher
setup
that
we've
got
test,
infer
kind
of
has
it
set
up
so
that
they
can
arbitrarily
build
images
on
post
submit,
and
there
are
a
few
interesting
things
that
come
with
being
able
to
set
that
up.
So
we
had
an
issue
open
and
due
to
to
about
kind
of
determining
what
the
image
building
policy
was
going
to
be
for
the
K
release.
C
So
there
are
a
few
things
that
I
noticed
which
didn't
exist
before
within
test
infra.
There
is
a
there
is
an
image.
Do
two
images
builder
and
you
can
you
can
view,
though
the
readme
for
builder?
It
is
not
super
important
to
go
over
in
the
call,
unless
you
want
to.
Essentially
it's
some
sugar
around
the
G
cloud,
build
some
it
right.
So
that's
submitting
a
bill
to
GC
b
or
google
cloud
built
right.
C
So
one
of
the
cool
things
about
this
setup
is
this
variance
file
right,
so
you
can
see
the
variance
file
in
use
if
we
go
to
the
cubic
ins,
été
test
right,
so
variants
file
here
it
has
a
few
variants
of
it.
Defines
a
configuration
right,
a
configuration
variable
as
well
as
a
few
different
variants
for
go
version,
the
kubernetes
version,
the
basel
version
so
on
and
so
forth.
Right
when
you
call
one
of
these
variants,
while
using
while
using
that
builder
image,
it
will
build
what
you
specify
based
on
those
variants
right.
C
So
we
could.
We
could
potentially
wire
our
our
images
to
do
something
very
similar.
What
I
wanted
out
of
this
was
I
wanted
some
means
to
automatically
build
images
right
on
commit,
and
so
there
was
a
consideration
for
a
small
portion
of
time
about
whether
or
not
we
were
going
to
move
our
images
to
test
in
for
just
for
the
sake
of
getting
that
benefit.
But
since
we've
even
barely
broached
that
topic,
we
can
now
do
it
for
any
repo
right.
So
so
Katherine
has
done
some
work
on
the
image.
C
C
So
we
have
a
few
set
up
here
and
for
someone
who
is
seen
a
does
not
seen
a
pro
job
before
this
is
essentially
it's
essentially
a
kubernetes
manifest
right
with
some
sugar
on
top
of
it
right
so
annotations,
which
define
where,
where
we
want
the
job
to
report
into
on
test
grid.
So
we
have
a
cig
release.
Image
pushes
and
sake
release
master
informing,
and
we
can
view
these
boards.
C
Well,
let's
go
to
the
image
pushes
right
so
flaky
jobs,
and
this
is
kind
of
where
the
PR
came
from.
So
we've
got
a
few
things
to
notice
about
this
job.
One
is
the
decorate.
True
decorate
true
turns
on
pod
utilities,
which
are
again
some
sugar
around
building
testing
for
jobs.
If
you're
building
a
new
job
and
testing
for
you
should
use
pod
utilities,
the
next
piece
is
branches
are,
is
run,
have
changed.
C
And
run
have
changed
is
as
a
regex
that,
basically,
if
something
in
the
repo,
if
you've
touched
something
in
the
repo
that
matches
that
reg
X
it
will,
it
will
trigger
this
job,
post
submits,
are
after
your
PR
emerges.
There
are
set
of
jobs
that
will
be
triggered,
so
you
can
define
that
by
just
specifying
post
minutes
your
repo
name
of
the
job,
the
cluster
that
it's
running
on
this
is
important.
C
It's
running
on
the
test
in
for
a
trusted
cluster,
because
the
trusted
cluster
has
access
to
some
of
the
credentials
and
the
most
important
one
is
the
credentials
that
we're
using
to
to
trigger
jobs
within
GCP
right
across
various
projects.
So
now
that
we've
got
the
CNC
F
so
for
each
GC
BP
project,
there
is
basically
we've
granted
permissions
too
proud
to
run
within
the
project
right
and
that's.
If
anyone
wants
to
see
that
at
some
point
it
is
in
I
know
I'm
all
over
the
place.
C
C
This
will
ensure
that
they
have
access
to
staging
access
to
the
GCS
portion
of
that
GCP
project,
as
well
as
setting
up
owners
and
and
the
image
promotion.
Stuff
is
not
really
important
to
this,
but
essentially,
we've
got
configs
4
we've
got
configs
for
a
set
of
different
staging
repos
within
I.
Think
it's
groups.
C
Yeah,
so
here
you
can
kind
of
see
I'll
speed
through
it,
since
it's
got
email
addresses
but-
and
you
can
read
more
about
it
here-
automation
of
Google,
Groups
maintenance.
So
we
we
use
that
file
to
automate
the
Google
Groups
maintenance,
and
then
we
use
those
we
use.
Those
email
addresses
to
grants,
privileges
to
GCP
for
gtp.
C
C
Alright,
so
down
here
and
I
realized,
maybe
I
should
make
that
bigger.
Okay
right,
so
it
it
parses,
a
few
flags
log
directory
a
scratch
bucket
a
project
whether
or
not
allowed
dirty
will
determine
whether
or
not
it'll
allow
you
to
run
the
job
on
the
repo
if
the.
If
the
repo
is
not
clean,
or
rather,
if
things
have
not
been
committed
to
the
repo
right
variant
with
these
are
the
variant
files
I
was
specifying
before.
So.
C
So
we're
checking
to
see
if
the
last
the
last
flag
is
a
positional
argument,
we're
checking
to
see
if
the
image
directory
has
been
provided
and
if
not
we're
going
to
set
an
exit
code
of
one
right.
If
it
is
provided,
then
we're
going
to
use
that
as
the
image
directory
CD
into
the
root
directory
and
then
run
the
build
jobs
right.
So
the
CD
route
is
due
to
right.
C
It's
doing
a
few
things
like
setting
the
scratch
bucket.
It's
taking
the
working
directory,
copying
the
working
directory
up
to
basically
tearing
it
and
then
uploading
it
into
the
scratch
bucket
and
then
it's
it's
referenced
for
the
GCP
job,
the
GCV
job
right.
So
basically
on
tarred
and
then
that's
used
as
a
workspace.
C
Then
it
gets
the
variants
that
you
may
have
specified
if
it
doesn't
see
any
of
the
variants,
it's
going
to
just
try
to
run
a
single
job
with
your
default
config
right
and
then
it
this
sink.
Weight
group
is
essentially
a
mutex
that
waits
for
each
of
those.
If
it's
not
a
single
job,
it's
going
to
wait
for
each
of
these
jobs
to
complete
before
returning
a
set
of
errors
across
these
shops.
Basically,
we
don't
want
to
block
on
the
failure
of
one
job
if
the
rest
and
the
chain
could
potentially
succeed.
C
So
the
PR
to
actually
answer
Tim's
question.
The
PR
changes
a
few
things
so
we're
taking
out
the
CD
2
root
directory
because
it
was
actually
within
the
scope
of
within
a
different
function
scope.
So
it
was
essentially
seeing
if
it
could
CD
into
the
directory,
but
not
actually
doing
that,
which
means
the
jobs
that
were,
which
means.
Essentially,
we
weren't
going
to
the
image
directory.
So
so
this
tool
was
only
working
for
top-level.
C
So
cannot
find,
you
know,
cannot
find
module
module
for
pass
dot
right
because
it
was
not
in
a
module
path.
The
top
level
of
our
repos,
not
a
module
path,
unable
to
evaluate
some
symlinks
in
docker
docker
file
path,
no
such
file
or
directory
again,
there's
no
docker
file
in
the
top
level
of
kubernetes
release
right.
C
So
we
can
look
at
my
changes
really
quickly.
I
also
did
a
test
build
of
the
Builder
tool
and
then
ran
that
locally
to
see
if
it
would
work
right.
So
my
changes
do
to
do
right.
We
add
a
validate
image
directory,
so
I
noticed
that
these
were
these
are
failing
and
they
kind
of
should
not
fail.
If
they're,
if
certain
I
mean
they
kind
of,
should
fail
early.
C
C
If
the
file
info
says
that
it
actually
is
a
directory,
maybe
it's
a
file
that
was
named
the
same
thing:
if
it's
not
actually
a
directory
it'll
actually
fail
it'll
fail
again
right,
then
we
take
the
build
the
cloud
bill,
build
file
and
the
docker
file,
and
let's
just
go,
let's
loop
through
them
again
check.
If
they
is
try
to
satin.
If
they
don't
exist,
say
they
don't
exist
right.
C
So,
let's
bail
out
of
anything
or
Alyssa
bail
out
of
stuff
super
early
before
actually
submitting
it
to
cloud,
build
just
renaming
the
uploading
working
directory
to
uploading
image
directory.
Remove
removing
this
piece,
because
there's
really
no
reason
to
specify
this
config
since
we're
going
to
assume
that
the
default
is
cloud
build
yanil
and
if
we're
actually
seeding
into
the
directory,
then
it
will
be
called
called
cloud
build
yeah.
No,
it
won't
be
appended
with
the
with
the
basing
with
the
directory
name.
Excuse
me,
then,
running
those
jobs
right.
A
C
Okay,
alright,
so
we
did
a
few
things
so,
okay
right,
so
the
cloud
build
for
the
Debian
builder,
for
example.
Right,
it's
just
doing
a
simple
go,
build
a
show
Debs
with
some
environment
variable
set
so
see
you
go
so
you
go
off,
the
OS
is
Linux
and
our
architecture
is
amd64
and
it's
really
just
doing
a
build.
So
it's
using
the
cloud
builders
docker
file
right
and
you
can
read
more
about
this
stuff.
If
you
want
to
here
great.
C
So
the
substitution
loose,
the
substitution
option
is
a
allow
loose
essentially
allows
you
to
if
some
of
the,
if
some
of
the
variables
that
you
specify
are
incorrectly
specified,
it
won't
bomb
on
those
the
right.
So
the
the
the
cloud
builders
docker
container
is
essentially
just
it's
an
entry
point
to
docker
right.
So
we're
saying
docker
build
here.
Are
the
tags
right?
I'm
gonna
use
the
get
tag
of
the
thing
that
just
merged
as
well
as
tag
it
with
latest
right
and
then
build.
C
What
do
I
want
to
build
I
want
to
build
the
the
current
directory
right
and
then
yeah,
so
this
images
portion
will
again
using
the
tagged
and
the
latest
it'll
check
to
see
if
these
were
actually
pushed,
as
these
are
not
pushed
specifying
them
in
images.
These
are
actually
not
pushed
to
the
container
registry.
Within
that
GCP
project.
It
will
fail
right
here,
we're
also
adding
a
module
for
for
the
dead
builder
and
the
reason
for
the
module
is
essentially
because
cloud
build
likes
to
see
a
workspace
where
everything
is
self-contained.
C
So
for
submitting
the
Deb
stuff
to
that
workspace,
we
essentially
need
to
make
sure
it's
self-contained,
so
it
can
build.
It
can
build
itself
without
having
to
look
to
the
top
level
modules
file.
The
top
level
go
mod
and
go
go
some
so
separate
module
for
that
and
again
it's
the
small
amount
of
dependencies.
So
it's
not
too
onerous
to
maintain
the
RPM
builder
very
similar.
A
C
So
this
will
rebuild
the
the
the
sums
for
the
for
the
the
go
repos
that
you're
importing
as
well
as
there's
some
other
stuff
that
it
does
so
we
also
add
the
hack
dot
tools
in
the
second,
so
I
made
some
very
cosmetic
changes
to
the
docker
files,
because
I
basically
wanted
to
trigger
that
run,
have
changed
so
really
they're.
Just
spacing
things
around
the
docker
file
in
the
go
mod
for
the
next
time.
I
ran
the
update.
C
You
can
see
that
the
release
build
Debs
gets
pulled
in
because
it
recognizes
it
as
a
module
now
and
then
in
tools.
We're
also
adding
hacks
hack,
slash
tools
that
go
we're
also
adding
a
blank
import
for
that
sub
module
and
the
reason
for
that
is
so
if
someone
go-go
gets
the
top,
if
someone
go
gets
the
repo,
they
will
still
have
the
dependencies
of
the
lower
sub
modules
within
that
right.
So
if
they
go
get,
is
they
go
get
the
top
level?
C
The
top
level
will
include
all
the
dependencies
for
all
of
the
sub
modules.
If
you
add
one
here,
this
kind
of
is
used
across
like
tests,
infra
and
I,
think
K
org
and
within
the
cluster
API
projects,
essentially
to
make
sure
that
before
someone
starts
working
on
the
project,
they
have
all
the
dependencies
that
they
need
and
that's
yeah.
That's
that's
kind
of
it
right.
So
when
these,
when
you
know
when
this
runs,
what
else
was
changed?
C
Yeah
I
just
moved
very,
very
little
of
the
logic
around
to
make
sure
that,
like
one
we're
actually
seeding
into
the
directory-
and
we
wanted
to
see-
if
we're
you
know
we're
in
the
working
directory
if
it's
the
right
working
directory
like
let's
just
print
that
out,
if
not
you
know,
if
not,
let's
bail
and
before
we
start,
we
do
the
validate,
we
do
the
validate.
So
we
fail
early.
That's
that's
it!
It's
a
pretty
simple
go
script!
A
I
would
hope
that
that
was
useful
for
people.
If
you
look
at
the
the
minutes
for
somebody
to
and
to
see
this
PR
come
in
and
implicitly
understand
all
of
those
other
things
that
chain
together
to
make
that
work
I'm
guessing
most
of
our
attendees,
don't
understand
that
big
picture,
so
I
would
imagine
that
was
useful
orientation
to
people
day.
C
So
yes,
we're
waiting
for
that
to
emerge
once
that
emerges,
I'm
going
to
make
another
cosmetic
change
to
one
of
these
files
to
let
that
build
trigger
and
part
of
the
reason.
This
is
happening
right
now,
at
least
as
I
know,
Tim's
is
working
on
some
improvements
to
getting
the
cap
G
tests
working
the
cap
chant
and
tests
working,
so
he
also
wants
to
verify
that
he
can
build
kind
of.
You
wants
to
start
building
the
flow
of
run
the
test.
The
test
is
successful.
C
Okay,
now
build
the
debs
RPM's
based
on
the
artifacts
within
that
commit
right
and
then
do
stuff,
so
he's
trying
to
verify
that
the
build
and
the
RPM
jobs
work,
and-
and
they
do
they
do
hopefully
right,
but,
but
he
is
also
asking
for
he
was
also
asking
for
like
a
rebuild
of
the
containers
that
would
run
the
jobs
and
I
was
like
this
should
be
automatic.
We
like
I
was
like
let
me
just
wire
this
up.
I.
Think
it's
simple
enough.
C
Now
that
Katherine
has
added
some
of
those
improvements,
so
this
was
not
possible,
maybe
a
month
month
and
a
half
ago
for
folks
on
the
call.
This
is
like
relatively
new,
so
we're
gonna
I,
don't
know
if
aces
on
the
call,
but
you
know
we're
gonna
chat
later
about
getting
back
into
the
the
image
stuff.
Now
that
we've
kind
of
laid
the
Frant,
the
the.
B
A
Okay,
all
right
we've
got
so
we
got
a
dozen
people
on
the
call.
You
just
spend
a
half
hour
describing
something
that
probably
took
you
many
many
hours
to
come
to
understand.
Originally,
as
the
dozen
people
on
the
call
spin
that
same
many
many
hours,
we've
just
invested
a
hundred
hours
coming
to
understand
what
you
already
know.
So,
sharing
that
knowledge
saves
everybody
time
it
allows
them
to
get
to
contributing
code
sooner
yeah.
C
So
in
in
you
know
talking
about
that,
you
know
we
have.
We
have
introduced
a
new
set
of
release,
manager,
associates
and
I
know.
A
few
of
you
are
asking
me
like.
How
do
we
do
this?
What
is
what
is
this
role?
Even
right?
The
role
is
loosely
defined
and-
and
it's
you
know,
the
reason
for
that
is
essentially
like
we
want
to
bring
in
people
that
we
can
trust.
It's
I
realized
this
cycle
that
it's
even
easier
to
bringing
people
that
we
already
trust.
C
So
the
selections
for
the
release
manager
associates
were
only
made
from
previous
release.
The
previous
release
team.
So
this
time
around
I
want
you
to
ask
questions
like
Tim
asks,
like
I
saw
a
PR
I,
don't
even
know
how
it
works
like.
Can
you
walk
me
through
that
right?
The
the
role
essentially
exists
to
pay
down
release
engineering,
technical
debt
right,
so
every
time
you
see
a
thing
that
might
be
wrong,
there's
a
good
chance
that
it
is
wrong
or
like
if
it's
undocumented,
like
there's
a
good
chance.
Someone
doesn't
understand
it
right.
C
You
know
for
us
to
rewrite
rewrite
the
release
tools
that
we
have
right
now
in
go
right.
If
that's
something
that
you're
interested
in
right,
maybe
take
a
moment
and
trying
to
find
my
PR
right,
maybe
take
a
moment
and
look
at
this
PR
right
and
you
know,
and
I
can
I
can
walk
through
it.
It's
fairly,
you
know
this
one's
fairly
simple
again:
I
am
NOT,
a
I
am
NOT,
some
mega
go
coder
or
anything
like
that.
C
It's
it's
just
usually
me
spending
a
few
hours
banging
my
head
on
a
wall
and
trying
to
like
hack
on
something
right.
So
you
know
this
one
is
you
know
this
one,
essentially
we're
adding
a
branch
fast
forward
tool
right
and-
and
it's
doing
exactly
the
same
thing-
that
the
the
bash
based
branch
fast
forward
tool
is
doing.
But
you
know
this
is
I
essentially
wrote
this
up
to
to
demonstrate
that
like
yes,
we
can
do
this,
so
let's
start
doing
this.
C
So
if
people
are
interested
in
doing
this
kind
of
stuff
like
what
I
would
love
to
land
first,
like
alongside
this
tool,
is
a
package
detail
right,
so
we
have
a
set
of.
We
have
a
set
of
libraries
quote
unquote
in
bash
that
we
use
for
a
lot
of
the
things
that
we
do
in
the
and
the
kubernetes
release
repo
right
so
and
they
do
things
like
make
sure
that
logs
exist
when
to
rotate
logs,
how
to
get
the
token
or
get
the
the
authorization
URL
for
pulling
things
from
github
or
pushing
things
to
github.
C
All
right,
so
if
you
go
to
Lib
and
we
can
take
common,
for
example,
there's
release
Lib
and
then
there's
also
get
lip
right.
So
common
does
some
stuff
like
right.
It's
you
know
it
set
some
pretty
printing
stuff
for
your
terminal
and
log
echo
right
so
log
echo
will
write
to
logs
and
also
standard
out.
C
We
can
specify
common
exit
and
common
eggs
at
zero
or
one
right
and
no
kind
of
rap
our
exit
codes
within
within
some
of
these
scripts
right,
the
one
that
I
thought
for
ever
was
like
ask
yarn
I
was
like
who
is
yourn
what's
going
on?
Finally
realized,
like
that's
ask
yes
or
no
right,
so
this
little
problem
to
you
during
some
script,
to
answer
yes
or
no
right
so
branch
fast
forward
is
an
example
of
that
it
will.
You
know
once
you've
verified
that
the
fast-forward
looks
good.
C
It
will
ask
you
like
hey:
did
you
go
verify
it?
Okay,
are
you
sure
you
want
to
push
it
now
cool
right,
so
we
have
that
in
a
few
scripts
as
well
step
header,
step
powder,
if
you
specify
step
header
it
will
capture
like
the
function,
call
or
something
and
say
let
go.
This
is
step
one
of
blah
right
and
then,
but
you
get
the
idea
right.
So
these
are
common
functions
that
we
do
across
across
a
lot
of
the
scripts.
There's
release
Lib
and
again
common
functions.
C
It
will
you
know,
it'll
get
a
job
cache
from
something
that's
in
in
prowl
or
saved
to
GCSE
storage
as
a
result
of
a
prowl
build
run
so
pulling
the
logs,
and
you
know,
and
and
doing
some
fancy
Wow
lots
of.
Ok,
ok,
alright,
to
find
what
the
it's
a
fine
with
the
the
job
cache.
Is
that
what
the
build
number
is.
So
we
can
determine
what
the
next
build
number
is
going
to
be
setting
the
build
version.
C
So
these
build
versions
are
eventually
used
in
how
we
do
the
staging
and
releasing
of
kubernetes
versions
right
and
again,
I
won't
I
won't
try
to
like
I,
don't
understand
all
of
it
as
well
right.
So
that's
exciting
so
like
this
is
an
opportunity
like
if
you're,
like
wow
I've,
never
seen
this
like.
Oh,
go
check
it
out.
A
lot
of
it
is
easy
to
read.
Some
of
it
is
doing
black
magic.
So
take
some
time
with
this
stuff,
we've
we've
run
into
you
like.
C
Well,
we've
broken
the
bill
because
someone
missed
the
comma
or
so
so
take
time
and
take
care
with
making
changes
to
these
files
and
reading
through
them.
Checking
the
release
bucket,
see
if
see
if
it
exists,
but
again
you
get
the
idea.
Those
are
those
are
functions
used
for
releasing
these
are
functions,
to
figure
out
how
how
we
push
things
to
get
or
how
we
get
things
from
get
right.
C
So
it's
like
here's,
the
repo
here
you
know,
here's
the
root
of
you,
know:
here's
the
the
kubernetes
repo
like
and
so
on
and
so
forth
right.
So
the
reason
you
know
I'd
love
to
get
to
a
place
where
kubernetes
kubernetes
depends
on
this
repo
and
it
depends
on
the
repo
primarily
because
of
the
push
build
script
right
and
so
Ben
had
opened
a
an
issue
to
remove
the
like
to
move
the
push,
build
script
over
to
kubernetes
kubernetes
right
because
then
they
kind
of
like
they
can
build
without
us
right.
C
So,
if
we
break,
if
we
break
kubernetes
release
at
any
one
time,
it
doesn't
affect
the
CI
process.
It
may
affect
the
staging
and
release
process,
but
it
won't
affect
day-to-day
operations
for
like
the
kubernetes
project
if
anyone's
working
on
like
main
kubernetes
right.
So
the
problem
we
see
here
is
that
push
build
sources,
the
common
library
at
the
gate,
Lib
library
and
the
release,
Lib
library,
which
means
it's.
Essentially,
it's
essentially
tied
to
where
it.
Wherever
these
libraries
live
right,
we
could
move
it.
C
We
could
do
packy
stuff
to
move
it
around,
but
it
makes
sense
to
leave
it
where
it
is
for
the
moment,
at
least
so,
to
move
it.
We
would
need
to
make
these
libraries
portable
right
and
then
that
kind
of
leads
back
into
starting
to
write
common
utilities
and
starting
to
write
rewrite
some
of
these
things
and
go
instead
right.
C
So
yeah,
that's
some
of
my
thought
process
around
it
again.
If
people
are
interested
in
doing
this,
we
should
probably
merge
this
soon
as
Ultem
I'm
gonna
look
back
over
it
and
and
it's
not
in
the
critical
path
for
anything,
so
it
probably
makes
sense
to
just
get
it
into
it
can
be
used
as
an
example
yeah
few
more
things
to
do.
But
you
know
it's,
you
know
it's
also
asking
yes
or
no,
or
you
know
doing
a
prompts
for
you
know,
and
then
there's
like
there's
some
get
stuff.
C
It's
you
know,
pin
to
my
repo
right
now,
but
that's
gonna
be
fixed
right.
Does
a
branch
regex
and
yeah
says
so
a
lot
of
the
things
that
it's
doing
this
in
in
that
library
that
you
saw
checking
if
the
branch
exists
or
sinking
the
repo
or
just
getting
the
rev
first
right
or
a
short
ref
first
merge
base
stuff
like
that
right.
C
So
those
were
things
that
were
all
required:
I,
essentially,
don't
want
to
try
to
rewrite
all
of
the
library
all
of
those
libraries,
but
I
want
the
way
we
write
the
tools
to
determine
what
we
need
in
the
libraries
right.
So
that's
why
we
took
like
a
small
tool
branch
fast
forward
to
start
carving
out
what
is
required
there
right.
So
any
questions
on
that
I
know
I'm
kind
of
like
jumping
around
all
over
the
place
like,
but
this
is
what
the
inside
of
my
head
looks
like
kubernetes,.