►
From YouTube: BuildKit CLI for kubectl: A New Way to Build Container Images - Daniel Hiltgen & Patrick Devine
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
BuildKit CLI for kubectl: A New Way to Build Container Images - Daniel Hiltgen & Patrick Devine, VMware
The Dockerfile is a tremendously popular format used to build container images. Various projects exist that focus on CI for Dockerfiles inside a kubernetes cluster, but what about developers and their inner loop? In this talk we’ll introduce a new CLI plugin optimized for developers which implements a familiar UX for building container images. It utilizes a powerful open source component called BuildKit to build Dockerfiles into images directly inside your Kubernetes cluster. When you build images, they are immediately available on the cluster for testing without requiring pushing to an external registry, providing an extremely efficient inner loop for development. The builder supports the two most popular container runtimes–containerd and dockerd–and will auto-detect the correct runtime in most cases. kubectl build -t myimage:latest -f Dockerfile .
B
A
A
So
let
me
explain
a
little
bit
about
what
we've
got
here.
This
is
a
single
node
cluster,
a
single
dome,
coupe
cluster,
which
is
set
up
on
my
mac
laptop,
it's
running,
mini
cube,
and
but
this
should
work
just
as
well
on
any
other
flavor
of
kubernetes
as
well
too.
I've
already
gone
ahead
and
installed
the
control
build
client
which
I'm
just
going
to
run
here,
and
so
what
this
is
doing,
because
I
haven't
run
control
build.
A
Yet
it's
setting
up
a
builder
which
is
you
know
if
you
run
the
subsequent
times
that
you're
not
going
to
need
to
go
through
that
particular
step
first,
but
now
what
it's
doing
is
taking
the
docker
file,
which
I
have
sitting
inside
of
this
directory,
and
it's
going
and
building
the
various
different
stages.
In
fact,
there's
a
cache
miss,
which
is
why
it's
trying
to
pull
part
of
the
alpine
it's
going
in
getting
some
of
the
dependencies
which
are
specified
inside
of
the
docker
file,
and
now
it's
actually
compiling
this
particular
doodle.
B
B
Once
the
builder
pod
is
running,
it
uses
the
equivalent
of
a
kubecontrol
exact
to
get
a
pipe
into
the
pod.
Then
it
uses
the
build
kit
grpc
api,
to
talk
to
the
builder
over
that
pipe
by
using
exec
to
talk
to
the
builder
we're
able
to
rely
on
kubernetes
native
rbac
for
access
control.
Now
by
default.
These
builder
pods
are
privileged.
B
B
B
B
So
if
you
have
multiple
nodes
in
your
cluster,
you
can
scale
up
the
deployment
for
the
builder
to
get
pods
running
on
all
the
nodes.
When
you
build
an
image
it
gets
built
on
one
node
and
then
at
the
end
of
the
build
we
use
the
cli.
The
cli
helps
transfer
that
image
across
to
all
the
other
codes,
all
the
other
pods.
B
B
This
does
mean
that
you'll
need
a
fast
network
between
the
cli
and
your
cluster
unless
your
images
are
really
small.
Now,
if
you
push
to
a
registry,
we
skip
this
replication
step
in
the
future.
We're
planning
to
implement
a
pod
to
pod
transfer
model
so
that
you
can
build
larger
images
on
a
distant
cluster
without
a
performance
here.
A
Okay,
so
why
did
we
build
this
thing?
We
know
that
there's
a
lot
of
different
ways
to
build
container
images
and
kubernetes
that
are
out
there,
and
some
of
those
tools
are
actually
really
great.
There
are,
however,
still
a
lot
of
people
who
are
using
dr
build.
It's
just
really
easy.
It
just
works
so
a
lot
of
those
other
tools,
though
they
require
you
to
use
a
registry,
and
many
of
them
require
a
lot
of
setup
and
the
images
aren't
available
locally.
A
The
other
thing
that
we
see,
which
is
happening
around
kubernetes,
is
that
people
are
moving
away
from
the
docker
from
docker
d
as
being
the
default
container
runtime,
but
people
have
built
up
scripts
and
automation
and
muscle
memory
around
building
images.
So
we
wanted
to
create
something
which
was
close
to
the
the
same
experience
that
they
already
have.
B
B
It
only
transfers
files
from
your
local
directory
if
they're
actually
used
during
the
build,
and
it's
smart
about
tracking
your
files
have
changed
for
better
incremental
builds
to
make
incremental
builds
faster.
It
can
cast
those
build
results
locally
on
the
builder
or
even
within
a
registry,
so
that
enables
multi-node
build
farms
to
build
faster.
A
B
B
B
So
it
is
a
simple
ubuntu
vm
running
container
d
as
the
runtime
okay.
If
I
do
a
get
pods,
you
can
see.
I've
got
nothing
running,
so
I
haven't
booted
up
my
my
builder.
Yet
so
let's
go
ahead
and
look
at
the
demo
setup
itself.
So
we'll
start
with
a
really
simple
docker
file,
so
the
docker
file
uses
busybox
as
the
base
layer
and
all
it
does-
is
copy
in
the
command
and
set
that
as
the
entry
point.
B
B
B
A
few
other
settings
of
note
in
here
I've
got
my
image:
pull
image,
pull
policy
set
to
never
that's,
to
make
sure
that
I'm
always
using
the
image
that
I've
built
locally.
We're
not
trying
to
pull
this
from
a
registry
again
we're
trying
to
do
a
fast
inner
loop,
where
it's
local
development.
On
my
local
cube
environment.
B
I've
got
a
restart
policy
set
to
always
so
it's
always
going
to
restart
the
pod
and
a
termination
grace
period
set
to
zero.
So
that's
going
to
make
it
faster
to
terminate
and
shut
down
the
old
pods.
So
all
of
this
should
help
to
kind
of
build
up
that
fast
inner
loop.
So
as
a
developer,
I
write
some
code
compile
and
then
test
now
in
a
production
environment.
Obviously
you'd
have
different
settings,
but
this
kind
of
helps
helps
optimize
that
inner
loop
all
right.
B
So
let's
go
ahead
and
actually
build
the
image
so
I'll
just
cut
and
paste
that
in
all
right.
So
what
it's
doing
right
now
is
actually
booting
up
the
builder,
and
you
can
see
that
it's
actually
attempting
to
use
docker,
but
the
docker
runtime
failed,
and
it's
going
to
retry
with
container
d
and
the
reason
we
do,
that
is
docker
d
is
still
the
most
popular
container
runtime
out
there
in
the
wild.
B
Even
though
a
lot
of
folks
are
switching
over
to
container
d,
as
many
folks
know,
docker
actually
has
container
d
under
the
covers.
So
if
we
started
with
containerity
by
default,
we
might
incorrectly
assume
that
your
cluster
is
using
containerity
and
not
realize
that
docker's
sitting
on
top
of
it
so
kind
of
for
simplicity.
For
now
we
we
default
to
docker
and
then
we'll
flip
back
to
container
d.
If,
if
docker
is
not
detected,
so
you
can
see
that
was
all
automatic
didn't
have
to
do
anything
you
can.
B
If
you're
manually,
creating
the
builder,
you
can
explicitly
specify
that
you
want
to
use
the
continuity
runtime
and
that
will
kind
of
short
circuit
that
auto
detection
logic.
So
you
can
see
it
took
about
17
seconds
to
attempt
docker
fail
and
then
restart
or
retry
with
continuity.
Now
that
at
this
point
the
builder
is
already
running.
So,
if
I
do
any
subsequent
builds
it'll
immediately
be
available,
and
I
can
just
jump
right
into
my
build.
B
B
Things
in
the
output
here,
so
we
can
see
it's
it's
pulling
the
base
layer
of
library,
busy
box
and
that
was
able
to
resolve
in
the
continuity
image
cache.
Then
it
copied
in
the
image-
and
that
was
pretty
much
it
for
this
dockerfile
right,
there's
no
compilation
or
anything
like
that,
like
we
saw
in
patrick's
demo.
B
Finally,
what
it's
doing
is
just
exporting
it
to
an
image
and
tagging
it
with
the
the
tag
that
I
specified,
and
so
at
this
point,
that
image
is
now
loaded
into
my
container
runtime
and
available
on
those
nodes.
Now,
if
I
had
my
second
node
running,
it
would
have
transferred
the
image
over
to
that
second
node
as
well,
and
so
now
both
nodes
would
be
able
to
run
on
this
image.
B
And
if
we
do
it
get
pods,
we
can
now
see
both
the
build
kit
builder,
which
is
running
and
ready
to
go
for
the
next
indication
or
the
next
time.
I
build
an
image
and
I've
got
my
app
running
as
well.
Now.
What
I'm
going
to
do
over
in
this
terminal
window
is
I'll,
actually
do
a
little
loop,
so
just
kind
of
an
infinite
while
loop
control
logs,
I
do
a
follow
on
the
logs,
I'm
going
to
use
a
label
selector
for
app
equals
my
app.
What
this
will.
B
Let
me
do
is
in
this
terminal.
It
will
continue
to
follow
the
log
output,
even
as
I
iterate
right
so
we'll
go
ahead
and
start
that
up.
So
you
can
see
it's
just
continually:
spitting
out
hello
world
with
a
date
date
stamp
every
three
seconds
or
so.
Okay.
So
now,
let's
kind
of
do
a
simulation
of
a
developer
inner
loop.
So
I'm
going
to
write
some
code,
make
some
changes
recompile
and
see
my
code
running
so
we'll
go
ahead
and
modify
this
program
to
keep
it
simple,
we'll
just
say
hello,
world
2.!
B
B
So
what
that's
going
to
do
is
if
the
build
failed,
it'll
stop.
But
if
the
build
succeeded,
it's
going
to
go
ahead
and
delete
the
pod
where
app
equals
my
app,
so
that
should
delete
this
pod
and
allow
the
deployment
and
the
kubernetes
scheduler
to
detect
the
pods
dead
and
automatically
restart
the
pod,
which
will
pick
up
the
new
image
that
I
just
built.
So
let's
go
ahead
and
see
that
work.
B
B
So
there
you
go
nice
fast.
Inner
loop
optimized
for
developers
immediately
have
your
images
available
on
your
local
system
works
on
single
nodes
works
on
multi-nodes
as
well
all
right.
So
let's
actually
look
at
how
you
would
go
about
creating
a
customized
builder
if
you
wanted
to
so.
First,
let's
go
ahead
and
delete
the
builder
that
we
already.
B
That's
actually
an
alias
to
build
kit
build,
and
so
we
have
a
number
of
different
commands
underneath
kube
control
build
kit,
so
we
say:
control
build
kit.
Help
we
can
see
build
is
one
of
the
sub
commands
of
build
kit,
which
again
is
just
an
alias
to
cube
control,
build
there's
other
commands
that
you
can
do
under
the
build
kit
top
level
command.
So
you
can
create
a
new
builder
list
existing
builders
because
you
can
create
multiple
builders
with
different
configurations.
B
B
So
there's
a
lot
of
command
flags
here,
we're
not
going
to
go
through
all
of
them,
but
just
a
kind
of
a
real
quick
summary.
So
this
allows
you
to
manually
or
explicitly
create
a
builder,
so
you
can
tune
and
and
optimize
the
configuration
of
it
based
on
what
you
want
in
your
environment.
So
this
would
be
how
you
would
set
it
up
to
run
rootless.
If
you
wanted
to
have
kind
of
a
de-privileged
environment,
you
can
pass
in
specific
flags
to
the
build
kit
demon
inside
of
the
builder.
B
If
you
want
to
tune
and
again
kind
of
modify
the
behavior
of
build
kit
and
pass
in
a
specific,
build
kit
config
file,
if
you
already
have
one
that
you've
set
up
or
if
you
want
to
customize
that
that
setup,
the
one
I'll
show
just
real
quick
as
an
example,
is
the
runtime
flag.
So
by
default
we
have
it
set
up
to
do
auto,
which
kind
of
does
that
auto
detection
logic
I
talked
about.
B
You
can
also
specify
the
runtimes
explicitly
so
for
this
one
I'll
show
you
kind
of
using
container
d
explicitly,
so
it
doesn't
have
to
do
the
automatic
detection.
So
let's
say:
control
build
kit,
create
runtime
container
d,
and
so
now
we'll
go
ahead
and
boot
up
the
the
builder
with
container
d
immediately.
We
don't
have
to
try
docker
or
fail
and
then
fall
back
to
containity.
So
you
can
see
it
was
much
faster
to
get
started
and
I'd
be
ready
to
run
and
start
doing,
builds
now.
A
B
A
A
So
if
you
pulled
something
like
postgres
or
the
redis
images,
it'll
just
work
on
whatever
architecture
you're
using
what's
cool
about
buildkit
is
that
it
lets
you
easily
build
these
types
of
images
and
there
are
three
different
ways
of
doing
it.
You
can
build
natively
on
the
target
architectures,
which
we
don't
support
quite
yet,
but
we
are
working
on
that.
A
You
can
do
this
in
languages
like
c
and
c
plus
plus,
but
it
usually
takes
a
little
bit
more
effort
to
get
your
tool
chain
to
work.
One
big
caveat
that
I
should
mention,
though,
is
with
multi-arch
images.
Is
that
if
you
for
now,
you
have
to
to
push
it
to
push
your
images
to
a
registry
when
you're
doing
the
build,
but
we
have
been
looking
at
ways
to
assemble
them
locally,
which
wouldn't
require
you
to
do
that.
Push.
A
A
So
unfortunately,
it's
a
lot
bigger
than
just
using
from
scratch,
but
then
I
copy
over
the
newly
built
executable
over
to
it
and
then,
of
course,
I
set
up
the
entry
points
for
both
of
those.
The
last
stage
is
actually
kind
of
like
the
magic
sauce
that
build
kit
uses
to
assemble
each
one
of
the
architectures.
B
All
right,
so
what
what
environments
can
you
run
this
on?
So
we've
tested
this
on
kubernetes
versions,
114
and
up
so
pretty
much.
Every
supported,
stable
version
of
kubernetes
today
works
as
far
as
run
times.
We
support
both
container
d
and
docker
d
run
times.
Additional
runtimes
can
be
added
in
the
future
as
far
as
distros
most
distros
should
just
work.
B
All
right,
so
this
is
a
really
young
project.
We
do
have
native
os
packaging
for
mac
os
windows
linux.
To
make
it
a
little
bit
easier
to
get
installed,
go,
take
a
look,
give
it
a
try,
we're
always
looking
for
help.
So
if
this
is
interesting
to
you
give
us
a
hand,
pick
up
an
open
issue
submit
a
pull
request.
Join
the
community
help
us
define
what
the
1.0
release
should
look
like.