►
Description
Join jay as we go through how upstream Kubernetes builds both container and OS images across the project. This includes recent work to address multi-arch scenarios.
00:03:00 - audio check
00:05:00 - Week in review
00:10:00 - Intro to multi OS containers, pause image, agnhost on kubernetes
00:21:00 - Dockerfile for windows
00:30:00 - peri's blog post on buildx
00:38:00 - speaking of manifest
00:35:00 - deep dive into OCI and umoci
00:45:00 - intro to OVAs on windows, VHDs, cluster api image-builder
01:00:00 - nssm, systemctl, windows, linux
01:25:00 - ricardo sais.... Freebsd on k8s ?
A
A
B
C
C
B
C
A
A
C
B
A
A
A
bunch
of
stuff
related
to
images
today
right
so
we've
got
james,
is
here
my
my
friend
from
microsoft
and
he's
a
windows,
windows
guy
and
actually
luther's
here
too,
over
from
rancher.
So
thanks
for
coming
to
hang
out-
and
so
let's
start
out
with
the
with
the
week
in
review
here
and
of
course,
I'm
jay,
you
all
know
who
I
am
so.
A
A
A
First
thing
that
we've
got
here
is
riso,
so
this
this
week
is
using
kubernetes
to
rethink
your
system
architecture
and
ease
technical
debt.
So
what
do
we
got
here
so
talking
about
ansible,
playbooks
and
managing
ec2
instances
like
pets,
so
this
sounds
like
a
before
after
story.
So
what
happened?
A
no
new
infrastructure.
A
A
A
B
So
this
is
the
thing
that
powers
k3s.
It
was
originally.
It
was
darren's
idea
to
get
rid
of
fcd
and
shim
in
sq,
lite
we've
added
more
to
it.
If
you
go
into
package,
there's
some
place
in
there,
there's
gonna
be
a
driver
section,
so
you
can
see
there's
postgres
and
a
bunch
of
other
stuff
now,
but
it
was.
It
was
kind
of
like
how
can
you
get
that
cd
always
kind
of?
Was
the
you
know
the
gorilla
in
the
room?
B
C
B
A
C
A
A
I
couldn't
do
it
so
if,
if
folks
want
a
cool
homework
assignment,
dig
through
kk
kubernetes
kubernetes
and
try
to
find
where
the
data
interface
for
ncd
is
it's
somewhere
in
here
you
know
tim
saint
claire
knows
where
that
is
but
he's
I
don't
know
if
he's
here
today,
but
he
he
was
the
he
was
the
guy
who
originally
wrote
it.
So
there's
an
interface
and
you
can
implement
the
interface
so
yeah
all
right,
let's
jump
out
of
here
and
go
to
I
wanted
to
talk
about.
A
A
A
That
should
be
better
okay,
so
if
folks
want
to,
let
me
know
if
they
got
the
right
thing
now.
Let
me
know:
okay,
cool
yeah
and
well.
Thank
you
for
jumping
in
here
so
yeah.
Thank
you.
Whoever
noticed
that
morteza
husseini
and
and
so
all
right,
we
got
the
notes.
So
do
you
all
remember
last
episode,
so
this
is
real
cool
thanks
for
that
luther.
A
lot
of
people
talking
about
that
whole
thing
like?
Can
we
just
not
use
that
cd?
A
So
we
over
on
the
ping
project
a
couple
of
episodes.
We
were
showing
you
the
next
generation
coupe
proxy
and
if
you
all
remember,
I
couldn't
get
it
working
when
we
found
out
why
the
reason?
Why
was
because
nf
tables
and
ip
tables
conflict-
and
so
there's
a
bunch
of
articles
and
stuff
about
this.
But
it
turns
out
that
there's
like
a
legacy,
iptables
sort
of
to
deal
with
this
whole
thing
and
it's
like
renamed
and
in
newer
versions
of
ubuntu.
This
is
fixed
and
so
on
and
so
forth.
A
So
but
nf
tables
and
ip
tables
kind
of
they
don't
play
nicely
together
right
because
nf
tables
is
the
new
thing
and
iptables
is
the
old
thing
and
so
there's
a
lot
of
stuff
in
nf
tables
that
we're
sort
of
resolving.
But
if
you
anybody
wants
to
try
kaping
out
and
go
into
the
the
local,
the
local
up,
coping
recipes
and
try
out
the
next
gen
coupe
proxy.
We
now
have
support
for
ipvs
and
ip
tables
to
test
it
out
and
mikhail,
also
added
node
ports
to
it.
D
Yeah,
so
it's
been
a
long
time
in
incoming,
but
we've
got
container
d
support
for
windows,
server
and
preview.
Now
this
opens
up
a
lot
of
new
functionality
for
windows,
containers
being
able
to
single
mount
files.
It
has
the
host
process
it'll
enable
host
process
containers
when
they
come,
which
just
landed
in
122
and
there's
a
bunch
of
other
awesome
features
for
container
d
for
windows.
So
it's
a
big
one.
A
Yeah,
that's
exciting
for
me
too,
because
we
have
container
d
on
the
way
over
with
with
over
a
vmware
tonzo.
We've
got
it
on
the
way
very,
very,
very
soon
as
well
for
windows
based
on
cluster
api.
Now
we've
got
increment
dot
container.
So
I
don't
know
what
this
is.
This
issue
shares
practical
consideration,
functional
processing,
containerization.
A
So
this
is
another
really
good
sort
of
cio-
I
guess
level
sort
of
or
may
yeah
like
why
why
you
want
to
move
to
kubernetes
kind
of
at
a
large
scale.
So
this
is:
is
this
a
book
or
is
this
a?
I
mean
this
is
one
of
the
most
nicely
created
sort
of
introductions
to
this
whole
ecosystem
that
I've
ever
seen
this
increment
dot
about
so
increment.about.
A
It's
definitely
worth
checking
out.
I
guess
how
teams
and
build
teams
build
and
operate
software
systems
at
scale,
so
it's
kind
of
just
a
good
okay
cool.
So
we
got
that
and
now
we
get
into
it.
Okay,
so
we've
got
a
lot
of
image
based
topics
today,
so
if
anybody
has
any
questions
about
containers
or
images
or
anything
else
feel
free
to
just
dump
them
in
the
dump
them
in
the
chat,
I
don't
know
if
I
can
answer
them.
It's
not
like.
A
A
This
is
where
the
this
is
where
the
gooblet
stuff
starts-
and
this
is
kkk
so
agn
host.
So
do.
Does
anybody
in
the
audience
know
what
agn
host
is?
Does
anybody
wanna,
wanna
wanna
contribute
their
their
knowledge
of
upstream
kkk
to
this
podcast
or
or
live
stream?
Before
I
tell
you
what
it
is,
so,
let's
go
to
kubernetes,
kubernetes,
okay
and
let's
go
to
it
on
get
well
I'll.
Just
you
know
I
can
go
straight
to
it:
agn
host
yeah!
I
can
go
straight
to
it.
A
So
hi
sevy
nice
to
nice
to
see
you
here,
happy
tgik
to
you
too
so
agn
host
is
it's
it's
it's
a
container
that
we
use
in
upstream
k8s
to
to
test
things,
and
I
you
can
you
can
docker
run
it.
I
think
I
have.
I
think
I
have
it
somewhere
around
here.
I
think
I
have
it
in
one
of
these
terminals
floating
around
so
yeah.
If
you
look
in
here
actually
so,
if
I
do
so,
agn
host
is
is
yeah
we
can
just.
A
You
know:
we've
got
thousands
of
tests,
we've
got.
You
know
five,
six.
What
four
five
thousand
tests-
and
you
know
the
thing
is
that
you've
gotta
like
you've,
got
to
be
able
to
run
these
tests
not
only
on
linux
but
on
windows
too
right.
So
each
one
of
these,
if
you
go
into
test
e2e
right
you'll,
see,
we've
got
like
windows
tests.
We've
got
all
these
tests
in
here,
we've
got.
You
know:
we've
got
a
bunch
of
tests
for
sig
network.
A
Obviously,
most
of
these
or
a
lot
of
these
are
most
commonly
exercised
on
linux
and
so
on
and
so
forth,
but
we
need
to
be
able
to
run
on
multiple
operating
systems.
So
the
solution
to
this
is
the
agn
host
and
that
the
agn
host
does
is.
If
I
go
back,
where
is
it?
Where
do
we
go?
Oh
where'd,
I
go
here.
It
is
yeah.
It
has
all
these
different
programs
in
it
right
and
each
one
of
these
right.
A
A
A
I
thought
that
agn
host
maybe
had
a
help
command,
but
maybe
it
doesn't
oh
yeah.
It
does
sorry.
I
just
wasn't
looking
properly.
So
here's
all
the
commands
that
it's
got
all
the
little
sort
of
cross-platform
things
it
can
do
right.
So
I
can
run
guestbook.
I
can
run
light.
I
can
run
so
like
we
use,
for
example,
as
an
example.
Here
we
use.
We
use
this
quite
a
bit
in
sig
network
right
for
for
various
tests
right.
A
It
starts
a
little
tiny
web
server
and
it
allows
you
to
ping
each
other
for
to
see
whether
one
thing
can
talk
to
another.
We
use
net
exec,
pretty
exp
pretty
extensively
because
we
we
use
these
in.
We
use
netexec
in
the
in
the
network
policy
tests.
As
I
recall,
let
me
confirm
that
actually,
while
we're
here
because.
A
Yeah,
so
here's
here's
where
we
use
here's
where
we
call
out
agn
host
in
in
in
these
tests-
and
we
could
see
if
I
go
in
here-
agn
host-
we
call
it
and
actually
no,
we
actually
use
serve
hostname.
So
so
we
use
surf,
serve
hostname
and
then
we
use
we
use-
I
guess
porter
over
here,
so
we
we
call
these
particular
programs
here
and
we
spin
up
like
a
tcp
endpoint,
a
udp
endpoint,
depending
on
what
kind
of
policies
we're
trying
to
test
and
so
on
and
so
forth.
A
So
these
are
very
extensively
used
right
throughout
the
code
base
and
anyway
so
yeah.
This
is
the
way
this
is
where
they're
built
and
if
we
go
back
here,
we'll
see
that
there's
a
docker
file
and
there's
a
docker
file,
windows
and
actually
james
knows
the
dockerfile
windows
parts
better
better
than
I
do,
but
james
we
were
just
talking
about
this
actually
a
little
while
ago.
A
Where
was
I
have
it
in
the
urls
here
we
were
just
talking
about
the
pause
image.
Actually,
that's
that's
actually
the
the
pause
images.
So
so
these
are
all
the
test
images
and
then
I
I
think
the
next
thing
we
wanted
to
talk
about
was
the
pause
image.
So
everybody
knows
about
the
pause
image:
it's
anytime,
you
run
any
container.
A
You
you
fire
off
one
of
these
right,
it
sort
of
sits
around
until
your
linux
network
namespace
comes
up
and
if
you
look
in
the
pause
image,
the
way
the
pause
image
is
built,
there's
a
docker
file
and
then
a
docker
file
for
windows
right,
and
so
these
have
separate.
This
is
these
have
separate,
separate,
build
patterns
in
them,
and
so,
if
you
look
in
the
docker
file
for
windows
right
this,
actually
this
doesn't
require.
This
doesn't
require
windows
server.
A
E
A
D
So
one
of
the
things
that
we
initially
did
was
you
had
to
have
a
windows
server
to
build
all
of
the
test
images
and
that
was
challenging
because
the
windows
server
had
one
per
version,
and
so
we
had
linux
vms
and
they
would
connect
out
to
the
windows
docker
host
and
build
the
images.
But
it
was
just
expensive
and
challenging
to
maintain
and
so
claudio
claudio
who's
in
sync
windows,
kind
of
started
to
figure
out
that
you
could
actually
use
buildex
to
build
windows,
container
images.
D
But
there's
just
a
few
caveats
so,
like
you
can't
do,
run,
commands
and
there's
a
couple
other
small
things,
but
the
run
commands
is
the
big
one,
and
so
we
actually
build
all
of
the
windows
container
test
images
on
linux
using
buildback.
So
you
can
you?
Can
boot
up
your
linux
machine
run,
build
and
hit
build
all
the
images
and
push
them
all
up
to
gcr.
A
A
Like
how
do
you
I
don't
know
how
to
how
to
start
so
it's
it's
kind
of
like
I
guess
build
x
was
built.
Was
it
built
specifically
for
multi-arch?
Was
that
I
guess
that's
why
it
was
built
right,
because
you
have
windows,
you
have
linux,
but
you
have
different.
You
have
x86,
you've
got
arm,
you've
got
all
these
different
and
each
one
of
these
might
require.
You
know
a
different
type
of
like
architecture.
A
So
I
guess
that's
kind
of
the
reason
like
build
x
was
sort
of
sort
of
created
from
from
like
the
docker
team
right
and
then
it
allows
you
to
build
any
any
architecture.
So
if
you
go-
and
you
look
so
people,
for
example
that
ship
the
pause
image
you
know-
and
you
know
who
has
a
great
blog
post
on
this-
is
perry.
Let's
we
can
go,
pull
his
up
the.
C
A
D
A
Yeah,
so
if
you,
if
you
look
at
a
container
right,
if
you
look
into
a
container,
it's
got,
these
manifests
right.
If
you
look
into
an
oci
container,
so
oci
is
the
container
interface
right
that
that
everything
runs
off
of
since
whenever
it
was
created
as
the
alternative
to
just
running
docker
images
right,
it's
a
like
a
a
an
unbranded
container,
runtime
interface
right.
So
any
anybody
can
write
their
own
container
runtime
and
you
know
to
conform
to
that.
A
You
have
to
you,
know,
make
these
these
manifest,
and
then
these
manifests
inside
of
them
have
have
information
about
the
architecture
right,
so
you've
got
you've
got
the
architecture
right
and
you
have
like
a
you
know.
You
have
an
image
and
the
image
has
you
know
a
one-to-many
relationship
with
this
with
with
architecture
right.
This
is
your
lci
image,
and
so
you
have
so
you
know
all
these
different,
architectures
and
sort
of
os.
A
Has
yep
see
I'm
learning
how
to
use
these
little
bubbles,
where
I
can
like
make
everybody
pop
up
so
yeah,
one
of
build
kits
features
is
multi-arch
right,
so
so
enter
so
so
perry's
little
article
here
just
was
just
about
to
get
into
that
right.
So
the
way
that
you,
the
way
that
you
ship
distribute
kube,
is
you've,
got
to
give
people
a
pause
image.
You
just
have
to
give
people
pause.
You
can't
do
anything
without
it.
It's
it's
like
it's
so
fundamental.
A
A
How
you
do
it
right
so
so
so
you
you
basically
wind
up
creating
this
sort
of
loop
and
what
you
do
is
when
you
in
when
you
build
your
image,
you
actually
use
buildex
to
build
it
for
all
these
different
architectures
right
right,
which
is
done
right
here
and
then
you
kind
of
append
the
manifest
list
right.
So
you
add,
add
these
entries
up
here
right
and
then
each
one
of
these
entries
winds
up
getting
pulled
down
by
your
container
runtime.
So
as
an
example
to
see
how
that
works.
C
A
If
I
try
to
run
an
is
image
all
right,
let's
say
I
tried
to
run
this
right.
So
if
I
tried
to
docker
run
dash
t
I
this
image,
you
see
how
it
blows
up
instantly
right.
It
says
there's
no
manifest
for
that
linux
amd64
image
right.
So
because
it's
it's
a
it's
a
microsoft,
iis
image,
that's
meant
to
be
run
for
like
windows,
containers
right,
whereas
you
know
actually
I
could
jump
into
that
iis
image.
A
A
I
can
group
ctl
get
pods
tube
ctl
edit
pod,
and
I
can
see
this
image
here
and
I
got
the
same
image
here
and
it
runs
just
fine
on,
but
that's
because
I
have
nodes
show
wide.
I
have
a
windows
cluster.
I
have
a
windows
node
here:
windows,
server,
2019
right
so,
like
you
know,
same
same
same
container,
totally
different
life
experience
depending
on
what
what
planet
you
put
it
on
and
so
yeah
next
holy
s.
What
did
he
say
holy
yeah?
I
think.
Yes,
you
can
run
that's
yeah.
A
D
Yeah,
I
think,
what's
cool
about
that,
the
the
multi-arch
manifest
that
you're,
showing
off
with
the
different
versions,
is
that
ang
host
container
that
same
url.
You
can
go
in
to
your
windows
machine
and
you
can
run
the
same
exact
image
name
there
and
it
will
pull
down
the
correct
one
based
based
off
of
those
os
platform,
values
which
is
really
cool,
because
now
you
just
build
you
just
can
distribute
that
single
url
and
anybody
can
pull
it
on
all
the
different
os's.
As
long
as
it's
in
that
manifest
list.
A
A
D
A
A
I
yeah
so
I
don't
know
how
long,
but
I
so
I
mean-
and
the
only
reason
I
asked
that
is
I'm
just
wondering.
Is
this
always
been
the
case
that
we've
had
this
but
anyways?
It
is
that's
a
really
important
thing
like
what
james
mentioned,
because,
like
it's,
it's
subtle
and
you
may
have
missed
it,
but
in
here
when
we
build
this
right,
we're
doing
this
build
x
here
and
we're
appending
to
this
manifest
list.
A
D
A
A
A
Oh
there's
not
a
manifest,
so
there's
a
bundle.
There's
a
config,
let's
see
before
I
pose
the
prior
one
I
had
said
build
kit
build
x,
build
kit
is
much
more
than
multi-arch
yeah
eric.
Do
you
want
to
elaborate
on
what
what
what
other
parts
you're
interested?
I
mean
it's
a
lot.
I
know
it's
a
lot
faster
and
it
caches
things
and
stuff
like
that.
But
was
there
the
other
stuff
that
you're
thinking
of
glossary?
Let's
go
in
here
manifest.
A
Have
a
manifest
in
the
glossary,
so
if
I
was
to
clone
this
down,
I
don't
know
I
I
well
I'm
interested
in
this,
but
I
I
in
the
interest
of
time
I
think
we
should
jump,
but
we
should
come
back
to
this
later.
So
all
right,
let
me
get
let
me
scribble
over
to
my
other
monitor,
so
I
don't
have
to
look
at
it
right
now
and
I
can
jump
over
to
like
so
we've
gone
through
the
pause.
We've
gone
through
the
makefile.
A
A
A
A
I
I
get
so
I
get
so
anxious
when
perry
isn't
around.
I
just
don't
know
what
to
do
with
myself.
Where
is
it?
Where
is
his
blog
post
here?
It
is
yeah,
so
they're
saying
that
yeah
each
one
of
these
okay
and
so
then
this
top
level
thing
is
a
docker
well
perry's,
calling
it
a
docker
manifest,
but
I
don't
know
what
we
would
call
that
top
level
thing.
I
guess.
A
A
D
C
A
A
A
If
folks
really
want
to
see
it
just
shout
in
the
youtube
and
we'll
jump
in
there
and
we'll
look
at
the
toml
files
for
container
d
on
a
windows
or
a
another
host
and
looks
like
we
lost
luther.
I
don't
know
where
luther
went,
but
thank
you
suresh
for
bringing
that
up.
Yeah
eric.
Definitely
okay
yep
build
next
gen
yeah
build
kit.
Could
maybe
that's
the
next
episode
eric
so
yeah
eric.
A
A
What
are
we
gonna
do
now?
Let
me
go
back
to
my
speaking
of
manifests.
Let
me
go
back
to
my
manifest
for
this.
For
this
lovely
day.
Where
are
we
okay?
Here
we
go
there's
a
lot
of
stuff
here,
james,
I
don't
know
if
we
can
get
through
it
all
here
we
go
building
windows,
ova's
on
osx,
so
yeah,
so
so
so
I'll
just
get
a
quickly
talking
through.
D
A
A
Yeah
we
could
yeah,
we
could
yeah.
We
could
squeeze
that
in
so
well,
let's,
let's
the
audience,
do
you
all
want
to
go
deeper
into
oci,
or
do
you
want
to
see
some
some
image
builder,
building
virtual
machine
stuff
first
and
come
back
to
oci
later?
Does
anybody
have
an
opinion?
If
not
we'll
go
one
step
deeper
on
some
some
container
stuff
I'll
give
you
five
seconds?
Five.
A
Four,
three
two!
Nobody
cares!
Okay,
all
right!
Here's!
What
we're
gonna
do
we're
gonna.
Do.
I
think
we're
gonna
do
james's
idea.
Let's
try
this
out
so
yeah.
So
once
you
get
into
the
business
of
yeah,
okay,
we'll
do
it
we're
gonna,
go
deep
on
the
okay
go
deep,
says:
suresh,
let's
go
deep
in
oci,
so
so
you're
right
james!
So.
A
The
the
thing
that
happens
when
you
start
playing
with
images
the
first
in
the
is
is,
is
you
you
you
wind
up
needing
to
you
end
up
needing
and
or
wanting
to
like
investigate
them,
pull
them
down?
You
might
want
to
do
this
for
your
for
security
purposes,
you
might
use
a
tool
like
black
duck
or
something
that
goes
and
grabs
images
and
and
and
scans
those
images
for
for
vulnerabilities.
A
In
fact,
I
don't
know
if
this
project
still
exists,
but
there
was
a
project
called
ophelia,
ophelia
ophelia
black
duck
software,
which
which
did
this-
and
I
I
don't
know
where
I
thought
we
had
a
link
to
it.
I
don't
know-
maybe
it's
not
there.
Maybe
it's
not
there
anymore
ophelia
yeah,
it's
not
there
anymore.
I'm
not
sure
this
is
a
long
time
ago.
A
It
allows
you
to
pull
an
image
with
no,
you
know
so,
for
example,
if
I
just
go
to
temp
and
do
this
again,
so
I
could
show
you
like
it
allows
you
to
just
pull
down
an
image
without
having
any
kind
of
docker
dependency
right,
so
make
your
test
one.
I
go
in
here
and
I
just
scope
you'll
pull
this,
so
I
could
pull
down
this
image
and
I
could.
A
I
can
pull
it
down
and
you
can
see
I've
got
open
suse
in
here
and
then,
if
I
go
and
open
cc,
I've
got
an
index
and
I
can
see
that
index
and
I
can
really
dig
into
exactly
what
my
container
is
made
of.
Let
me
make
this
full
screen
right:
hey
washing
your
hand
watching
your
hands
over
there,
okay
you're
better
than
me.
I
don't
do.
I
don't
do
that
enough.
Okay,
jq!
A
A
A
So
if
I
tree
this,
okay
you'll
see
that
I
have
an
index
and
then
that
that
index
has
this
manifest
it
has
a
digest
and
then
it
has
a
an
oci
layout,
let's
see
what's
in
there
layout,
okay
and
that
doesn't
have
anything
other
than
the
layout
version.
So
there's
multiple
different.
I
guess
ways
that
you
can
organize
this
data
and
then
these
things
are
like
tarballs
right.
I'm
pretty.
C
A
Okay,
ascii
text
with
very
long
lines-
I
didn't
know
I
didn't
know
that
was
a
type
okay.
So
if
I
go
in
here,
oh
yeah,
this
is
also
json,
so
I
can
go
in
here
and
I
can
cat
it
type
it
to
jq
and
see
what
it's
all
about.
Okay.
So
if
I
go
into
this
individual
manifests
right,
then
you'll
see
here
each
one
of
these
has
like
a
bunch
of
metadata
and
then
ultimately,
the
layers
are
all
in
here
and
thanks
to
thanks
by
the
way
to
liz
rice.
A
C
C
A
A
But
the
reason
all
this
stuff
is
important
is
because
in
kubernetes
nowadays
you
don't
have
docker
and
you
don't
have
all
those
nice
command
line
like
expectations
from
your
from
your
command
line
from
your
driver
anymore,
because
you're
using
container
d,
which
is
much
lighter
weight,
so
tools
like
scopio
and
umochi
and
stuff.
What
you
do
is
you
kind
of
use
those
tools
to
investigate
and
inspect
your
containers
and
those
tools
are
much
lighter
weight
and
they
can
be
composed
much
more
easily
like
into
ci
jobs
and
things
like
that
right.
A
So
from
an
operations
standpoint,
it's
really
nice
that
these
exist.
One
note
is
that
it's
not
trivial
to
install
scopio.
I
found
that
when
I
was
installing
it,
I
had
to
apt-get
install
it
because
it
had
a
lot
of
dependencies
on
like
file
system,
layer,
libraries,
stuff,
and
so
I
did
like.
I
think
it
might
be
in
my
history
somewhere,
but
I
did
something
like
an
apt
app
get
yeah.
A
I
did
this
and
before
doing
that,
I
had
to
trying
to
run
it
from
sorts
was
tricky,
because
I
had
to
install
this
like
lib
gpg
and
all
this
other
stuff
yeah
dive
dive.
So
yeah
there.
It's
like
there's
a
stack
right,
there's
three
things:
there's
like
the
scopio
there's
dive
and
I
don't
know
what
else
but
there's
like
three
different
things:
that
kind
of
fit
into
the
to
the
to
the
layers
here.
So
we
can
check
out
dive
and
then
jump
over
to
images.
A
Okay,
yeah
dive
so
tool
for
exploring
a
docker
image
layer
contents
and
discovering
ways
to
shrink
the
size
yeah.
So
dive
is
another
one
of
these
tools
that
you
might
might
use
and
actually
it
looks
like
they
have
a
demo
here
for
us.
So
let's
look
at
what
they're
doing
in
this
little
in
this
giphy.
So
it
looks
like
they're,
oh
yeah.
How
do
I
get?
A
Oh
here
we
go
so
they
they
are
opening
up
the
file
tree
and
then
they're
jumping
around
and
they
can
look
at
every
shaw
for
every
intermediate
image
and
then
I
guess
they
can
see
the
diff
in
the
file
tree.
That's
really
cool
through,
like
four
that's
yeah
wow,
so
they
can
compare
the
file
tree.
I
guess
at
a
given
image,
add
a
diff
at
a
given
step
in
the
docker,
build
that's
cool,
yeah,
so
yeah.
I
I'm
kind
of
scared.
A
I'm
scared
of
what
I
would
see
if
I
like
use
this
on
my
stuff
like
okay.
So
so
you
can
make
your
make
your
images
pull
down
a
lot
faster.
That's
really
important
people
talk
about
how
it's
not
that
important.
But
the
reason
it's
important
is
that
nowadays,
like
every
app
has
like
50
micro
services,
so
like
it's
like
optimizing
any
given
one
image,
isn't
a
huge
deal
but
like
when
you're
deploying
an
app
that
has
50
containers
like
optimizing.
A
How
those
containers
pull
down
actually
is
is
is
super
important
because
people
are
making
smaller
and
smaller
micro
services.
Nowadays,
google
crane
is
useful
to
pull
an
image
and
play
with
remote
message:
okay,
cool,
so
we've
got
google
crane,
that's
worth
looking
into
and
then
we're
gonna.
Okay,
that's
the
last
one
though
google
crane
and
then
we're
gonna
go
to
okay,
managing
container
images.
Okay
and
it's
got
all
these
okay.
A
It's
got
all
these
tools
to
to
dig
through
a
repository
yeah,
and
I
don't
know
if
folks
use
harbor,
but
harbor
allows
you
to
do
a
lot
of
this
stuff.
I
guess
has
a
lot
of
cool
server-side
tools
by
the
way
too.
So,
let's
go
and
let's
go
and
switch
over
to
the
this
is
really
cool.
It
scans,
stuff
and
sign
stuff,
and
you
know
it's
like
an
end
and
I
think
you
can
upload
helm
charts
to
it.
A
I
haven't
done
much
with
with
harbor,
but
you
know
it's
a
very
developer,
friendly
docker
registry,
if
you're
in
the
ci
business.
So
let's
jump
over
to
image
builder.
So
we've
talked
about
containers
right.
So,
let's
just
let's
just
zoom
out
here
and
let's
go
let's
go
over
here
and
let's
make
a
new
sort
of
diagram
and
let's
talk
about
where
we
all
fit
so
in
a
cluster
in
a
normal
cluster.
Here's
what
you
have
right!
You
have
nodes!
A
Okay,
and
you
know
I
might
have
a
linux
node
and
a
windows
node
and
a
linux
node,
and
maybe
this
is
an
ubuntu
node,
and
maybe
this
is
a
windows,
server,
node,
2019
or
whatever,
and
then
maybe
this
is
also
a
linux
node.
But
maybe
this
is
a
linux
node
that
has
some
libraries
for
like
gpu
or
something
or
maybe
it's
man.
Maybe
it's
not
ubuntu.
Maybe
it's
a
different
just
maybe
it's
sent
to
us
right.
A
So
I
have
these
three
different
os's
and
in
a
cluster
right
you
might
have
apps
that
run
in
different
places
for
different
reasons
right.
So
each
one
of
these
is
a
kubelet
and
what
I
want
to
do
is
I
maybe
want
to
update
these
images.
I
want
to
update.
I
want
to
build
new
versions
of
these
images
and
in
order
to
do
that,
each
one
of
these
needs
kublet.exe.
A
It
needs
coup,
proxy
dot,
ex
or
well.
The
windows
one
needs
kube
kublet,
exe
and
coopproxy.exe,
but
then
these
need
the
kubelet,
the
linux
executable
and
the
cuproxy
executable
right
cube
these.
My
writing
is
horrible
right
now,
yeah,
executable
right
and
then
I
may
actually
also
want
to
put
like
binaries
on
here
like
I
might
want
to
put
cni
binaries
right
like
say,
I'm
running
andrea
right,
it's
like
say
I'm
running
andrea.
A
So
then
each
one
of
these
here
I'd
want
a
different
I'd
want
andrea
agent
and
then
here
maybe
I'd
want
andrea,
executable
right
same
thing
for
calico
right.
Maybe
maybe
I
don't
want
andrea,
maybe
I'm
using
calico
for
my
ci
ci
in
the
same
situation,
I
need
to
put
calico
here.
I
need
to
put
the
calico
executable
here
on
the
windows
machine,
so
each
one
of
these
images
is
totally
different
and
I
need
a
way
to
to
build
these
right.
I
need
a.
I
know.
A
A
A
A
The
idea
of
the
golden
image
right
and-
and
this
is
not
just
the
kubernetes
thing-
this
is
a
like
azure
and
amazon,
and
every
cloud
has
some
kind
of
image
building
service
that
allows
you
to
build
os's
like
kind
of
an
immutable
way
right,
and
so
so
so.
But
this
is
the
upstream
kubernetes
image
builder
that
I'm
that
that
I'm
talking
about
right
now-
and
so,
if
I
cd
here
and
I
clone
this
clone
this
down,
I
can,
let
me
make
sure,
there's
nothing
in
output,
because
I
tested.
A
A
Oh
there's
nothing
in
there,
okay,
so
if
I
do
make,
I
can
what
I
can
do
here
is
I'm
running
vsphere
vmware
fusion
here,
but
you
don't
need
vmware
fusion,
you
can
you
can
make
an
os
image
I'll
show
you
how
this
work
we'll
go
through
this
again,
but
you
know
you
can
make
all
these
different.
There's
all
these
different
ways
to
build
these
right.
So
this
is
really.
This
has
been
curated
upstream
and
you
can
just
grab
this
for
your
for
your
internal
systems
or
whatever.
A
If
your
boss
tells
you
they
want
you
to
preload
all
of
your
kubernetes
stuff
on
there.
So
the
vms
start
up
a
lot
faster
and
whatnot.
You
can
make
these
and
then
you
can
so
we've
got
all
these
different
things
and
you
can
you
can
run
these
in
azure.
You
can
run
you
can,
so
you
can
build
azure
images,
you
can
build
amis,
you
can
build,
you
can
build,
build
ovas
and
that's
what
we
do
for
here
at
vmware
right.
A
A
So,
but
if
you
do
local,
what
it
does
is,
it
will
use
vmware
fusion
and
it
will
like
fire
up
a
fusion
instance,
as
you
can
see
here
and
what's
happening
now,
is
it's
going
to
fire
this
fusion
instance
up
and
in
this
case,
I'm
using
image
builder
to
actually
build
a
windows,
kubelet,
okay,
and
so
that's
where
so.
So.
A
A
I
don't
actually
have
that
on
me,
but
anyways,
as
you
can
see,
I've
spinning
up
of
windows,
vm
here
and
now
what's
happening,
is
image
builder
is
running
a
bunch
of
commands
against
that
vm
to
set
stuff
up,
and
I'm
not
going
to
run
this
whole
thing
because
it
takes
a
while.
But
what
it's
doing
is
it's
executing
in
here
and
wait.
A
A
As
you
can
see
here,
it
is
it's
copying
stuff
over
the
os
is
booting
up
now
and
then,
if
I
was
to
let
this
run
all
day
or
whatever,
however
long
it
takes
like,
usually
takes
like
half
hour,
it
would
eventually
pull
down
stuff
from
the
internet
and
like
after
pulling
that
stuff
down.
It
would
like,
let's
see
where'd
it
go
here,
it
is
yeah
after
pulling
all
this
stuff
down.
A
So
I
could
just
mount
this
into
my
hypervisor
or
whatever
it
was,
was
a
vsphere
or
whatever,
and
and
start
it
up,
and
as
long
as
I
put
some
basic
information
into
its
cloud
in
it,
you
would
be
able
to
find
the
kubelet
so
you'd
be
able
to
find
the
api
server
and
cube
adm
joined
to
it.
So
this
is
image
builder
and
now,
but
this
is
more
like
from
my
perspective,
the
vmware
perspective
james,
is
going
to
kind
of
show
you
how
he
looks
at
this
whole
story.
A
A
A
D
D
D
So
there's
the
image
builder
for
cluster
api
and
that's
the
one
that
jay
was
just
talking
about
and
then
there's
a
couple
other
tools
here
for
other
types
of
configuration
which
I
won't
go
into,
but
finding
just
the
image
builder
for
the
cluster
api
and
they
say
it's
for
cluster
api,
but
really
what
it
does
is.
It
creates
a
vm
image,
and
this
is
for
linux
or
windows.
D
It
creates
the
the
base
os
image
and
it
basically
preps
it
so
that
you
can
run
cube
adm
against
it,
and
so
I
just
thought
I'd
quickly
show
you
kind
of
what
this
looks
like.
So
if
we
go
in
here,
it's
underneath
this
images,
folder
and
then
cappy
and
in
here
there's
the
make
file.
So
that
was
the
when
jay
kind
of
showed
hey
make
this.
This
is
where
we
wire
everything
up
for
for
setting
everything
up
and
here
the
way
it's
the
way
it
works.
D
D
Goss
validate
and
it
will
go,
make
sure
everything's
in
the
right
places
for
those
servers.
So
so
we
have
that
wired
up
for
the
linux
vms
and
the
the
windows
vm.
So
that
makes
sure
that
you
know
cubelet's
in
the
right
place
and
configured
properly
has
all
the
cube
adms
there.
It's
the
right
version
of
cube
8m
so
that
we're
confident
that
our
vhd
that
we're
or
the
image
the
vm
image
that
we're
generating
that
we're
going
to
use
across
in
our
entire
cluster
is
is
is
exactly
what
we
expected.
D
So
we
wire
all
that
in
and
and
run
the
golf's
test
against
it
and
then
so
that's
where
kind
of
packer
is
and
you
can
see,
we've
got
the
amis
and
they
all
look
pretty
similar.
So
let's
see
yeah
so
so
they've
got
the
builders
and
packer
gives
you
a
bunch
of
different
builders
across
the
the
solution
here
and
then.
D
The
other
thing
that's
in
here
is
the
ansible.
So
to
do
all
the
configuration
we
we
do
that
all
through
ansible
and
in
the
roles
we
can
see.
There's
we
install
container
d.
We've
got
like
all
the
different
kubernetes
components.
So,
if
we
look
in
here
we'll
see,
this
is
where
we
pull
down
cube
adm,
make
sure
that
we
pre-load
any
images.
Let's
see
what
else
is.
D
A
A
A
D
The
init
process
so
we're
actually
using
cloud-based
init.
So
this
is
a
tool
that
is
in
cross-platform
version
of
cloud
init.
It
was
it's
developed
by
cloud
base
and
it
has
the
same
type
of
interface,
so
it
has
the
user
data.
So
just
it
kind
of
implements
all
the
same
user
interfaces
that
you
would
see
in
cloud.
Init.
D
A
Network
yeah
so
yeah,
so
that
whole,
you
know
we
do
a
lot
of
stuff
in
the
cube.
Adm
join
because
you
can.
You
can
have
like
post
join
stuff
that
you
do
so
it
in
cluster
api.
We
do
a
lot
of
tricks
inside
the
kube.
Adm
join
configuration
so
that,
like
after
you
know
it
could
it's
called
like
a
kubernetes
post.
What's
it
called
a
kubernetes
post
post,
coupe,
adm
command
and
you
know
in
the
in
the
post,
cube
adm
command.
A
That's
when
we
we
install
andrea
agent,
dot,
exe,
so
suresh.
The
big
thing
to
think
of
is
this
like
if
you're
booting,
a
windows
node
like
you're
booting,
a
linux,
node
you're,
going
to
run
your
cni
in
a
container
you're
booting,
a
windows
node,
you
do
kube
adm
join
and
you
join
the
cluster
and
that's
like
a
windows,
executable
step
and
it's
very
similar
to
the
linux
side
of
things.
A
But
then,
after
that
is
totally
different
and
the
initialization
might
use
nssm
to
run
certain
services,
for
example,
to
run
your
cni
right
or
to
run
csi
proxy,
which
allows
you
to
do
csi
on
windows,
because
there's
no
privileged
containers
on
windows,
so
you
kind
of
have
to
do
you
said
systemd
and
that's
you're
kind
of
exactly
right.
There
right,
you
need
to
use
like
nssm,
which
is
a
windows
system
d
like
thing
to
sort
of
start
services
up.
A
D
A
Yeah
so
some
of
the
windows
services
self-install
themselves
as
windows
services
in
so
they're
kind
of
like
system
d-ish.
In
that
sense
right
they
have
the
same
sort
of
like
they
just
work,
but
other
things
like
andrea
and
calico
and
things
you
might
want
to
run.
They
don't
have
that
functionality
embedded
in
them
and
so
yeah
nssm
yeah
nssm
yeah.
So
what
you
do
is
you
wrap
those
in
an
ssm
and
that
will
start
those
services
for
you.
A
C
D
We
go
here,
I
think
this
is
where
oh,
that's
going
down
into
the
actual
service
creation,
but
yeah
you
just
pass
it
some
extra
flags
and
it
will
actually
register
itself
as
a
windows
service.
And
so
you
can
go
like
look
at
the
services
in
windows
and
you'll,
see
container
d
or
cubelet
or
any
of
them
running
in
there.
A
D
So
I
so
so
you've
got
packer
and
then
you've
got
ansible
and
then,
if
we
provide
in
cluster
api,
we
provide
quite
a
few
like
configurations
out
of
the
box.
You
just
you
know
you
can,
just
as
you
saw
you
just
run,
that
make
make
command
and
it'll
spin
up
and
it'll
install
the
latest
versions
of
everything
and
make
sure
it's
all
wired
together,
but
oftentimes.
You
need
to
be
able
to
customize
these
things.
D
So
the
way
that
you
typically
do,
that
is
you,
create
a
custom
json
and
then
you
can
provide
additional
information.
So
right
now,
I'm
working
on
adding
host
process
support
and
I
need
a
custom
binary
to
do
that
for
for
the
windows.
So
with
the
it
gives
you
these
different
lists
to
be
able
to
pull
different
binaries,
and
so
here
I
just
configure
it
to
pull
this
extra
binary.
D
That
isn't
you
know,
cluster
api
or
image
builder
doesn't
know
anything
about
it,
but
I
can
tell
it
to
go,
pull
it
and
then
I
can
go
tell
it
to
stick
it
into
the
container
d
folder
or
any
folder,
and
this
works
for
windows
or
linux,
and
it
works
the
same
way
and
then
the
the
way
that
you
end
up
using
that.
So
you
can
kind
of
write
your
own
packages
and
you
can
just
say
packer
extra
bars
and
you
can
customize
this
thing
down
to.
D
You
know
the
versions,
extra
images
that
you
want
to
pre-pull
onto
the
to
the
vm,
so
that
you
know
when
a
container
starts
it
doesn't
have
to
go
and
pull
it
just
starts
up
and-
and
you
just
you-
can
export
it
through
these
extra
files,
like
that,
so
it's
pretty
cool
to
be
able
to
you
know
customize
it
and
take.
You
know
what
the
community
has
given
you
out
of
the
box
and
then
you
know,
add
all
those
little
bells
and
whistles
or
removing
some
things.
If
you
need
to
so.
A
Yeah
cool
so
you're
right
there
in
that
burrito
area.
Do
you
want
to
show
them
the
hack
script
that
we
use
and
and
in
the
upstream
and
then
I
can,
I
can
show
them
the
what
it
actually
looks
like
you
know,
in
terms
of
when
you
actually
sort
of
extend
it,
because
you
know
the
the
thing.
That's
really
you
want
to
do
with
windows
images.
A
So
the
only
difference
is.
I
would
go
edit
those
that
json
file,
that
that
was
just
that
james
was
showing
and
and
so
that
it
pulled
the
kubelet.exe
and
all
these
different
things
from
from
from
the
ip
address
of
where
I
was
running
this
process
at
and
then
this
process
would
be
running
from
a
directory
that
had
all
my
kubernetes
executables
in
it,
and
then
they
would
pull
down
those
executables.
So
that's
how
a
vendor
packages
kubernetes
for
windows
right
and
for
linux.
A
A
I
don't
know
if
he
calls
his
things
that
host
all
the
crew,
all
the
kubernetes
artifacts
burritos,
but
that's
what
I
call
mine,
and
so
I
have
my
burrito
over
here
and
it's
got.
I
don't
know
why
my
screen
is
like
frozen
here,
but
yeah.
It's
it's
got
all
these
artifacts
in
it
so
like
when
I
was
to
run
image
builder
right.
A
A
A
A
A
D
There
are
some
docs
for
image
builder,
specifically
the
the
images
that
they
are
designed
and
typically
used
in
a
like
a
cluster
api
providers
component,
so
in
azure
the
way
that
we
bootstrap
them
is
using
cloud
init,
and
you
just
pass
that
data
to
the
cloud
init
and
there's
a
command
you
just
it's.
Basically,
the
cube
adm
join
command
that
gets
passed
to
the
cloud
init,
and
then
that
knows
how
to
boot
the
vm
and
join
the
cluster
via
cube8m.
D
If
you
build
one
of
these
images,
it
will
come
ready
and
you
all
you
need
to
do
is
log
into
the
vm
and
run
a
cube
adm
command.
So
if
you
wanted
to
do
it
that
way,
you
could
do
it
manually.
That's
how
I
actually
test
these
things
like
these
images
with
when
I'm
not
working
inside
cluster
api,
I
just
remote
into
the
vm
itself
and
then
do
I
go
to
my
control.
D
Plane,
node,
get
a
cube,
adm
join
token
and
then
run
it
right
on
the
vm
itself
and
then
it'll
join
the
the
cluster.
So
you
can
you
basically
follow
the
same
exact
docks
that
are
in
upstream
on
how
to
do.
Cubitium
joins
it's
just
that
all
the
components
and
everything
is
placed
into
the
right
places
and
wired
up
properly
for
you
to
just
run
that
command,
so
that
that
and
that's
essentially
what
clap
cluster
api
does
to
join
in.
You
know
a
new
node
when
you're
when
you're
running
it.
C
A
Like
you
know,
you
don't
want
to
install
ssh
from
the
internet
every
time
you
every
time
you
have
to
add
a
new
node
to
your
cluster
right,
especially
because
I
don't
know
why,
but
like
lately,
we've
been
finding
that
ssh
installation
fails
like
20
percent
of
the
time
in
some
of
our
some
of
our
image.
Builder
runs
so,
but
maybe
our
friends
at
microsoft
can
figure
that
out
for
us
at
some
point.
D
Yeah,
that's
awesome.
I
was
just
looking
to
see
if
I
could
find
the
cube
adm
like
command
for
the
cloud
init
so
that
you
can
see
kind
of
what
yeah.
So
this
is
the
cube8am
bootstrap
script,
it's
basically.
This
is
what
cubadm
and
our
cluster
api
injects
into
the
cloud
in
it
for
for
new
new
vms
joining,
and
so
you
can
see,
there's
some
retry
logic
and
things
in
here,
but
the
where
is
it
somewhere
in
here?
You
can
see
it's.
D
They
they
they
go,
get
the
cube,
adm
commands
and
then
they
just
that's.
Basically
the
only
command.
That's
inside
the
the
cloud
in
it
there's
a
bunch
of
retries
and
other
things
in
there.
But
beyond
that,
that's
that's
how
you
join
the
the
note.
It's
there's
not
too
much
happening
behind
the
scenes
beyond
that,
so
yeah
just
thought.
I'd
show
that
there.
C
D
C
D
A
So,
while
you're
opening
that
up
me
and
james
were
talking
about
this
earlier
and
we
both
realized,
we
needed
to
google
what
an
ova
and
an
ovf
and
all
that
stuff
were-
and
I
put
a
link
in
the
thing
in
the
notes
here
about
how
all
this
stuff
is
works.
But
there's
ova,
oh
wait:
did
I
lose
you?
Do
I
need
to
add
you
again
yeah
there.
D
A
A
D
A
D
Yeah-
and
so
we
can
see
that
there's
a
bunch
of
different
things
on
here
and
by
convention,
I'm
not
sure
who
created
this
convention,
but
the
k
folder
is
where,
like
all
the
cubelet
and
other
cubed
m
and
all
those
things
land.
So
we
can
see
that
here
we've
got
cubectl
we've
got
cubelet,
we've
got
nssm,
there's
some
additional
bootstrap
stuff
that
we
put
in
a
powershell
script
for
for
windows,
but
then
we
also
have
image
builder.
D
D
There
is
something
with
that:
ps
ps
file,
but
yeah,
so
it
runs
there.
So
what
we
could
actually
do
is
do
a
cube,
adm
join
here.
If
I
had
the
control
plane
nodes
up,
I
could
actually
just
run
a
cubadium
join
and
we'd
see
that
this
this
cluster
was
actually
joined
to
the
cluster.
The
other
thing
that
we
have
in
here
is
the.
D
If
I
go
into
temp-
and
here
is
where
all
the
goss
stuff
is
so
this
this.
These
are
those
goth
specs
that
I
was
talking
about
before.
If
I
do
gossip
spec
here,
we
can
see
these
are.
This
is
we're
saying
we
want
a
windows
service
that
is
called
cubelet
and
we
want
to
make
sure
that
that
service
is
exists
and
we
make
sure
that
it's
got
the
right
required
services.
So
here
we're
also
we've
also
installed
the
ssh.
We
want
to
make
sure
that
service
is
running
down
here.
D
D
D
A
Yeah,
so
this
is
for
what's
his
name's
question,
I
forgot
sir,
was
a
serrations
question
right
like
what
do
you
do
for
the
systems
right?
You
do
get
dash
service
and
and
then
that's
that's
the
equivalent
of
system,
ctl
status
right
on
a
windows
box.
So
the
typical
thing
we
do.
You
know
the
first
thing
you
do
when
you
get
into
a
windows.
Note,
is
you
do
get
service
star
kube
and
you
look
at.
D
A
D
D
C
A
All
right
so
I
mean
that's
it
like
that's
everything
like
I
mean
that's
everything
now
I
mean
we've
got
usually
we
go
an
hour
and
a
half.
When
I
do
this.
Does
anybody
want
to
like
dive
in
anything
else?
Like
we
didn't,
I
I
don't.
I
can't
think
of
anything.
I
showed
everybody
the
file
tree
for
a
container
right.
We
we
did
that
did
we
do
that.
I
don't
even
remember
anymore.
We
dug
into
that
right
and
we
did.
A
We
got
to
look
at
vms.
We
got
to
look
at
the
the,
so
we
got
to
look
at
the
ovas
and
the
vhd's
that
are
actually
the
currency
of
the
kubelets
and
the
and
the
api
servers
and
all
that
stuff
that
are
actually
running
in
a
real
cluster
and
those
that
get
made.
Then
we
got
to
look
at
the
agn
host.
We
looked
at
that
and
and
and
the
pause
container,
which
were
both
kind
of
you
know,
used
for
internal
kubernetes
development,
but
are
pretty
good
analogs
to
what
happens
in
the
real
world.
A
If
you
wanted
to
build
a
multi-arch
image-
and
I
mean
I
think-
that's
probably-
I
think
that's
probably
most
of
what
we
wanted
to
dig
into
and
if
you
want
to
learn
more
about
this,
the
best
places
to
look
at
look
at
the
pause
image.
It's
a
great
example
of
how
to
build
a
multi-architecture
container
image
and
you
can
look
at
the
makefile
for
that
in
upstream
k-8s.
A
D
A
A
Okay,
cool!
Well,
I
mean,
I
think
that
that's
I
think
I
think.
That's
I
think.
That's
that's!
That's
gonna!
Be
that's
going
to
be
it
for
today,
then
so
so,
thanks
david
for
all
your
thanks,
david
and
suresh,
and
everybody
else
for
all
your
questions.
Ricardo
ricardo's!
Here!
If
ricardo
had
a
question
about
bsd
he
he
actually
was
working
on
trying
to
port
coop
to
actually
build
bsd
support
in
so
ricardo.
Before
we
sign
off.
Do
you
want
to
like
sort
of
tell
us
how
that
all
went
where
that
ended
up
yeah.
E
I
I've
just
joined
it
just
to
explain:
we've
we've
been
trying
to
make
cubelet
run
with
with
3bsdjs,
so
jay
asked
me
about
the
images
as
well.
I've
been
trying
to
work
on
something
like
that,
because
I
I
don't
know
if
a
few
folks
know
about
how
jails
works,
but
actually
it's
we
can
say
that's
a
pretty
pretty.
E
That
was
an
implementation
of
containers
of
freebsd
and
but
when
you,
when
you
use
that
image
in
in
freebsd,
you
will
need
to
bootstrap
everything
like
a
new
kernel
image
and
a
new
base
image
and
what
we've
been
trying
to
do.
Actually,
I'm
pretty
freeze
now,
but
I
wanna
I
wanna.
Try
again
is
that
you,
you
might
download
that
base
image,
which
is
every
time
the
same
for
freebsd
and
then
over.
Those
images
put
layers
like
I
want
to
run
an
apache
or
I
want
to
run
a
ph
php
fpm
or
something
like
that.
E
A
E
Yeah
I
I've
started
to
work
in
a
virtual
cubelet,
which
you
can
simulate
a
cubelet
in
freebsd,
and
then
it
it
watches
the
pods
that
needs
to
run
into
the
hosts
yeah
you
you
got
it
right
so,
but
I
but
I
have
stopped
it
like,
because
I
didn't
I
didn't
get
got
yet
how
to
deal
with
the
networking
thing
and
the
start
thing
just
basics.
Just.
A
Yeah
cool,
okay,
so
so
yeah,
like
I
mean
I
you
know,
and
this
is
like
you
know,
we're
laughing
and
everything,
but
like
this
stuff
is
important,
because
you
know
why
this
is
really
cool
stuff.
To
dig
into
is
because
it
gives
you
a
break
from
thinking
about
things
in
the
conventional
way,
and
it
gives
you
time
to
really
understand
the
internals
of
how
kubernetes
works,
what
it
means
to
create
a
cluster,
what
the
apis
actually
represent,
what
it
means
to
create
a
cni
provider.
A
I
didn't
really
understand
what
cni
was
until
I
tried
to
get
it
working
on
windows,
you
know
and
then
failed,
and
so,
like
you
know,
that's
it
really
shows
you
what
the
actual
interface
is
between
between
between
the
implementation
of
kubernetes,
that
you're,
that
you're,
that
you're,
using
and
and
what
kubernetes
itself
is
right
and
there's.
No,
you
can't
put
that
into
words.
The
only
way
you
can
only
experience
it,
so
I
would
encourage
everybody
here
to
try
to
run
a
mixed
operating
system.
Kubernetes
distribution
at
some
point
at
some.
A
E
You
can
run
you
can
run
cni
you
can.
You
can
create
a
shell
screen
that
simulates
a
cni,
and
you
like
it's
because
cni
after
all,
is
just
google
ad
calling
another
program
and
asking
hey
allocate
an
ip
address,
so
we
we
can.
We
can
do
that.
Some
some
in
some
future
jay
like
programming,
our
own
cni
yeah,.
A
There's
and
there's
definitely
youtube
talks
out
there
that
have
done
this.
So
if
you
look
for
you
know,
building
a
cni
from
scratch
and
stuff,
you
could
do
that.
We
could
do
that
on
tgik
sometime.
So,
if
you
all
want
us
to
do
that,
let
us
know
and
leave
some
comments
on
this
live
stream,
video
or
whatever,
and
thank
you
thanks
thanks
for
coming
yeah.