►
Description
Jaime Magiera and Josef Meier demonstrate automated installation of OKD on vSphere with User Provisioned Infrastructure (UPI).
A
B
C
All
right,
I'm
going
to
just
pop
into
the
other
ones,
just
know
that
you're
recording
you've
got
about
10
people
watching
you
and
take
it
away
and
you
don't
have
to
go
for
six
hours.
There
are
I
set
it
up,
so
you
could
go
as
long
as
you
wanted,
but
both
six
hours
you're
doing
good
and
you're
putting
tech
support.
So
that's
the
other
thing
remind
people
that
you're
not
doing
tech
support.
You're
you're
demoing
take
care
thanks.
B
B
A
A
Is
this
upi
or
ipi?
So
oh
you're,
asking
okay,
so
essentially
a
worker
is
bound
to
its
control
plane
and,
if
you
are
going
to,
if
you
want
to
use
a
worker
on
a
different
cluster,
you
basically
need
to
redeploy
it,
and
so
that's
what
the
that's
one
of
the
foundations
of
f
costs
of
fedora
core
os
is
that
when
you
want
to
make
a
change,
you
basically
just
redeploy
the
node,
so
you
would
just
redeploy
the
node
by
taking
the
metadata.
A
You
could
do
this
via
upi.
If
you
did
upi
take
the
metadata
that
was
generated
in
the
ignition
config
file
and
then
insert
that
into
the
vm
and
there's
a
flag
you
have
to
set
it.
I
can't
think
about
the
top
of
my
head,
but
basically,
when
you
boot
up
the
vm
next
time,
it'll
reprovision
and
make
that
connection
to
the
other
control
plane
that
you
want
to
move
it
to.
B
B
Okay,
two.
Okay!
I
think
it's
worth
two
yeah
three!
Okay,
three
is
a
three
is
a
set.
I
could
okay,
I
will
spend
a
few
minutes
about
that.
So
the
first
thing
is:
you
should
clone
this
repository
here
say
okd
repository
I've
done
that
in
advance.
I
I
built
a
cluster
an
hour
ago
from
that
this
is
a
repository,
I'm
in
the
vsphere
terra
from
folder
a
few
days
ago.
I
I
can
try
to
make
it
bigger.
Maybe
that
helps
oops.
It
scrolls
away.
B
B
B
B
B
This
is
the
installation
I've
bought
with
my
150
bucks,
vm
user
group,
this
version
of
vsphere,
I'm
using
that
in
my
home
lab
here
you
see
the
data
center
and
the
storage
is
called
datastore2
and
you
have
to
fill
out
that
in
this
terraform,
but
the
apples
file,
you
have
to
say
how
much
masters,
how
much
workers
you
want
to
have
deployed.
You
have
to
provide
a
few
more
ignition
configuration
based
informations.
B
You
can
leave
this
tool
fixed
and
here
you
have
to
provide,
say
url
of
a
of
web
server
where
you
serve
the
bootstrap
ignition
file
yeah,
because,
as
maybe
you
remember
from
my
slides
that
in
the
first
step,
the
bootstrap
node
will
fetch
its
ignition
file
from
somewhere
and
that's
what
you
have
to
provide
here.
You
have
to
provide
the
location
of
web
server
where
the
bootstrap
ignition
policies.
B
I
will
show
you
where,
how
you
create
that
in
a
few
minutes
then
yeah
here
I
you
have
to
you-
can
provide
the
mac
addresses
of
your
vms.
The
bootstrap
vm
has
the
smac
address
here:
control,
plane,
three
masters.
Three
mac
addresses
three
workers,
three
make
addresses
for
the
workers
and
that's
pretty
much
everything
you
have
to
provide
here.
That's
a
most
work
you
have
to
do.
B
B
If
you
go
to
the
openshift
okd
site,
you
have
your
releases
here
on
the
right
and
you
simply
have
to
choose
the
installer
binary
for
the
version
you
are
interested
in.
You
can
download
it
yep.
In
this
case
I
would
download
yeah,
maybe
this
openshift
installer,
because
I'm
on
linux,
I
would
download
this
one.
B
B
B
So
if
you
later
in
your
running
cluster,
want
and
to
dynamically,
create
more
workers,
you
can
open
shift
where
okd
will
use
the
vsphere
api
with
the
credentials
provided
here
in
this
phase
and
to
provision
more
more
workers
on
the
fly.
Also,
an
auto
scaler
could
use
the
b
sphere
credentials
provided
here
to
create
more
nodes
dynamically.
B
The
next
thing
is
you
have
to
do
this.
You
have
more,
you
have
a
few
possibilities.
What
you
can
do
here,
I
create
ignition
config
files
also
create
well.
You
also
could
create
the
manifests
of
the
cluster
operators
in
this
stage
and
configure
them
and
afterwards
build
ignition
config
files
from
this
manifests.
B
That's
useful,
if
you
don't
want
to
create
a
cluster
and
configure
it
afterwards,
but
if
you
want
to
create
a
cluster
that
is
pre-configured
with
your
with
your
implementation
details
this
example,
I
don't
do
anything
like
that.
I
straightforward
create
my
ignition
files,
and
here
we
have
the
most
important
one.
It's
a
bootstrap
ignition
file.
We
can
have
a
look
inside
of
it
looks
like
it's
a
huge
json
file.
B
B
B
B
I
don't
know
the
exact
size
it's
possible
to
support
a
vm
in
vsphere
for
the
ignition
files,
but
it's
it's
much
smaller
than
what's
needed,
and
this
two
phase
ignition
fetch
is
used
to
overcome
this.
This
limit
in
vsphere
also
for
the
bootstrap
vm.
We
have
to
use
a
small
stub,
it's
called
append.
Bootstrap
looks
it's
even
smaller
than
the
master
ignition
file,
it
contains,
say
address
of
a
web
server.
I
used
apache
simple
apache
web
server
on
this
helper
node
here,
and
this
apache
provides
a
bootstrap
end
file.
B
I
hope
you
can
read
it.
Okay,
can
I
make
it
bigger
here?
We
see
all
cpms
and
this
is
a
running
cluster
yeah.
It
should
look
like
cis
in
the
end,
if
you
see
a
dashboard
and
what
is
this,
the
samples?
Okay,
it's
don't
worry
about
that
and
you
have
normally
three
green
check
marks
and
you
are
fine.
Then
you
are
have
your
first
running
okd
cluster.
B
B
Okay,
great
because
I
think
lots
of
people
are
struggling
in
this
first
phase
that
the
bootstrap
vm
comes
up.
It
gets
its
bootstrap
ignition
file,
but
afterwards
things
get
stuck
yeah,
and
maybe
it's
also
interesting
how
you
can
debug
that
jamie?
We
can
maybe
do
a
debug
session.
Maybe
we
can
produce
a
problem,
a
common
one
and
try
to
find
a
solution,
because
you
will
find
videos
in
the
internet
that
show
you
the
perfect
world.
B
E
B
B
D
B
To
this
section
here
you
have
also
to
provide
the
information
about
your
vsphere
cluster.
You
can
provide
disk
space
memory
of
your
vms
that
are
created
in
bisphere,
cpus
and
so
on,
data
center
data
store
and
so
on,
and
if
you
provide
this
machine
set
to
vsphere,
you
can
simply
do
two
things.
I
I
don't
have
a
machine
set
up
tonight
and
try
to,
but
I
don't
think
it
will
work.
Okay.
B
B
B
B
B
B
B
B
B
B
B
B
If
you
use
resources
like,
for
example,
put
disruption
budgets,
then
it
can
be
that
if
the
cluster
auto
scaler
tries
to
delete
nodes
that
this
pdb's
support
disruption
budget
are
blocking
the
eviction
of
notes,
because
then
you
are
yeah
our
I
don't
know
the
english
word,
our
yeah,
destroying
the
contract
of
how
much
pods
must
run
in
your
deployment.
That's
exactly
what
rpd
or
pdb
does
said.
It
defines
a
minimum,
a
number
of
pots
that
always
must
run,
but
the
autoscaler
works.
Pretty
good.
B
B
A
Any
more
questions
before
we
move
on
to
sort
of
automating
the
process
api,
and
I
did
put
two
polls
in
the
polls.
There's
are
you
currently
running
okd
and
if,
yes,
what
version
are
you
running
it'd,
be
interesting
to
see
what
folks
have
and
be
able
to
get
a
sense
of
what
folks
are
running,
and
so
now,
let's
talk
a
little
bit
about
automating
the
upi
process.
A
So
if
you're
doing
upi,
you
know,
as
mentioned
before,
you're
going
to
want
to
have
a
proxy
or
a
load,
balancer
and
possibly
a
proxy
if
you're
on
a
private
network
and
so
there's
a
link
to
some
scripts
or
a
script
that
I
wrote
I'll
put
this
in
the
chat.
This
is
a
project.
I've
been
working
on
for
a
while,
it's
called
oct
and
it's
basically
automating
the
upi
process
so
that
you
can
continuously
test
okd,
cluster,
installs
and
and
everything
after
that.
That
goes
with
that.
A
Basically,
you
need
your
load
balancing
to
handle
the
api
in
ingress.
So
you'll
need
two
different
pools
for
that
and
again,
if
you're
on
a
private
network
you're
going
to
want
to
use
a
proxy
for
outgoing
traffic,
that's
for
the
installation
itself
to
download
the
containers
off
of
quay
or
those
for
from
the
testing
releases,
and
also
for
regular
operation
of
your
cluster.
A
For
your
pods
to
do
any
outgoing
traffic
like
running
a
yum,
update
or
composer,
install
or
retrieving
network
resources,
resources
out
on
the
net,
and
so
squid
is
a
good
one.
For
that,
squid
is
something
that
you
can
utilize
pretty
easily
and
configure
pretty
easily.
So
let
me
share
my
screen
here
and.
A
There
we
go,
and
so
here
is
the
repository
for
the
tool
that
I
have
been
working
on
and
it's
oct
and
it's
a
command
line
tools,
simplify
the
process
of
building
and
destroying
okd
clusters
and
vsphere.
This
utilizes,
the
govc
command
and
the
oc
and
cube
control
tools
that
come
with
openshift.
So
govc
is
a
separate
project
that
this
tool
utilizes
and
then
also
oc
and
coup
control
that
are
provided
with
the
okd
and
open
shift
installs,
and
so
here's
the
command
line
argument.
So
we'll
go
through
all
of
them.
A
But
basically
it
allows
you
to
automate
all
of
the
stuff
that
joseph
was
just
talking
about
in
terms
of
having
your
configuration
file
and
it
also
has
a
bunch
of
extra
features.
So
I've
got
a
list
of
the
functions
here,
for
example,
the
tool
checks
if
you
have
oc
installed
and
if
you
don't
have
oc
installed,
it'll
pull
a
version
of
the
tool
down
to
your
machine
to
your
working
directory,
a
bin
folder
in
your
working
directory-
and
this
was
mentioned
before
by
vadim
and
I'll,
expand
on
a
little
bit.
A
So
if
you're
running
a
4.6
cluster-
and
you
happen
to
have
the
oc
tool,
you
can
use
that
to
download
the
installer
and
and
the
oc
binary
for
a
particular
version
like
4
7
or
a
nightly
release
and
whatnot.
That's
something
that
a
lot
of
folks
aren't
really
aware
of,
but
allows
you
to
do
testing
very
efficiently,
just
by
pulling
the
installation
tools
and
the
and
the
relevant
oc,
and
this
script
also
checks.
A
If
you
have
the
govc
tool,
which
is
a
tool
for
working
in
the
command
line
remotely
or
locally
with
vsphere,
so
you
can
create
vms
import
templates,
ova
templates
and
do
all
sorts
of
stuff
very
simply
with
the
govc
tool.
And
so
my
script
works
with
that
and
there's
a
couple
of
functions
within
the
script
that
do
the
the
heavy
lifting
the
first
one
is
install
cluster
tools
and
that
one
installs
oc
control
and
the
open
shift
installer
binary
for
the
version
that
you
want.
A
A
You
generate
those
ignition
config
files
and
the
installer
utilizes,
the
ssh
account
that
was
created.
The
core
account
to
go
in
and
trigger
the
installation
and
whatnot
launch
pre-run
helps
you.
It
basically
generates
those
files
for
you
and
modifies
them
in
the
way
like
inserting
your
what
they
call
a
pool.
A
It
takes
care
of
doing
all
that
for
you,
so
you
once
you've
created
a
template
of
the
config
file.
This
actually
copies
the
template
into
a
fresh
one,
installs,
the
necessary
information
that
you
need
and
then
gets
everything
set
up
so
that
you
can
do
a
deployment
and
deploy
node
function
in
the
script.
Actually
does
the
part
of
generating
a
vm
in
vsphere
for
each
of
the
types
of
nodes
that
you
need:
worker
control,
plane
and
bootstrap,
and
then
inserts
the
appropriate
ignition
config
into
that
and
can
also
booted
up.
A
So
this
is
for
the
individual
node.
This
class
gets
called
by
the
individual
node
build
cluster
is
what
calls
deploy.
Node
build
cluster
basically
takes
all
of
the
information
and
then
calls
deploy
node
for
each
node
that
you
need
deploy.
Node
can
also
be
used
for
deploying
standalone
fedora
core
os
nodes.
So
if
you
want
to
play
around
with
fedora
core
os,
we
talked
about
that
a
little
bit
in
the
main
opening
session.
A
A
A
Okay,
so
I
have
this
append
bootstrap
and
install
config
template,
and
this
script
is
being
run
on
a
control
on
a
on
a
machine
that
I
use
an
installer
machine
that
has
apache
running
on
it.
So
for
upi
you,
if
you're
doing
upi
vsphere
you're,
often
going
to
be
pulling
the
bootstrap
script
off
of
a
web
server.
A
Oh,
for
some
reason
it's
not
working,
I'm
not
sure
why.
But
let
me
see
real
quick
what
I
missed
here,
but
basically
you
can
delete
all
of
this
stuff
or
sorry.
That's
why
destroy
and
cluster
name.
I
forgot
okay,
so
this
allows
you
to
work
with
multiple
clusters
and
I
need
to
put
the
cluster
name,
and
so
we
do.
A
A
All
we
have
is
the
append
bootstrap
which
we
can
reuse,
because
it
doesn't
have
any
unique
data
and
it
just
has
the
web
address
and
then
basic
parameters,
and
then
we've
got
a
template
here
that
template
I
won't
cat
it
out.
But
basically
it's
the
template
that
you
see
in
the
instructions
for
an
install.
A
That
calls
all
the
different
functions
to
do
those
different
stages.
So
I
have
a
script
called
build
logo
since
sort
of
a
wrapper
script
and,
as
you
can
see,
it
calls
all
of
the
different
functions.
This
allows
you
I'm
disabling
it
right
now,
because
it
just
it's
you're
watching
it's
like
watching
paint,
dry
you're
watching
the
import,
but
you
can
import
from
url
into
your
vsphere.
A
The
template
that
you
want.
The
ova
template
that
you
want
to
use
for
your
okd
install
and
you
set.
Your
basic
parameters,
appeared
how
many
masters
workers
the
template,
url
that
you
wanted
to
use
or
the
template
name
if
you're
going
to
skip
that
step,
your
library
within
vsphere,
your
cluster
name,
and
where
you
want
to
put
that
cluster
in
your
vsphere,
what
network
it's
gonna
use!
So
I'm
using
vm
network
the
the
default
network
and
then
the
the
folder
for
the
installs
there.
A
This
source
is
just
reading
in
my
credentials
for
the
tasks
and
then
again
here's
the
imp.
I've
commented
this
out,
but
this
is
importing
the
template
at
the
library
that
you
wanted
this
installs
the
tools,
so
I'm
going
to
install
the
tools
for
a
4.6,
okd,
installation,
pre-run
and
auto
secret.
This
is
auto.
A
Secret
is
a
flag
that
I
created
that
inserts
a
sort
of
dummy
secret
and
this
it
prevents
you
from
having
to
go
to
the
open
shift
portal
to
get
a
a
generated,
pull
secret
and
there's
a
dummy
secret
that
you
don't
really
lose
any
functionality
by
using
the
dummy
secret.
Eventually,
there
is
talk
in
in
okd
of
utilizing
the
functionality
that
the
pull
secret
provides.
A
But
that's
it's
right
now,
it's
not
as
as
doesn't
have
as
many
ramifications
just
using
the
dummy
one,
and
here
we
have
the
the
build
a
call
to
the
build
where
you
provide.
You
know
the
cluster
folder
cluster
name,
node
count
all
of
those
things
that
we
discussed,
and
this
is
a
little
trick
that
I
do
so
that
I
can
use
reserve
dhcp.
A
So
I
know
what
the
name
and
the
number
that
the
nodes
are
going
to
be
at
the
name
and
I
p
number,
but
I'm
not
doing
a
static
ip
installation
static.
Ip
installation
is
a
little
less
flexible
and
this
turns
the
nodes
on
and
then
this
runs
the
openshift
installer
here.
So
I'm
you
know
in
my
installation
folder
and
I
am
going
to
run
that
script-
that
we
were
just
looking
at
build
logos.
A
A
A
It's
done
that
it's
modified
the
manifests
to
make
sure
the
control
nodes
are
unscheduleable
for
worker
pods
and
basically
set
up
the
control
plane.
So
you
don't
have
to
do
those
manual
steps.
It
copied
the
bootstrap
to
my
var
html
for
the
web
server,
and
now
it's
deploying
those
individual
nodes.
So,
as
you
see
here,
the
bootstrap
is
getting
deployed
here
and
now
it's
going
to
go
through
each
one
and
it's
adding
again
that
ignition
metadata
so
that
when
these
are
booted
up,
they
will
automatically
start
performing
their
tasks.
A
The
boot
no,
the
bootstrap
node
will
automatically
start
will
be
available
to
for
the
installer
to
pull
down
the
initial
containers
and
the
worker
nodes
will
be
booted
and
do
their
fedora
core
os
update,
and
then
they
will
restart.
So
that's
a
process
that
folks
may
or
may
not
be
familiar
with
where
essentially,
when
your
nodes
first
boot
up,
they
install
the
updated
version,
the
most
recent
version
of
fedora
core
os
and
then
boot
into
that
again.
A
B
B
A
A
B
B
A
So,
if
you're
utilizing
vsphere's,
but
it
depends
on
your
v-series
configured
like
it,
will
automatically
put
them
in
the
best
place
for
resources.
If
you
have
that
enabled
you
there's
another
route
that
you
could
do,
which
my
script
doesn't
do
that,
but
you
can
do
this
very
easily
and
actually
an
old
version
of
my
script.
You
should
do
this
where
you
can
set
the
v,
the
data
store
on
each
individual
node,
and
it
won't
make
a
difference
for
the
cluster.
A
I
can
share
the
code
with
you
if
you,
if
you
want
to
share
me
your
email
was
it
mark
delaney.
If
you
want
share
me,
your
email
and
I'll
show
you
share
with
you
the
old
code
that
I
have
that
actually
does
allow
you
on
a
per
node
basis,
to
select
your
data
store,
and
that
is
something
that
some
folks
might
want
to
do.
If,
for
example,
they
anticipate
their
workers
are
going
to
have
a
larger
size
versus
your
control
plane
or
whatever.
A
B
D
B
D
A
4.6,
if
it's
not,
I
can
find
that
for
you
and
I'm
happy
to
to
post
it.
If
you
leave
some
contact,
actually
we
can
post
it
in
the
blog
post.
We're
gonna
be
doing
a
blog
post
sort
of
post
this
session.
That's
it's
a
blog
post
of
like
all
the
stuff
that
we
covered
in
this,
and
I
can
put
that
info.
If
I
can't
find
it
real
quick
here.
B
A
B
A
Okay,
so
now
it's
powering
all
of
the
nodes
up
it's
going
through,
so
here's
the
bootstrap.
So
here
is
that
first
run
of
the
bootstrap
node
and
it
always
takes
six
times
on
the
sixth
attempt.
It's
able
to
get
the
ignition
config
off
my
server,
and
now
you
see
it's
reading
that
ignition
config
in
and
configuring
network
and
all
of
that
stuff
resizing.
A
B
A
A
D
A
Okay,
so
it's
going
through
that
process
and
then
you'll
see
it
reboot
again
and
then
a
couple
seconds
later,
the
masternodes
will
be
pulling
their
their
info
and
again,
this
is
calling
that
pool
that
of
the
master
plane.
That's
set
up
in
the
f5
that
I
have,
and
for
for
joseph
it'd,
be
the
h
a
proxy.
A
Yep
here
we
go
rebooting
into
the
latest
version
and
you
can
see
down
here.
The
installer
is
waiting
for
that
initial
20
minutes
for
the
bootstrap
to
start
and
make
itself
available,
and
then
once
the
bootstrap
is
done
in
a
couple
seconds
after
it
reboots
and
and
turns
everything
on
to
provide
the
machine
configs.
This
will
change
and
you'll
then
see
it
switch
to
waiting
for
the
control
plane
for
30
minutes,
I
think,
is
the
amount
of
time
that
does
we.
B
B
A
D
B
D
B
A
So
this
is
the
one
you
want
machine
os
and
was
it
not?
Were
we
missing
it
or
no,
it
really
isn't
up
there.
Okay,
that's
strange!
Okay,
yes,
so
machine
os.
That
tells
you
what
version
of
fedora
core
os
that
it
works
with.
There
are
some
bugs,
though,
whereas
like,
for
example,
the
most
recent
version
of
fedora
os,
you
don't
want
to
start
fresh
with
that.
You
want
to
start
with
a
previous
version
and
that
will
okay
so
and
that'll
work.
A
A
A
A
B
A
A
And
here's
a
step
that
I
need
to
automate.
I
should
put
this
in
my
script.
You
will
have
to
approve
once
the
installation
is
done,
certificates
and
those
certificates,
the
process
of
approving
those
can
take
a
while,
and
so
it's
kind
of
handy
to
have
the
certificate
approval
process
sort
of
there.
It
is
you
have
to
approve
all
the
certificates,
there
won't
be
any
yet.
I
don't
think
because.
B
A
Yeah
and
there
there's
a
handy
little
tool
that
you
can
use
once
this
is
done.
Well,
we
don't
have
to
wait
for
it
to
be
done,
but
essentially
you
call
this
that
will
get
all
of
the
csrs
and
approve
them.
They
added
a
nice
little
flag.
I
noticed
recently
no
run
if
empty
in
the
documentation
which,
before
you
would
get
an
error,
if
there
were
so,
it
actually
is
like
so
x.
Args
actually
doesn't
run
if
there's,
if
it's,
if
it
gets
enough
back
so
yeah.
That
is
good
so
anyway.
A
A
A
You
know
my
production
cluster
has
been
up
my
production,
I've
got
a
production,
ocp
production,
okd
and
then
a
testing
okd
and
rebuilding
those
and,
having
you
know,
disaster
recovery,
essentially
to
get
them
up
quickly.
I
wanted
something
that
simplified
the
process,
and
this
is
the
actual
script
code
here
you
know
and
if
there's
any
features
that
folks
would
like
to
see
happy
to
write
those
in-
and
you
know,
there's
a
lot
of
folks
doing
upi.
A
Yes,
yeah
it'll
work
for
both
there's
no
difference.
The
only
thing
is
that
for
ocp
to
get
that
support,
you
want
to
have
the
pull
secret.
So
in
the
code
let
me-
and
I
sorry
sorry
for
like
flipping
through
really
fast.
I
hope
I'm
not
making
people
dizzy.
But
if
you
look
in
my
code,
where
is
the
pre-run
right?
So
basically,
if
you
don't
add,
if
you
don't
say
to
use
the
dummy,
pull
secret
or
auto
pull
secret,
as
I
call
it,
it
actually
says.
A
A
Add
another
else
to
this:
where
it'll
read,
if
in
your
config
file,
if
there
is
a
pull
secret
already
there
and
if
it
is,
then
it
won't
it'll
just
use
that
and
duplicate
it.
Pull
secrets
are
good
for
was
it
just
24
hours
or
48
hours?
I
can't
remember,
I
think
it's
24
hours
of
the
pull
secrets
are
good
for
it.
I
don't,
I
think
they
expire,
there's
a
certificate
on
the
other
side
that
expires.
B
A
Sure
so,
with
the
reason
that
I'm
not
using
ipi
is
because
we
wanted
to
have
the
f5
there's
a
couple
reasons
we
wanted
to
have
the
f5
as
front
facing
for
the
cluster,
because
we
could
then
route
requests
through
that
load,
balancing
and
have
things
such
as
if
the
cluster
in
its
entirety
goes
down
still
have
monitoring.
Notifications
still
have
redirecting
those
urls
to
outage
pages
and
whatnot.
A
So
it
not
relying
on
everything
being
internal
to
v
sphere
was
the
way
that
we
wanted
to
go
right
now
and
also
at
the
time
it
wasn't
clearly
document.
When
I
first
started
this,
it
wasn't
clearly
documented
how
to
set
your
subnet
for
your
cluster
and
whatnot
they've
added
a
lot
of
functionality
and
a
lot
of
documentation
recently
about
setting
your
subnets
for
the
cluster
and
for
the
pod
network
and
stuff,
like
that.
A
lot
of
that
wasn't
clearly
defined
early
on
in
the
four
releases.
A
Okay-
and
it's
done
so
now-
it
completed
in
17
minutes
so
now
we'll
see
the
workers
are
now
booting
up
and
if
we
go
to
here
and
go
check,
there's
no
certificates.
No,
not
yet.
So
we
see
now
that
the
masters
are
ready
and
they're
running
the
most
recent
version
of
of
4
6,
ok,
d46
and
and
fedora
core
os,
and
now
the
worker
nodes
are
going
to
come
up.
That's
one
thing
I
didn't
mention
is
so
in
your
call
to
the
script.
A
When
you
put
the
release
version,
you
can
do
the
whole
release
version
like
this
like
copy
that
string
for
a
very
particular
release
version
like
from
the
nightlys
or
whatever,
or
you
can
just
put
a
major
minor
string
like
that,
like
four
six
or
four
seven,
and
what
you'll
get
back
is
the
most
recent
version
for
that
major
miner.
That's.
B
A
The
rest
calls
to
an
api
address,
api
dot,
cluster
name,
dot
domain
name
and
those
mapped
to
a
pool,
whether
you
do
it
upi
or
ipi,
that
domain
name
maps
to
a
pool,
load,
balancing
pool,
and
that
can
be
a
any
subnet.
They
don't
necessarily.
I
don't
think
that
you
necessarily
have
to
have
the
same
subnets
for
in
terms
of
ip
numbers
for
those,
but
you
want
different
pools
because
requests
to
the
api
address.
A
The
other
subnet
is
the
actual
subnet
that
the
pods
are
running
on
and
the
pool
for
that
to
connect
to
that
is
another
address,
and
that's
the
apps
dot
cluster
name,
dot
rest
of
the
domain
name,
and
that
is
anytime.
You
spin
up
a
pod.
It's
gonna
have
the
the
or
spin
up
an
application.
The
application
is
gonna,
be
have
a
route
that
is
application,
name
or
whatever
dot,
apps
dot,
cluster
domain
name,
and
you
can
do
mapping
to
that.
A
C
A
A
Okay,
so
it's
pools
basically
there's
two
different
pools
and
those
may
or
may
not
necessarily
map
to
different
ip
subnets,
but
it's
different
pools
for
different
tasks.
Okay,
so
now
this
is
up
and
there
we
go.
We
just
approved
the
csrs
for
the
two
worker
nodes
and
here's
the
other
one.
There
we
go
and
then
there's
another
one,
another
set
of
certificates,
that'll
pop
up
and
there
they
are
again-
and
I
just
approved
them
so
now.
A
If
we
do
get
nodes
in
a
couple
of
seconds,
usually
in
about
a
minute
the
worker
nodes
are
then
there
they
are
so
with
yeah.
So
with
a
with
the
script-
and
you
know
in
just
a
couple
of
of
clicks,
if
you
wait
patiently
to
approve
the
csrs
within
an
hour,
you
can
have
a
an
open,
an
okd
cluster
running,
and
so
let's
do
get
notes
up
still,
not
quite
yet
any
questions
about
any
of
this.
B
The
thing
is
that
you
can
customize
lots
of
of
everything
with
upi
yeah.
You
can
export
all
yaml
manifests
right
with
the
installer.
You
can
patch
everything
and
create,
with
the
installer
from
the
patched
manifests
as
ignition
files,
so
that
your
cluster
right
from
the
start
is
completely
pre-configured
and
that's,
I
think
it's
it's
not
possible
with
ipi.
A
Right
and
oh
here
we
go,
we've
got
one
ready
and
I
bet
when
this
returns,
the
other
one
will
be
ready
and
we
will
have
a
working
cluster.
Now,
it's
going
to
take
some
time.
If
you
do
oc,
get
co
you'll
see
that
the
machine
operators
are
not
done
yet
there's
still
one
that
needs
to
finish
here
and
ingress
always
needs
to
finish
and
monitoring.
B
A
So
if
we
go
well,
here's
an
exam,
here's
a
brief
one.
So
if
we
actually
go
to
the
code
of
oh
yeah
right
in
pre-run,
so
here's
here's
an
example
of
some
things
happening
in
terms
of
manifest.
So
there's
a
manifest
that
gets
created
when
you
run
the
gen
the
create
manifest
call
from
the
opening
a
little
bit.
It's
a
little
bit.
D
B
A
B
A
A
B
A
A
I
thought
it
was
this
one
of
like
creating
all
sorts
of
modifications
to
your
config
files
to
do
things
on
install
like
what
was
what
was
the
example
that
they
give
oh
setting
your
ntp
server,
so
there's
an
example
of
using
a
config
to
set
the
ntp
server
on
the
nodes
when
they
boot
up,
and
things
like
that.
So
it's
it's
there's
a
lot
of
examples
in
the
documentation
for
that.
A
A
B
B
B
B
B
B
Yeah,
no
everything
is
fine.
I
can
upgrade
on
okd
4.5
some
repositories
on
the
fedora
code
core
as
nodes
were
enabled,
and
if
you
upgrade
during
the
upgrade
process,
then
it
if
the
repos
are
enabled
it
tries
to
pull
images
packages.
Sorry
from
the
internet,
which
sometimes
could
be
a
problem,
so
you
have
to
disable
them
before
you
upgrade.
B
B
B
B
B
B
B
In
this
situation
you
will
see
that
always
one
master
and
one
worker
get
are
getting
unscheduled
and
the
nodes
were
automatically
drained
and
then
new
and
uos
version
is
applied
and
the
system
automatically
boots
into
the
new
fedora
cores
version.
In
this
case
I
have
fedora
core
is
32
released
last
year
in
june
and
after
the
nodes
are
upgraded,
we
will
see
that
we
have
fedora
cos
version
33
from
I
think
january
as
this
year,
and
you
have
to
do
nothing
it's
running
completely
on
its
own.
B
B
Okay,
it's
it's
already
over.
It's
upgrading!
It's
installing
a
new
version
of
etcd
and
automatically
does
the
migration
from
the
old
etcd
to
the
new
version.
You
can
have
a
look
yeah
if
it
if
it
really,
if
you
watch
it
like.
I
do
here
said
always.
If
something
pops
up,
you
will
see
all
the
pots
that
are
spawned
to
do
the
upgrade.
Here
we
see
a
new
etcd,
some
quorum
guards
are
created.
B
B
B
B
Also,
if
you
delete
an
etcd
port,
you
will
see
that
it's
upgrading
yeah,
because
the
operator
always
tries
to
get
to
its
desired
state
and
if,
even
if,
it's
not
upgrading
but
only
getting
to
its
desired
state,
you
will
see
that
it's
getting
getting
a
new
version
yeah.
Now
we
have
three
nodes.
Sorry.
B
It
will
create
vms
with
the
new,
a
good
question.
I
have
to
think
about
that.
I
think
it
will
if
at
first
it
will
create
a
version
with
the
older
os
version,
and
if
this
one
is
created
and
has
joined
the
cluster,
it
will
do
automatically
an
upgrade
to
the
target
os
version.
So,
in
the
end,
all
nodes
have
the
same
version,
regardless
of
where
you
started.
A
So
are
you,
let
me
ask
this:
are
you
asking
if
new
nodes,
so
a
machine
set
can
be
used?
A
machine
set
is
used
at
two
different
times.
It's
used
for
the
initial
building
of
the
cluster
and
then
also
for
adding
nodes
and
applying
to
it.
So
if,
if
you're,
applying
and
adding
new
nodes,
those
new
nodes
will
brought
up
to
be
brought
up
to
the
same
across
the
machine
set
and
it
would
be
new
vms.
So
that's.
B
A
As
I
and
jesper
says,
as
I
understand
for
ipi,
then
upgrades
are
when
upgrades
happen
by
creating
new
vms
and
if
they
join
successfully,
the
old
vm
will
evict
containers
and
then
it
will
spin
up
containers
in
the
new
vm
and
continue
good
question.
I
I
don't
think
so.
I
think
for
ipi
it's
it's
the
same
that
so,
if
you're
updating
the
os,
it's
rebooting
that
same
vm
with
the
new
os
and
any
new
changes
from
the
machine
config.
It's
it's
not
spinning
up
a
new
vm.
I
don't
think
in
ipv.
B
B
D
A
A
A
These
manifests
that'll
then
get
put
into
the
nodes
for
things
such
as
disk
encryption
for
crony
configuring,
crony
to
use
a
particular
ncp
server
and
anything
that
you
can
think
of.
So
this
is
actually
factors
into
why
I
was
encouraging
folks
to
familiarize
themselves
with
fedora
core
os
fedora
core
os.
A
Does
everything
through
manifests-
or
I
should
say
everything
through
ignition
config,
and
so
you,
you
create
a
yaml
file,
that's
similar
to
a
manifest,
and
you
put
it
through
a
tool
that
they
have
and
then
it
creates
the
json
based
ignition
config,
and
then
you
apply
that
to
a
fedora
core
os
node
and
all
of
these
things
that
we're
talking
about
and
a
lot
of
stuff
of
the
os
can
actually
be
configured
through
these.
So
if
we
go
down
here,
this
is
like
pre-creating,
the
ignition
configs,
here's
the
manifest.
A
So
if
you
go
to
manifests,
these
are
the
manifests
that
go
into
creating
the
ignition
config
and
an
example
of
a
modification.
Is
I
found
that
my
nodes
were
timing
out?
My
worker
notes
were
timing
out
when
trying
to
connect
to
the
control
plane
when
they
would
first
boot
up,
okay
and
and-
and
I
think
we
too-
we
talked
about
this
a
little
bit
in
the
channel.
So
what
I
found
is,
if
you
modify
in
the
master
and
worker
ign's
a
time-out
value
yeah,
you
do
that.
A
Okay,
nice,
let's
see
which
one
is
it
it's
in
the
open
shift,
one
there
we
go.
Oh,
no!
Actually
it's
in
it's!
I
have
to.
Actually
you
can't
do
it
in
the
manifest
you
have
to
do
it
actually
in
the
in
the
ignition
that
gets
generated
after
I'm,
not
sure
why,
but
basically
you
can
modify
the
timeouts
that
the
nodes
have
when
trying
to
connect
to
machine
config
from
the
api
server.
B
D
B
Okay,
so
do
you
see
my
screen?
Yes,
okay,
so
here
we
have
one
of
the
coolest
features
that
has
to
do
with
core
os
this
other
machine
configs
under
compute
machine
configs.
You
find
yeah
some
manifests
that
are
telling
openshift
or
okd
what
you
want
to
configure
on
your
host
yeah
formally.
If
you
wanted
to
install
something
or
write
a
file
or
change
your
file,
you
had
to
ssh
into
the
node
on
ok
d3.
It
was
very
common
to
use
ansible
to
change
something
on
all
nodes.
B
You
don't
have
to
do
that
anymore.
With
a
fedora
or
red
hat
core
os.
You
can
almost
change
everything
with
machine
config
manifest.
Let
me
show
you
an
example
am
searching
some
random
yeah
here
we
have
an
example.
Let's
fold
that
away
this
is
a
machine
config,
it's
applying
configuration
to
all
workers
like
you
can
set
the
role
here.
B
B
B
Is
yeah
it's
some
yeah.
I
don't
know
what
this
is.
It's
a
content
of
a
file
and
if
you
apply
that
machine
config
here,
it's
only
a
short
ignition
snippet,
you
can
can
write
everywhere
on
the
host,
where
you
have
read,
write
permissions
normally,
I
think
etc,
and
the
other
one
was
the
var
I
think
the
var
folder
you
can
write
into
it
with
this
method
and
was
and
what
happens
then,
if
you
apply,
that
is
that
you
get.
I
will
sort
the
date
here.
B
You
will
get
a
new
render
file
yeah.
We
have
here
a
rendered
worker
and
a
rendered
master
file,
and
I
always
sort
it
to
the
date
because
say
every
time
you
apply
something
here,
you
will
get
a
new
rendered
file.
This
files
contain
all
configurations,
the
base
configurations
and
the
ones
here,
the
additional
ones
they
are
merged
together
in
one
big
ignition
file.
Let's
have
a
look
into
it
here.
You
see,
we
have
c
ssh
some
ssh
files.
B
B
You
see
lots
of
services.
What
is
this?
This
is
a
hypercube
service
and
you
can
change
what
new
neos
will
see
by
applying
machine
config
objects.
One
thing
we
have
done
in
my
company
is:
we
wanted.
We
have
an
an
aircapped
cluster
and
we
were
not
wanted
to
pull
the
images
not
from
the
internet
but
from
our
internal
registry,
our
internal
registry.
B
B
I
will
open
a
terminal
on
the
host.
Normally
you
never
have
to
go
to
with
ssh
to
your
nodes.
You
can
use
this
mechanism
here,
it's
upgrading,
so
I
have
a
terminal.
Now
now
I'm
gonna,
I
will
enter
a
change
root
command
and
now
I'm
on
the
host
yeah,
I'm
not
using
ssh
to
go
to
the
my
my
master,
zero,
I'm
using
this
method
here,
I'm
now
on
the
host
oops.
It's
upgrading
that
example.
B
B
You
can
elf
pokemon
2
at
always,
if
you,
if
someone
wants
to
pull
an
image
from
quay
or
from
docker,
io
or
whatever,
that
it
in
in
in
fact
should
go
to
a
different
registry,
and
this
process
is
completely
transparent,
yeah.
So
in
this
this
way,
with
the
spotman
file
configuration
file,
you
can
tell
portman
to
in
fact
pull
images
from
a
completely
different
registry
than
the
one
that
was
addressed
in
the
in
the
manifest,
and
we
write
this
file
with
machine
configs.
B
B
B
It's
a
kind
of
priority
prioritization
you
can
do
here
and
we
have
done
lots
lots
of
configurations
this
way
on
the
host
and
we
absolutely
not
use
ansible
for
changing
existing
clusters.
You
we
not
even
have
to
go
to
nodes
with
ssh,
because
you
know
normally,
you
don't
have
to
do
that.
If
you
want
to
change
your
cubelet
parameters,
I
think
you
even
have
an
configuration
object
for
that.
B
B
B
B
C
So
from
the
the
folks
who
are
listening
in,
I
I
hope
this
has
been.
This
is
still
really
helpful.
It's
I
think
it's
great
that
we've
got
these
two
guys
here
doing
this.
Are
there
other
things
that
you
would
like
to
see
us
document
better
other?
Have
they
gotten
it?
I
can
see.
Some
of
the
questions
in
the
chat
are
those
things
that
we
should
be
updating
in
the
documentation
guide.
B
A
C
C
C
C
C
A
C
B
B
But
more
a
work
around
for
a
buck.
You
don't,
if
I
don't
know
what
you
mean,
newish
quay
feature
for
transferring
images
yeah.
No,
it's
jasper.
I
think
what
you
mean
if
I'm
correct,
if
I
understand
it
correct,
is
that
series
of
there
is
a
bug
that
affected
one
of
the
core
libraries
that
have
to
do
with
docker
images
and
as
this
library
is
used
yeah,
this
library
is
used
by
lots
of
tools
by
scopio
by
podman
by
whatever
and
because
of
this
bug.
B
B
A
C
C
Somebody
wanted
to
do
a
complete
live
deploy
somewhere
on
the
hardware,
so
you're
not
obliged
to
stay
another
five
hours,
because
I
know
you
can
see
the
timer
at
the
top
in
the
back
room,
but
if
you're,
if
you
guys
don't
have
any
more
questions
and
anyone
else
next
up
we're
going
to
try
and
do
these
things
quarterly.
I
think
that
it
feels
good
feels
right
to
do
this
so
larry
your
team,
just
this
is
a
chess
game.
C
Your
checkmate
here,
your
team,
is
on
the
hook
for
doing
some
sort
of
deployment
demo
next
up
and
jesper
will
will
get
you
in
on
a
saturday
since
saturday
is
better
for
you.
I
don't
know
what
time
it
is
where
you
are.
I
think
you're
in
finland
or
someplace
like
that,
so
we'll
we'll
make
you
guys
do
some
of
the
demos
too
and
maybe
have
fireside
chats
as
well.
So
I
wanted
to
really
thank
both
joseph
and
jamie
jamie,
especially
for
getting
in
the
background
and
helping
with
organizing
stuff
this
week.