►
From YouTube: Antrea LIVE: Episode 10 (Tanzu in the wild: production networking topologies and templates)
Description
Come join Scott (VRabbi) to dig into real world customer scenarios around antrea and Tanzu, and his legendary TKGM-customizations suite !
- Antrea with Tanzu 1.5, and NSX-T integration plans
- Egg nog
- vrabbi/tkgm-customizations
- packaging... everything into carvel on customer sites
A
A
C
C
Exactly
you
know,
no,
so
we're
we're
a
services,
we're
a
service
integration
or
a
systems;
integration
company
based
in
israel,
but
also
working
in
the
states
and
in
europe.
We're
partners
of
vmware,
amazon,
google
and
a
bunch
of
others
and
really
working
hard
in
the
cloud
native
world,
public
cloud
and
on-premise,
and
I'm
the
practice
leader
for
cloud
and
automation
at
terra
sky
and
leading
the
tanzu
portfolio
from
our
side
as
well.
A
A
A
So
scott
is
collaborating,
collaborating
with
us
upstream
on
this,
so
this
is
going
to
be
a
cool
one
and
so
like
I
just
want
to
say.
One
of
the
things
I'm
really
excited
about
is
the
fact
that
we're
tanzu
is
now
we're.
Actually,
you
know
at
vmware
we're
developing
tanzi
with
people
in
the
community
like
scott,
which
is
really
cool
like
we're
doing
that
for
windows,
but
I
think
more
of
that's
going
to
be
happening
so,
like.
B
B
A
C
A
This
flag
is
irrelevant,
so
you
don't
have
this
weird,
and
so
scott
showed
me
this
yesterday
and
I
was
like
this
is
really
cool.
I
thought
it
was
really
cool
in
two
ways.
One
somebody
outside
of
vmware
was
showing
me
how
to
a
new
feature
in
danza
that
I
didn't
fully
understand
yet
so
I
was
like
wow.
A
This
is
really
cool,
but
then
the
other
thing
that
was
cool
is
that
this
is
genuinely
cool,
because
you've
seen
this
in
upstream,
where
there's
feature
gates
and
people
get
really
confused
about
when
to
enable
them
and
when
to
disable
them,
and
there
are
these
constants
that
sort
of
sort
of
like
follow
you
around
forever,
and
so
this
is
a
not
a
really
nice
nice
convention
that
that
folks
came
up
with.
I
don't
know
who
came
up
with
this
convention,
but
shimon
implemented
it.
A
A
I
don't
think
a
lot
of
people
know
about
that.
It
was
pretty
intuitive
to
me
because
at
one
point
someone
on
our
team
actually
had
to
do
that,
but
but
yeah
that's
real
interesting,
maybe
once
you
when,
maybe
once
you
fire
up
your
your
terminal,
you
can
just
show
us
that
really
quickly
for
sure
and
then
you've
got
all
these
links
in
here
and
I
I
feel
like
you've
got
so
much
stuff
going
on
that.
Maybe
we
should
just
jump
into
it
right,
so
yeah
yeah.
Why
not?
Why
don't
you
just.
A
A
A
B
C
A
Gotta
interrupt
you
really
quick
and
say
hi
to
people.
Mourinho
is
here,
he's
my
friend
from
vmware,
but
he
left
mourinho.
I
just
I
just
got
back
from
the
gym.
You
can
be
really
proud
of
me.
Pera
is
here
and
he's
one
of
our
new
kaping
friends
miss
you
too
mourinho
you'll
be
really
proud.
You
you'd
be
really
proud.
I
I
did
like
a
two
and
a
half
hour
workout
today
and
eight
and
a
24-hour
fast
okay,
robert
clisterius.
Where
are
you
from
robert
you're
new.
A
A
C
C
Right,
the
general
idea,
I
think,
is
we're
just
going
to
kind
of
hack
around
and
see
you
know
how
we
customize
tanzu
and
like
some
of
the
stuff,
especially
around
networking
that
we've
been
doing
and
to
go
to
the
point
that
jay
had
mentioned.
You
know
about
pausing
reconciliation
of
apps
and
all
of
that
that
actually
came
in
use
here
for
this
demo.
C
Basically,
where
I
created
a
cluster
that
the
new
integration
that
happens
between
andrea
and
nsxt
actually
requires
a
higher
version
of
antrio
than
comes
in
the
current
released
version
of
tanzu
okay,
so
I
actually
paused
the
reconciliation
of
andrea
and
then
installed
my
own,
the
upstream
version
of
1.3
of
andrea
into
this
cluster.
So
if
we
did.
C
And
just
took
a
look
one
of
the
cool
things
we
actually
deployed
here.
The
interworking
operator
is
how
this
works.
So
it's
a
basically
a
controller
that
runs
within
your
cluster
that
you
deploy
after
having
andrea
that
registers
it
against
tanzu
or
registers
it
against
nsx
and
then
once
it's
registered,
it
actually
becomes
really
cool.
Because
if
I
go
to
the
inventory
now
of
just
my
standard
nsx
within
the
containers,
I
get
all
my
name
spaces.
I
could
see
any
pods
that
exist
here
and
all
their
labels.
C
A
C
As
we
know,
naming
is
the
hardest
part
about
any
project
in
the
world
yeah
so
but
in
general,
and
the
really
cool
thing
here
is
that
we
get
all
of
our
pods
and
all
of
our
data
and
services.
Pods
everything
come
up
into
nsx
here
and
then
from
this
point
we
can
actually
fully
manage
our
clusters.
So
I
only
have
one
connected
right
now,
but
if
I
went
just
for
example,
into
like
the
yelp
namespace,
we
can
see
right
now.
C
C
C
A
C
What
is
really
strong
is
actually
that
we
can
go
here
and
in
nsx
we
have
this
idea
of
a
distributed
firewall
and
a
distributed.
Firewall
is
basically
we
use
it
for
whatever
we
need,
we
can,
you
know,
create
different
firewall
rules
so
for
like
proxy
networks
or
default
layer,
3.,
and
so
one
of
the
cool
things
that
till
now
it's
only
been
distributed
firewall.
C
A
One
second:
I
got
a
robert
roberts
making
a
pet
store
joke.
So
robert
I'll
tell
you
something,
you
know
the
original
k-8s
pet
store
app
in
kubernetes.
I
was
the
one
that
wrote
that
it
was
because
I
think
pet
clinic
is
the
best
demo
app
ever
and
I
don't
think
so,
and
then
we
made
a
synthetic
data,
generator
called
the
big
pet
store
data
generator
and
it
used
markov
models
to
simulate
pet
store
transactions
where
people
would
walk
within.
A
C
A
C
C
C
C
A
C
Like
the
cluster
network
policies
and
actually
showed
this
one,
what
we'll
see
is
they're
actually
using
android
groups
in
the
applied
to
and
from
where
yeah.
So,
if
I
did
a
cube,
cto
get,
what
are
they
called?
I
think
cluster
groups
all
right,
yeah,
cluster
groups.crd
entry,
io
and
went
and
looked
at
this
one.
A
C
C
Exactly
automation
for
free,
so
basically
it
requires
to
use
a
certificate
based
principle
identity
to
talk
to
nsx.
So
what
we
do
here,
basically,
we
just
generate
with
openssl
some
certificates
and
then
run
an
api
call
against
nsx
to
create
a
principal
identity
and
then
base64,
encode
them
out
and
just
print
out.
Basically,
hey
just
add
this
to
your
cluster
config
deploy
it,
and
this
will
deploy
upstream,
andrea
1.3
and
then
we'll
install
the
interworking
controller
yeah
automatically
into
your
environment.
C
For
you
as
well
so
yeah
out
of
the
box,
you
can
get
it!
You
just
have
to
run
this
script
before
and
then
it
automatically
connects
up
to
nsx,
which
is
actually
a
really
cool
thing
to
do
and
yeah.
So
I
just
generated
this
today.
I've
created
about
seven
clusters
this
way
and
it
works
well
and
no
ariel.
I
do
not
use
microsoft
word
as
an
ide.
C
C
C
Right,
yeah
no
sign,
I
think,
like
this,
andrea
nsx
is
actually
a
really
cool
integration.
I
think
it
shows
you
know
a
real
change
in
the
cni
world,
where
we'll
be
able
to
get
really
true
good
integration
between
our
legacy
apps
and
what
exists
in
the
container
world
and
bringing
the
two
together
here
is
actually
a
really
nice
capability
where
they
also
added
the
last
piece
of
the
integration
is
actually
that
we
can
run
trace
flows
with
andrea
directly
through
the
ui.
C
So
this
is
connected
to
all
of
your
clusters
that
are
running
andrea
and
you
can
now
go
and
say:
okay,
I
want
to
do
this
from
the
yellow,
namespace,
let's
say
ui
to
the
app
server,
and
I
can
trace
this
right
now,
and
this
is
going
to
tell
me
exactly
if
it
passed
or
not
and
if
I
actually
go
and
edit
this
and
change
this
to
port.
Let's
say
443,
which
shouldn't
be
open.
C
A
C
C
It's
just
the
two
uuids
at
the
end
that
are
always
a
mess,
and
here
we
just
get
the
full
drop
down
list
and
you
could
do
like
by
node,
because
it
has
all
this
data
and
then
no
matter
which
name
space,
so
it
just
becomes
really
easy
to
play
around
with
here
and
actually
just
get
our
trace
flows.
So
I
think
this
is
a
really
cool
cool
idea
that
they
have
this
year,
that
you
know
I'm.
A
A
C
C
Yeah,
so
I'm
like
calico
enterprise
exists
out
there.
There
are
some
stuff,
but
I
I
think
that
in
the
end
you
know,
the
really
nice
thing
here
is
that
it's
integrating
into
a
system
that
already
exists
in
a
lot
of
enterprises
like
calico
enterprise,
is
a
great
ui,
but
it
works
for
its
applications.
It
works
for
calico,
it's
its
own
ecosystem
versus
here,
we're
we're
really
integrating
with
a
whole
separate
stack
but
getting
a
unified
single
plane
of
glass
kind
of
yeah,
which
is
cool
but
anyways.
A
C
Yeah,
so
in
general
I
once
the
packaging
apis
came
out
for
carvel.
I
was
actually
just
really
interested
in
them
and
decided.
I
would
start
playing
around
and
one
of
the
things
that
was
getting
difficult.
As
a
you
know,
system
integrator
was
when
you
know
you
come
to
a
customer
they're
new
to
kubernetes
everyone's
heard
of
helm
and
now
all
of
the
installations
that
happen.
You
say
great
for
the
tanzu
packages.
C
So
I
took
just
a
little
side
project
and
I
built
a
tool
actually
that
I
linked
here,
which
is
a
helm
to
carvel
conversion
tool
that,
basically,
you
can
give
it
any
helm
repository
and
it
will
convert
them
into
carvel
packages,
including
air
gap,
support,
because
one
of
the
things
that
carvel
is
really
strong
with
is
that
it
can
do
full
air
gapping.
It
can
pull
down
all
of
them.
A
C
Exactly
so,
if
I
like
took
a
look
here,
this
is
just
like
a
command
right.
If
I
run
helm
to
package,
which
is
the
you
know
alias
for
this,
you
know
program
and
then
gave
it
a
repo
name
and
the
repo
url
that
I
want
to
pull
from,
and
I
told
it
yeah
the
number
of
chart
versions.
I
want
to
get
the
last
two
chart
versions.
A
C
C
Yeah,
you
can
do
it
like
this,
if
you
just
do
the
arrow
pointing
towards
the
source
and
then
if
we
did
a
tanzu
package,
let's
say
available
list
right
now.
One
of
the
things
that
we'll
be
able
to
see
is
that
in
this
cluster,
I've
got
all
of
these
different.
You
know
packages
that
are
available
to
me,
so
some
of
these
come
out
of
the
box
from
vmware,
and
some
of
them
are
custom
yeah.
C
A
C
A
C
A
C
A
C
Just
this
one
command
that
I'm
running
right
now
right.
So
what
this
is
going
to
do
is
this
is
actually
taking
the
bitnami
helm
charts.
I
gave
it
a
list
of
charts
because
I
don't
want
the
whole
thing
just
because
this
is
a
demo,
but
if
you
don't
pass
in
a
chartless
file
path,
it's
just
the
entire
helm
repository
and
what
this
is
actually
doing.
C
So
when
it
pulls
up
it's
going
to
pull
in
we'll
see
in
a
second,
we
can
see
it's
actually
using
k,
build
in
the
back
end
from
carvel
as
well
to
basically
come
and
understand
all
the
image
references
and
a
make
them
out
into
sha's.
But
once
I
go
and
copy
this
image
package,
which
is
the
package
repository
to
any
image
registry
in
the
world,
it
will
actually
import
all
of
the
container
images
as
well
as
all
the
manifests
in
a
single
repository
and
change.
C
All
of
the
references
for
me
at
deployment
time
to
point
to
my
local
registry.
So
this
becomes
really
a
strong
capability
and
we
can
see
this
is
just
pulling
down
everything
right
now.
We've
already
done
apache,
it's
doing
cube
apps
now
and
the
nice
thing
with
cubex
is:
it's
actually
got
two
different
levels
of
nested,
helm,
charts
and
this
tool
goes
through
all
of
the
nested
charts
as
well
and
pulls
them
all
in
so
so
now.
A
C
A
B
C
C
I
so
managing
them
is
actually
not
that
difficult.
It's
actually
really
easy.
You
know
in
the
end,
it
can
be
as
easy
or
as
complex
as
you
want
it
to
be,
if
it's
just
converting
over
helm,
charts
and
then
hosting
them
locally.
It's
as
easy
as
running
this
tool,
and
just
you
know
pushing
it
into
your
registry
and
you're
good.
C
If
you
want
to
write
your
own
custom
automation,
so
you
want
to
do
things
like
ytt
and
k
build,
and
all
of
that
once
you
have
those
manifests,
it's
a
matter
of
a
few
seconds
and
you
can
make
that
into
a
package
repository.
So
it's
really
easy
once
you
decide
what
you
want
to
build,
but
if
you
have
the
base
templates,
whether
that
be
ytt
or
helm,
charts,
it's
a
really
easy
process,
and
I'm
actually
in
the
middle
of
writing
some
documentation
on
that
that'll.
Make
it
much
easier.
A
A
C
A
C
C
C
C
We
can
actually
see
now
that
I've
got
two
versions
of
it.
I've
got
1226
and
1227,
which
are
actually
mapped
to
the
versions
of
the
helm
chart
itself
and
if
I
went
through
the
kansu
command
and
did
it
get
to
this
slash,
the
version
and
said
hey
give
me
the
schema
of
what
values
I
could
pass
into
this
because
in
helm
I've
got
a
values,
file
yeah.
How
can
I
know
what
values
to
pass
in?
A
A
C
C
Exactly
so,
it
originally
had
been
here
within
my
own
repository,
and
then
I
released
it
under
the
company
and
the
old
version
that
was
here
is
how
I
got
you
know
whatever
there
were
a
bunch
of
stars
because
of
this,
and
then
I
just
decided
that
it
deserved
to
be
on
its
own.
So
I
separated
it
out
from
the
repo
but
yeah.
C
We're
just
starting
also
so
there's
going
to
be
a
lot
more
coming
soon,
but
yeah,
and
so
that's
like
really
carvel
packages
right.
But
the
other
cool
thing
when
it
comes
to
carvel
packages
is
they're
used
in
tanzu,
and
sometimes
they
aren't
necessarily
exactly
what
we
need
right.
Sometimes
we
may
need
some
more
knobs
and
are
exposed
to
us
and
things
like
that
and
using
carvel.
We
can
actually
also
extend
that
using
like
overlays.
C
So
I
also
have
within
this
repo
this
tkg
extension
modifications
where
just
as
an
example
here
you
know,
prometheus
that
comes
in
tanzu
doesn't
have
tanos
within
it
and
that's
great
for
a
lot
of
environments.
But
we
had
a
specific
use
case
by
a
customer
that
wanted
to
ship
all
their
prometheuses
to
thanos
for
long-term
storage
and
then
from
there
up
to
grafana.
C
So
basically,
what
we
did
is
created
just
a
simple
overlay
file
to
add
thanos
into
the
package
from
prometheus,
and
then
we
actually
can.
We
just
wrote
a
simple
install
script
that
installs
the
package
and
then
it
just
annotates
that
package
with
the
secret
that
the
overlay
is
in
and
then
it
will
automatically
reconcile
that
package
and
add
the
thanos
sidecar
right.
So
it's
small
things
like
that
or
even
like
in
the
extensions
harbor
that
comes
in
tanzu,
doesn't
come
with
chart
museum.
C
A
A
C
Right
so
a
good
example
of
that
is
actually
cap.
Controller
yeah,
because
cap
controller
is
really
growing
a
lot.
It's
growing
very
fast,
and
it's
very
hard
for
downstream
projects
to
always
stay
up
to
date.
With
these
things
and
one
of
the
difficult
things
is
like
certain
projects
within
vmware,
so
like
controller.
A
Is
basically
a
canned
operator
right
like
it
allows
you
to
basically
build
an
operator
for
anything
that
does
most
of
the
things
that
an
operator
might
do,
or
a
lot
of
the
things
that
an
operator
would
do
without
you
having
to
write
anything
exactly
right?
It's
awesome!
That's
how
all
the
packages
are
working.
We've
gone.
C
So,
basically,
what
you
can
do
in
that
case
is,
if
you
go
to
your
management
cluster,
and
if
you
look
here
in
cube,
ctl
get
app
in
the
namespace
of
your
cluster.
You
can
see
that
you
actually
get
one,
which
is
the
name
of
the
cluster
minuscap
controller,
and
if
I
actually
went
and
took
this
cluster,
let's
say
and
edited
this
app
one
of
the
things
that
I
can
do
on
any
app
cr
is
just
under
spec.
I
can
just
add
paused
true
and
once
I've
added
that.
C
Once
I've
done
that
now,
cap
controller
will
not
try
and
reconcile
my
application
anymore
and
now
what
I
could
do
is
if
I
go
into
that
cluster.
If
I
did
a
cube,
cpx
back
to
andrea,
live
cls02,
I
could
just
do
okay
get.
You
know.
Minus
n
tkg
system
and
I
can
just
go
and
a
delete
deployment,
minus
ntkg
system,
cap
controller
and
cap
is
deleted
and
if
I
went
to.
C
A
A
C
We
can,
you
know,
pause
applications,
but
one
of
the
things
that
we've
experienced
from
a
lot
of
our
customers.
They
say
when
I
deploy
a
cluster
on
one
hand.
I
want
to
give
self-service
to
my
end
user
right.
I
want
the
developer
to
be
able
to
say,
hey,
give
me
a
cluster,
but
when
a
cluster
gets
deployed
we
want
it
to
have
certain
things
already
installed
in
it.
We
want
to
make
sure
that
it
has
prometheus
and
grafana.
C
So
one
of
the
things
that
I
added
into
here
is
actually
this
installation
directory,
and
these
are
just
some
ytt
files
that
if
we
look
at
the
readme
it'll
be
easier
to
see
here,
but
if
we
actually
looked
at
like
any
of
these
things,
let's
say
you
know:
if
we
look
at
this,
basically
all
you
need
to
do
is
copy
these
files
into
you
know
this
folder.
I
should
update
that
dot.
A
A
C
Dot
config
yeah
exactly
and
if
we
look
here,
we've
added
all
these
new
data
values
now
so
enable
can
zoo
software
auto
install.
Do
we
want
it
to
install
cert
manager,
contour
external
dns,
prometheus
grafana,
fluent
bit,
kanzu
build
service,
tanzu
postgres,
tenzu,
rabbit,
tanzu,
mysql
and
valero.
It
can
install
and
configure
all
of
this
automatically.
For
me,.
A
So
so
robert
has
a
question
so
robert
yeah
he
uses
that
paused
annotation
and
that
paused
annotation
present
prevents
reconciliation
of
those
of
that
local
cap
instance
exactly
and
so
that
local
cap
instance,
even
though
tana's
tanzania
management
cluster
isn't
going
to
say,
like
you
know
that
the
desired
state
of
the
workload
cluster
is
different.
That
local
cap
instance
is
not
going
to
implement
that,
because
it's
going
to
see
that
and
it's
going
to
pause
it.
I
think
that's
how
it
works.
C
C
Yes,
this
is
what
that
did
yeah
this
disabled
cap
in
a
workload
cluster.
So
one
of
the
cool
things
that
people
don't
know
about
an
app
cr-
and
this
is
a
really
strong
capability-
is
that
the
app
cr
is
not
just
local
within
a
cluster,
and
if
you
have
a
secret
with
a
cube
config
in
it
for
another
cluster,
you
can
just
add
and
inspect
this
cluster
cube,
config
secret
ref
and
just
give
it
that
cube
config,
and
it
will
reconcile
this
application
from
the
management
cluster
in
the
workload
cluster
yeah.
A
C
A
C
Put
cap
controller
in
the
workloads
you
put
it
there
so
that,
then
you
can
manage
from
within
the
cluster
and
tree
or
csi
cni
cpi
ako.
All
of
that
stuff
right,
but
you
technically
you
anyone
that
wanted
to
use
this
in
tce,
let's
say
or
wanted
to
use
this
in
their
own
distribution,
could
install
cap
controller
in
like
a
single
management
cluster
and
actually
manage
all
their
workload
clusters
with
cap
right.
That's
the
capability
that
cap
gives
us
so.
A
C
C
C
C
C
A
C
C
B
B
C
A
C
C
C
C
You're
on
the
cap
control?
No,
this!
No!
This
is
the
deployment
of
cap
controller
itself,
but
that
would
be
where
you
you
would
be
able
to
change
that.
No
that's
the
deployment
of
cap
controller
in
the
workload
cluster,
not
in
the
not
the
cap,
controller
app
that
gets
created
in
the
management
cluster.
You
also
have
the
app
itself
right.
A
A
Meanwhile,
you
can
hack
around
and
see
if
this
even
works,
yeah,
I'm
trying
to
see
if
I
can
find
this
yeah,
so
all
right
so
I'll
share
just
for
us
just
for
a
second
okay.
So-
and
we
did
this
into
the
dimitri
show,
as
you
mentioned
right,
but
right
so
in
general,
what
scott's
going
back
and
forth
and
looking
at
the
way
things
work
right
now
is
that
you
have
a
you,
have
a
management
cluster,
and
why
is
myro
like
spamming
me
on
christmas?
A
A
Then
reconciles
the
cap
instances
on
the
workload
cluster,
so
scott
already
talked
about
that
and
then
like
what
scott's
looking
at
right
now
is.
The
idea
of
like
is
there
a
way
that
this
itself,
since
this
is
actually
created
using
a
cluster
resource,
set
right
exactly
when
I
make
a
workload
cluster?
Is
that
also
created
by
a
cluster
resource
set
when
you
create
a?
What,
when
I
create,
is
cap
also
put
in
via
crs
on
the
workload
cluster
or
is
it
put
in
through
cap?
C
C
C
C
A
B
A
Possibly
move
like
we're,
not
100
sure
where
we're
going
to
go
with
this,
but
like
well
right
now.
The
way
it
works
is
that
cap
controllers-
I
mean
it's
okay
for
to
do
cap
controller
on
the
host
port,
that
that
makes
sense.
But
then
the
tricky
thing
is
that
there's
a
weird
thing:
we
do
that's
an
optimization,
which
is
that
we
install
the
cni
as
a
crs
and
then
we
adopt
it
into
cap,
and
that's
that's.
C
A
C
A
C
Why
would
you
ever
want
to
like
rebuild
it
exactly,
but
so
in
the
basically?
What
we
have
here
is
we
actually
had
you
know,
cube
proxy
by
default
is
used
in
ip
tables,
and
the
cube
proxy
with
ip
tables
has
some
performance
limitations.
Let's
say,
and
we
had
a
customer
that
had
very
large
scale
that
wanted
to
run
cube
proxy
in
ipvs
mode
and
cluster
api
actually
doesn't
enable
that
it's
not
the
tanzu.
C
Doesn't
it's
that
cluster
api
doesn't,
and
so
basically,
what
we
built
here
is
a
hack
that
actually
enables
ipvs
as
well,
so
a
customer
can
just
add
enable
ipvs
on
their
cluster
and
they
get
ipvs
mode
instead
of
ip
tables
and
the
way
that
works
is
pretty
ugly,
but
we
basically
run
an
overlay
here
on
the
cube,
adam
config
templates
and
add
in
the
different
mod
pro
commands
in
order
to
you
know
just
enable
everything
that's
needed
for
ipvs,
because
the
default
templates
don't
have
that.
So
you
know.
A
C
Let's
overlay
append
here
also
on
the
cube,
adm
control
plane
and
then
what
we
do
is
we
create
a
file
and
this
file
is
actually
we
just
call
it
temp
generatecubeproxy.sh
and
these
commands
are
run
in
a
section
called
precube
adm
command.
So
all
of
the
files
for
running
cube
adm
are
on
the
cluster,
but
before
it
runs
the
cube,
adm,
init
or
cube
adm
join.
It
runs
these
commands
yeah.
A
C
C
You
need
your
own,
you
need
the
special
cli
really
yeah,
and
so
sometimes
when
you
have
networking
issues
and
that's
why
you're
going
to
debug
it's
kind
of
hard
to
download
the
binary
when
you're
dealing
with
a
host
that
has
networking
issues.
So
I
just
install
it
automatically
on
every
node
when
it
comes
up
and
then
anytime
you
just
ssh
into
that
node
or
open
up
a
you
know,
console
to
that
vm.
You
can
just
run
the
celium
command
automatically
and
this
just
installs
it
onto
all
the
nodes.
A
C
C
Andrea
live
cls02,
so
let's
say
that
I
wanted
to
take
a
look
at
what
exists
in
this
cluster
and
just
do
show
all
conditions
all
right
awesome.
So
now,
oh
and
sorry,
oh,
what
is
it
show
group
members?
That's
what
I
forgot
to
do
awesome
so
once
I
have
this,
this
is
basically
just
a
nice
view
with
intense
cli
that
comes
from
cluster
cuddle.
C
That
gives
me
the
status
of
my
all
of
the
objects
right,
so
we
have
our
cluster
and
then
we
have
a
control,
the
cube,
adm
control,
plane
and
the
machines
that
are
under
that
and
then
the
same
thing
goes
for
workers
right
and
we
get
all
this
stuff,
that's
all
cool!
Now
when
it
comes
to
the
bootstrapping,
we
actually
have
a
few
other
tools,
a
few
other
crs.
C
We
can
actually
take
a
look
at
it
and
if
I
did
a
cube,
cto
get
secret
and
let's
see
here,
andrea,
yep
and
that
gets
pulled
down
and
then
there's
and
if
I
looked
here
right,
if
we
wanted
to
look
at
just
what
this
looks
like
right.
Just
as
the
example
and
I
said,
hey
control,
plane
right.
I
want
to
look
at
the
control
plane,
node.
A
A
C
A
C
C
We
get
this
cloud
config
and
basically,
what
happens
is
it
adds
all
these
right
file
commands
that
get
run
for
us,
and
you
know
this
where
we're
creating
the
qubit
manifest
and
all
the
certificates
are
being
passed
in
we're
also
creating
that
cube,
adm,
yaml
file
and
all
the
files
that
we
really
need
right
and
we
have
that
in
it
configuration
and
what
happens
automatically
after
those
files
are
created
in
the
system.
Nothing
has
been
run.
C
A
C
Those
configurations
out,
yeah,
there's
also
the
ability,
there's
post,
cube,
adm
commands
and
those
will
by
the
bootstrap
provider
those
get
added
after
the
cube
adm
init
command
yeah.
So
it
orders
them
accordingly,
so
the
files
are
created
and
then
it
runs
the
commands.
So
that's,
basically
the
ordering
that
happens
here
automatically
for
us
yeah.
C
A
C
A
A
C
Yeah
exactly,
but
so
here,
what
we're
doing
is
right,
like
so
just
as
an
example
of
like
what
does
exist
out
of
the
box
is
actually
this
adding
into
the
etc
hosts
adding
in
the
host
name
that
was
received
to
127.001,
and
the
reason
this
is
so
important
is.
There
was
actually
an
issue
upstream
and
cluster
api
that
someone
said
that
their
nodes
were
coming
back
up
after
they
were
shut
down
and
he
powered
them
back
on
and
they
were
coming
back
up
without
an
ip
address.
C
Without
an
external
ip,
the
nodes
were
working,
but
the
external
ip
wasn't
returning
yeah
and
when
we
debug
the
issue.
What
it
seems
to
be
is
that
it
was
basically
trying
to
run
health
checks
against
the
node
name,
because
that's
the
way
that
kubernetes
is
running
cubelet
was
running
against
its
own
node
name,
but
because
his
manifest
didn't
have
adding
into
etc
hosts
the
host
name
and
localhost.
A
C
C
A
C
A
C
Exactly
right
so,
like
that's
just
like
another
example
of
these
cases,
where
you
know,
we
really
can
automate
most
of
these
fixes,
because
cluster
api
is
so
pluggable
and
tkgm
specifically
happens
to
be
the
only
distribution
that
exists
from
any
vendor
out
there.
That's
a
supported
cluster
api
distribution
that
gives
you
full
access
to
the
raw
cluster
api
manifests
so
in
other
products
from
vmware
and
from
other
providers
as
well.
C
C
C
So
everything
if
we
looked
at
the
default
values
here
right,
we've
got
vsphere
worker,
num,
cpus
and
disk,
give
and
memory
megabytes
and
data
center
and
datastore,
and
all
these
things
are
possible
for
all
of
our
for
our
single
machine
deployment.
But
what
if
I
want
that
different
for
each
machine
deployment.
A
A
A
So
matt
this
is
a
cool
one,
so
I
know
you
like
over
at
synopsis.
Y'all
are
talking
about
like
multiple
clusters
for
multiple
people,
so
scott's
sort
of
solved
that
with
tkgm
customizations
on
tanzu,
like
he's,
got
a
recipe
that
he
uses
that
will
just
roll
out
like
you
know,
50
clusters
for
people
and
it
just
he
just
indexes
stuff.
C
For
example,
right,
so
if
we
look
at
the
auto
scaler
code
in
tanzu,
if
you
said
enable
autos,
closer,
auto,
scaler
and
didn't
set
work,
and
you
didn't
set
auto
scaler
min
size,
zero
to
whatever
or
max
size,
it'll
take
the
machine
count
now,
just
because
someone
is
adding
another
machine
deployment
to
their
cluster.
That
doesn't
mean
that
I
want
them
to
have
to
repeat
every
variable.
So
the
nice
thing
in
ytt
is
that
we
can
actually
come
and
say:
okay,
the
annotation
of
cluster
autoscaler,
basically
say:
okay.
C
If
there
is
auto
scale
or
min
size
at
the
place
of
I
so
auto
scale
or
min
size.
One
two,
three,
four:
whatever
machine
deployment
they're
creating
here
awesome,
if
not
use
worker
machine
count
one
two,
three,
four
whatever
it
is,
if
that
doesn't
exist,
go
and
use
worker
machine
because
that
needed
to
exist
for
machine
deployment
zero.
C
So
this
way
we
can
actually
play
around-
and
we
can
add
in
all
these
customizations
here
and
just
have
it
in
for
loop
here,
where
they
just
set
an
additional
md
count
and
say
how
many
machine
deployments
and
automatically
everything
around
cluster
auto
scaler
the
amount
of
replicas
everything
and
then
also
what
we
added,
because
it
doesn't
exist
in
tkg.
Right
now,
is
the
ability
to
add
node
labels.
C
So
what
if
I
wanted
to
have
labels
on
my
worker
nodes,
a
specific
label
when
they
came
up-
and
this
becomes
really
important
when
I
have
multiple
machine
deployments,
because
if
I
have
two
sizes,
one
has
gpus
the
other.
Doesn't
one
is
you
know
large?
The
other
is
small
and
I
want
to
be
able
to
target
them
accordingly.
C
A
A
A
A
C
Yeah
and
really
just
a
call
out
to
everyone
that,
if
anyone
you
know
has
questions
on
any
of
this
stuff
or
you
know,
is
trying
and
is
getting
stopped,
you
know
feel
free
to
reach
out
to
me
on
all
the
social
media
channels
and
whatnot.
I'm.
The
rabbi
everywhere
is
my
you
know,
handle
on
the
kubernetes
slack,
the
bmw
code,
slack
on
everywhere,
so
really
feel
free
to
reach
out
and
raise
issues
on
the
repo.
C
A
We
yeah
we
have
an
and
we
have
a
little
bit
of
a
carvel
show,
but
we
didn't
go
deep
into
the
carville
tooling,
that
configures
andrea
specifically,
and
how
that's
plumbed
from
tce
into
framework
and
the
life
cycle
of
that
and
a
meme
on
on
on
my
team
here
at
vmware
sort
of
owns
all
that
and
I'm
sure
he'd
be
happy
to
walk
folks
through
that.
So
yeah.
That.