►
From YouTube: TGI Kubernetes 024: Kubernetes Job objects
Description
Index:
0:02:58 - CoreOS and Red Hat
0:10:39 - Heptio Kubernetes Subscription (HKS)
0:12:07 - KubeCon EU contributor summit
0:13:14 - New k8s.io docs experience
0:14:00 - Removing the term "Master" from k8s
0:16:52 - Job object: intro
0:21:03 - Job object: one shot
0:38:46 - Job object: failures
0:43:42 - kubectl run semantics
0:47:34 - Job object: parallelism
0:52:54 - Job object: work queue
1:07:11 - Job object: sidecar gotcha
Come hang out with Joe Beda as he does a bit of hands on exploration of Kubernetes and related topics. Some of this will be Joe talking about the things he knows well. Some of this will be Joe exploring something new with the audience. Ask questions, comment and help decide where things go.
This week we'll dig into the Kubernetes "Job" primitive. We'll look at how it works, how it compares to other controllers like Deployments and common usage patterns.
Jobs object links:
* KUAR excerpt link (unfortunately doesn't have Jobs chapter): http://go.heptio.com/kubernetes-up-and-running
* k8s.io docs on Jobs: https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/
* Issue around sidecars and jobs: https://github.com/kubernetes/kubernetes/issues/25908
* kubectl run behavior based on flags: https://kubernetes.io/docs/reference/kubectl/conventions/#generators
* KUAR org with kuard and examples: https://github.com/kubernetes-up-and-running
Weekly review of k8s land:
* CoreOS joins RedHat: https://coreos.com/blog/coreos-agrees-to-join-red-hat
* Container Linux lives: https://groups.google.com/forum/m/#!topic/coreos-user/GR4YlF2c1dM
* CoreOS/Red Hat FAQ: https://www.redhat.com/en/blog/faq-red-hat-acquire-coreos
* Brandon is a dad! https://twitter.com/BrandonPhilips/status/958809760299565056
* Brian shaves his beard! https://twitter.com/brianredbeard/status/958455929694953473
* Heptio Kubernetes Subscription. The Undistro: https://blog.heptio.com/introducing-heptio-kubernetes-subscription-5415052ef374
* KubeCon EU Contributor Summit: https://github.com/kubernetes/community/tree/master/events/2018/05-contributor-summit
* New k8s.io docs experience! https://kubernetes.io/docs/home/
* Rename Master in Kubernetes: https://github.com/kubernetes/website/issues/6525
A
Hello
and
welcome
to
t
GI
kubernetes.
This
is
what
are
we
on
episode:
24,
I'm,
Joe,
Beda
I'm,
your
host
in
this
webseries
I.
Don't
know
what
you
call
this
thing,
but
in
this
show
what
I
do
is
I
every
week,
Friday
or
so
I
get
together
and
with
a
bunch
of
folks
who
talk
back
to
me
on
the
on
the
the
chat
Channel
here,
how's
it
going
Josh
and
talk
about
kubernetes
go
over
what
happened
over
the
last
week
and
then
pick
a
top.
It
can
go
deep
on
into
it.
A
Sometimes
these
topics
are
new
projects
where
I
play
around
with
them
sort
of
as
a
new
user,
and
we
explore
them
together,
sometimes
I'm
going
over
sort
of
some
of
the
fundamentals
of
kubernetes,
sometimes
talking
about
new
things
coming
in
kubernetes
itself.
So
it's
always
a
lot
of
fun.
To
do
that,
I
did
spend
a
couple
of
episodes
actually
doing
some
live
programming
that
was
kind
of
fun.
Also,
so,
let's
see
so
I
love
it.
When
folks
talk
back
to
me.
A
So
if
you
have
any
questions,
if,
if
something's
not
clear,
if
you
can't
read
what's
on
the
screen,
definitely
let
me
know
I'd
love
to
hear
that
stuff.
So
solo
Maddie
welcome
back
how's
it
going
and
Josh
there's
too
rash.
Keith
is
in
Scotland
Tim
in
Chicago.
Sean
Jim
is
in
Florida
Carlos
in
Cuba
good
to
see
you
I
think
it's
just
totally
crazy,
like
when
I
was
growing
up
the
whole
idea
of,
like
you
know,
doing
something
like
this
with
somebody
in
Cuba
just
totally
nuts.
It's
awesome.
A
Q
alle
in
sweden
LOL
rip
good
to
see
you
again,
man,
Orr's
justin,
vancouver
just
to
the
north
of
me
here
in
seattle
and
and
jason
good
to
see
y'all
alright.
So
today's
topic
is
we're
going
to
talking
about
the
jobs
object,
which
is,
is
a
way
of
doing
run,
to
completion
tasks
versus
daemon
tasks,
things
that
you
want
to
keep
up
and
running
all
the
time.
So
we're
going
to
dig
into
that.
A
A
Of
the
stuff
that's
happened
and
then
I'll
try
and
make
my
way
through
a
bunch
of
stuff
that
I
have
sort
of
sketched
out
in
my
brain
about
the
jobs,
object
I'm
a
little
feeling
a
little
underprepared.
This
time
have
been
so
busy
this
week,
but
I
think
I
think
it'll
turn
out
good,
regardless
so
I
think
we're
gonna
be
in
good
shape.
So
dan
and
Erin
good
to
see
you
alright
Sola
Mandy's.
What
do
you
think
about
core
OS
getting
bought?
What
that
happened?
I
didn't
hear
about
that.
Now.
A
We're
gonna
talk
about
that
vennett
and
India
David
good
to
see
y'all
alright.
So
let
me
go
through
and
I'm
gonna
try
and
keep
this
down
to
like
not
too
over
an
hour.
I'd
like
they've
been
getting
longer
and
longer
and
I
know
that
there's
a
limited
attention
span
for
folks
to
actually
watch
this
stuff.
I
know
you're
all
are
committed,
but
I
don't
think
you
need
to
be
two
hours
committed
to
learn
something.
So,
let's
see
if
we
can
do
better
with
that.
A
So
let
me
switch
to
my
screen
here
and
obviously
the
big
news
this
week
is
the
core
OS
folks
getting
purchased
by
Red
Hat,
really
exciting
stuff
going
on.
There
I
think
it's
it's
it's
good
to
see
that
Red
Hat's,
doubling
down
on
their
investment
kubernetes,
obviously
they've
been
they've,
been
partners
in
the
community
for
a
long
time
and
so
really
really
cool
to
see
that
happen
and
I.
Think
a
lot
of
people
have
a
lot
of
questions
around
this
I
think
and
and.
A
And
I
think
you
know
there's
still.
Some
of
those
questions
are
being
answered
over
time.
I'll
tell
you
sort
of
what
I've
heard
and
what
I've
known
and
what
I've
heard
about
here.
So
the
first
thing
is
the
purchase
price
was
was
250
million
dollars
as
somebody
with
a
start-up.
The
fact
that
you
know
when
you
read
these
things:
if
they,
if
they
talk
about
the
purchase
price,
then
it's
a
good
thing.
If
they
don't
mention
the
purchase
price,
then
there's
a
reason
why
they
don't
mention
it.
A
A
A
What
that
means,
because
I'm
not
going
to
swear
in
this
fine
family
friendly,
show
but
like
if
you're
on
call
and
it's
totally
like
coming
at
you
from
all
sides,
you
can
tell
your
phone
to
shut
up.
Nothing
feels
better
than
hitting
the
STFU
button
when
you're,
actually
on
call
and
getting
slammed.
So
that's
a
lot
of
fun,
all
right,
so
so
yeah
so
back
to
core
OS.
So,
yes,
a
good
purchase
price,
really
happy
to
see
those
folks
land.
Well,
there's
still
a
lot
of
open
questions
and
I.
A
Think
some
of
those
are
getting
answered
so
the
first
one
here-
and
this
is
an
email
here-
I'll-
make
it
a
little
bit
larger
that
was
sent
to
the
to
the
core
OS
user
Google
group-
and
somebody
pointed
this
out
to
me-
and
this
calls
attention
and
so
I
think
this
is
one
of
the
questions.
That's
been
on
everybody's
mind.
A
And
there
was
some
confusion:
there
were
some
questions
there.
I
I
think
this
goes
to
answering
some
of
it.
I,
don't
think
it
answers
sort
of
the
total
long-term
view
of
it,
but
I
think
what
it
says
is
that
like
I,
don't
believe
if
I'm
reading
this
right,
you
can
read
it
yourself,
I,
don't
believe
Red
Hat's
going
to
be
offering
commercial
support
for
core
OS
container
Linux,
but
that
doesn't
mean
that
container
and
Linux
is
going
away.
I
think
you
know
this
line
here
is
really
interesting.
A
Red
Hat
plans
to
continue
container
Linux
development,
so
that's
I
think
for
a
lot
of
folks.
Who've
bet
on
that
who
are
using
that
right
now,
I
think
that
definitely
helps
them
sleep
a
little
bit
better.
It
at
least
says
that
this
isn't
going
to
disappear
overnight.
They're
not
going
to
yank
the
rug
out
from
folks
and
I,
wouldn't
expect
them
to
I
think
both
Red
Hat
and
core
OS.
A
You
know
they
understand
that
that
they
have
a
commitment
to
users
and
they're
not
going
to
leave
people
people
hanging
like
that,
but
it's
good
to
actually
see
that
written
down
someplace.
So
David
saying
I'm,
especially
looking
forward
to
tectonic
possibly
getting
open
source.
Another
entry
and
the
facts
suggest
this
will
happen
but
more
definite,
no
definite
commitment,
yeah.
A
Question
is,
what's
going
to
happen
with
tectonic
I,
even
looking
at
something
like
way.
Generally
RedHat
open
sources.
A
lot
of
stuff
are
some
of
the
the
Quai
server-side
components
actually
going
to
get
open
sourced.
Also
those
so
there's
some
questions
that
I
think
folks
definitely
have.
It's
also
really
good
to
see
a
company
with
means.
A
You
know,
I
think
one
of
the
things
that
gets
lost
in
all
this
is
the
fact
that
that
core
OS
is
is:
is
the
company
behind
at
CD
there's
other
folks,
obviously
involved
at
CD
is
a
core
dependency
of
kubernetes.
It's
really
good
to
see
a
company
like
like
Red
Hat,
you
know
standing
behind
at
CD
as
it
continues
to
mature
over
time.
A
So
that's
really
really
good
as
wondering
if
they're
gonna
kill
tectonic
or
not
because
of
open
ships,
I
think
that's
you
know,
honestly,
that's
that's
one
of
the
one
of
the
questions
that
folks
are
wondering
is,
is
you
know
Red
Hat's
current
strategy
with
open
shift
is?
Is
they
ran
ahead
added
a
bunch
of
features
to
what's
kind
of
a
fork
of
kubernetes,
with
open
shift
they're
working
now
to
take
all
those
features
and
upstream
them
there?
A
You
know
that's
a
painful
process
just
because
you
know
people
are
taking
a
close
look
as
these
things
make
their
way
upstream.
You
know
our
back
is
a
good
example
of
something
that
was
upstream
from
open
shift.
Meanwhile,
tectonic
is
there's
relatively
close
to
to
upstream
kubernetes.
It's
you
know,
it's
it's
the
same
communities,
you
know,
I,
don't
believe
tectonic
actually
carries
any
patches
long-term
for
for
kubernetes,
and
so
so
that
sort
of
you
know
pure
kubernetes,
ish
type
of
thing.
A
That
tectonic
is
versus
the
open
shift,
which
is
kind
of
a
fork
of
Cooper
neighs.
With
with
with
extra
features,
it's
not
clear
yet
how
they're
gonna
reconcile
those
two.
So
the
one
thing
to
keep
in
mind
here
is
that
there's
this
FAQ
here,
that's
linked
off
of
this
and
I'll
include
all
these
in
the
show
notes.
That
has
a
lot
of
questions
and
answers,
and
so
this,
you
know,
isn't
going
to
give
you
sort
of
certainty
on
everything,
but
I
think
this
definitely
definitely
gives
some
certainty
there
but
related
to
chorus.
There's.
A
You
know
it's
a
it's
a
great
crew
they're.
You
know
super
friendly
with
a
with
a
lot
of
the
folks
there.
So
the
first
one
is
Brandon.
Phillips
is
the
CTO
of
core
OS
and,
and
he
had
a
daughter,
he
and
his
wife
had
a
daughter
on
January
10th
I
believe
is
when
she
was
born.
So
there's
the
picture
there.
So
congratulations
to
Brandon,
really
cute
picture
and
I
know
he's
not
getting
a
lot
of
sleep
he's
been
working
when
he
can
call
him
to
some
of
the
steering
committee
meetings
and
stuff
like
that.
A
A
Know
I
have
no
idea,
I,
don't
know
how
these
things
work
out.
All
I
know
is
I
love,
working
with
Brandon,
I'm
sure,
whatever
happens
with
him,
he'll
he'll
be
doing
some
some
really
interesting
stuff
and
then
there's
there's
Brian
Redbeard.
Okay,
so
you
guys
know
so
so,
like
I'll
click
on
his
picture
here
and
there's
like
so
this
is
this-
is
this:
is
Brian
red
beard?
A
Okay,
he
has,
or
had
this
this
amazing
beard
that
he
cut
off
and
I
believe
at
one
point
he
made
the
he
made
the
commitment
that
if
core
OS
IPO
he
cut
off
his
beard,
but
he
figured
this
was
close
enough
and
he
cut
off
his
beard.
So
anybody
who's
known
him
like
this
is
this
here
is
a
that's
a
pretty
big
change.
That's
hilarious!
So
congratulation
no
beard,
never
I
just
got
to
change
his
Twitter
handle.
So
congratulations
to
the
Korres
folks.
That's
super
super
exciting
good
for
them.
A
So
the
next
thing
I
want
to
talk
about
is
here
at
hefty
Oh,
where
we
started
talking
about
some
of
the
stuff
that
actually
we've
been
we've
been
doing
for
a
while,
but
we've
we've
been
working
on.
How
are
we
describing
it?
How
are
we
selling
this
to
customers,
and
so
we
announced
this
thing
that
we're
calling
the
hefty
o
kubernetes
subscription
I
gave
it
the
name,
the
undistribute
Anor,
the
undestroyed.
A
The
idea
here
is
that
a
lot
of
times
people
want
and
let's
see
if
I
can
find
the
so
so
here's
sort
of
like
the
the
summary
here
a
lot
of
times
they
want
a
they
want
somebody
to
have
their
back.
They
want
somebody
who
will,
you
know,
cut
them
new
binaries
with
a
hotfix
if
they
need
to.
They
want
somebody
to
call,
and
they
want
advice
on
sort
of
how
to
put
everything
together,
but
they
don't
want
to
get
locked
in
long
term.
A
To
you
know,
saying
you
know:
hey
I'm
gonna
have
to
to
go
through
and,
like
you
know,
repave
the
world
to
to
get
off
of
this,
offering
they
don't
want
to
feel
that
level
of
lock-in,
and
so
so
that's
how
we're
viewing
our
offering
here
around
HKS
is.
We
want
to
give
people
a
lot
of
the
good
stuff
about
a
distro,
but
also
as
much
as
possible.
A
A
You
might
have
seen
some
news
on
that.
Let's
see
other
stuff
that
we
have
going
on
if
you're
coming
to
cube
con
EU
the
day
before.
That
is
the
contributor
summit,
the
the
folks
in
contributes,
and
the
community
managers
and
I
think
this
is
mainly
being
driven
by
by
Paris
who's
at
Google
and
George
was
at
hefty.
A
So
if
you
want
to
get
involved
in
kubernetes
and
if
you're
going
to
be
a
cube
connie,
you
and-
and
you
want
to
become
a
cooper
contributor-
we
don't
want
that
contributor
summit
just
to
be
for
folks
who
are
knee-deep
in
kubernetes,
but
also
folks,
who
are
interested
in
perhaps
getting
knee-deep
in
kubernetes.
So.
A
Last
time
we
did
this,
it
was,
it
was
good,
but
we
didn't
get
the
word
out
soon
enough,
and
so
it's
it's
great
to
see
Paris
and
George,
starting
to
to
figure
out
the
details
here
and
see
this
start
to
shape
up
now.
So
so
that's
been
announced.
Also,
let's
see
other
interesting
stuff,
is
that
there
was
a
redo
of
some
of
the
kubernetes
documentation
on
the
site,
which
I
think
is
really
really
interesting.
It's
using
this.
You
know
the
idea
of
personas
is
like
I'm,
a
developer
operator
and
sort
of
what
level
am
I.
A
Looking
for
and
then
sort
of
what
tasks
do
I
want
to
do
some,
a
cluster
operator
and
I'm
intermediate,
so
I'm
interested
in
security,
best
practices,
for
example,
and
here's
some
resources
here
and
this
will
be
expanded
over
time,
but
the
idea
being
that,
instead
of
having
a
big
pile
of
documentation,
would
have
a
flow
that
really
works
with
who
you
are
and
what
are
you
looking
to
get
done,
that
that
gets
you
to
the
right
part
of
the
documentation?
So
that's
really
excited.
A
A
Will
we
referred
to
that
as
the
sort
of
the
master
machine
and
sometimes
we'll
refer
to
the
control
plane
in
general,
as
as
the
master?
But
it's
really
not
a
super
accurate
description
of
what's
going
on,
and
so
both
in
terms
of
like
hey,
let's
just
try
and
sort
of
not
bring
up
these
bad
connotations
that
that
can
be
offensive
or
bring
to
mind
some
really
unfortunate
things
Oh,
along
with.
A
Let's
use,
probably
you
know
more
meaningful
technical
terminology,
we're
gonna,
try
and
actually
not
use
the
term
master
as
we
work
through
as
we
talk
about
stuff.
So,
if
you
find
me
using
the
word
master
on
tea
gik,
please
call
me
out
on
it
and
I'll
slap
myself
here,
because
I
think
big
part
of
this
is
we're.
Gonna,
be
changing
the
documentation
it
doesn't
show
up
in
the
code
too
many
places,
I
think
cube.
A
So
so
Q
says
I
like
the
intention
behind
it,
but
with
so
many
different
terms
and
the
changing
it
just
makes
it
harder
to
keep
up.
I
think
that
is
a
concern.
I
totally
hear
you
here,
yeah
it's
it's!
We
are
yep
one
of
the
things
here
and
I
think
you
know,
there's
a
lot
of
people
coming
up
with
ideas
on
sort
of
like
what
are
different
names
we
can
use
here.
I
I,
think
one
of
my
criterias
here
is
I.
Just
don't
want
it
to
be
too
cutesy.
A
I
think
like
we
already
have
enough
sort
of
specialized
cutesy
terminology
and
kubernetes
like
everybody's,
like
what
the
hell's
a
pod
right,
I,
don't
think
we
want
to
reduce
another
one.
So
I
think
somebody's
like
you
know
talking
but
I,
think
like
control
plane
is
relatively
well
understood,
so
so
that's
kind
of
where
we're
I'm
thinking
right
now,
but
I
still
think
we
want
to.
We
want
to
give
a
folks
a
chance
to
sort
of
weigh
in
for
people
to
sort
of
wrap
their
head
around
this
okay.
A
The
outline
of
what
I'm
gonna
be
doing
is
going
through
so
there's
kubernetes
up
and
running
here.
So
this
is
the
book
that
Kelsey,
Brennan
and
I
wrote.
I
wrote
the
jobs
chapter
in
here.
This
was
chapter
10.
Let's
see
if
this
thing
focuses,
maybe
yeah
the
jobs
chapter
and
so
I'm
gonna
be
using
this
as
an
outline
to
actually
go
through
this
stuff,
but
along
the
way,
hopefully
I
can
give
some
more
color
and
you'll
actually
see
it
in
action.
A
I
think
that'll
be
kind
of
fun,
but
if
you
want
to
play
along
at
home,
you
can
go
ahead
and
if
you
have
a
copy
of
the
book,
you
can
get
that
out.
What
I
wanted
to
do
is
say:
hey.
You
can
go
download
a
free
copy
of
the
ebook
here
at
you
know,
because
we
sponsored
an
excerpt
of
the
thing
and
so
like,
and
we
actually
have
printed
versions
of
these
that
we
actually
give
out.
A
The
excerpt
for
free
and
there
you
go,
but
it
turns
out
that
that
the
expert
doesn't
have
the
chapter
that
we're
covering,
but
if
you're
looking
for
other
sort
of
you
know
reference
material,
you
can
go
ahead
and
download
this
so
I
had
this
all
prepped
and
then
I
went
and
grabbed
a
copy
I'm
like
Tom.
We
forgot
to
you
know
we
didn't
include
that
chapter.
It
was
tough
to
choose
what
to
include
what
not
to
include
in
that
in
the
excerpt
all
right.
A
So,
let's
see
I'm
going
to
reorder
some
tests
here,
cuz
I'm
going
to
get
to
yeah.
So
there's
that
sorry
I
had
some
extra
temp.
So
so
so
here's
the
the
documentation
on
the
side
on
the
jobs,
thing
and
I
think
this
is
pretty
good,
but
I,
don't
think
I,
don't
I,
don't
think
like
let's
sort
of
dig
into
it
a
little
bit.
A
So
this
is
good
reference
there,
but
what
I'm
gonna
be
talking
about
is
the
different
types
of
scenarios
of
where
you'd
be
using
jobs,
and
so
and
so
the
first
thing
is
like.
What
do
we
mean
when
we
say
a
job
versus
versus
regular
usages
of
kubernetes,
and
it's
it's,
it's
actually
a
pretty
simple
and
yet
subtle
thing.
Is
that
regular
kubernetes?
If
something
fails,
you
want
to
restart
it
so
the
goal
of
most
sort
of
like
it's
kind
of
a
damon
mode.
A
It's
it's
so
sometimes
people
talk
about
Damon's
versus
versus
jobs
or
Damon's
versus
batch,
but
the
idea
is
that
the
goal
is
to
keep
something
up,
no
matter
what
right,
and
so
that's
like
most
of
the
usage
of
kubernetes.
If
you
like,
do
a
deployment
or
stateful
sets
or
or
a
lot
of
the
other
usages
of
kubernetes,
it's
really
keep
this
thing
up,
no
matter
what,
by
hook
or
crook,
try
and
keep
this
thing
running.
A
Jobs
are
different.
Jobs
are
run
this
thing
until
it
exits
and
there's
different
ways
to
actually
sort
of
talk
about,
exiting
and
there's
different
like.
Maybe
you
don't
want
to
run
one
thing:
you
want
to
run
ten
things
and
but
it's
really
sort
of
run
to
completion
versus
run
to
run
forever,
and
that's
really
the
fundamental
difference
between
when
we
say
job
versus
when
we
talk
about
other
things
now.
The
interesting
thing
is
that
this
still
uses
pods
under
the
covers
right.
So
so
so
job
is
something
like
an
init
pod.
A
Is
it
well
I
mean
so
I
think
we
even
is
an
it
container
inside
of
a
PI,
and
so
the
idea
with
the
NIC
container?
Is
it
something
that
you
want
to
run
to
completion
before
you
start
the
real
task?
So
it's
sort
of
a
bookkeeping
sort
of
preparation,
type
of
thing-
and
here
this
run
to
completion
stuff-
is
really
the
the
main
event.
So
it's
the
thing
that
you're
trying
to
make
work
and
so
I'll
go
through
some
of
the
scenarios
here.
A
You
run
it,
but
then
it
actually
knows
when
it
wants
to
exit
cleanly,
and
so
that's
the
idea
behind
jobs,
and
so
there's
three
different
scenarios
that
I'm
going
to
be
talking
about,
and
these
are
the
ones
that
are
covered
in
the
book.
So
the
first
one
is
a
one-shot
job,
and
this
is
like
I
want
to
run
something
I
want
to
run
it
to
completion
right.
A
You
can
set
this
up
so
that,
like,
if
it
fails,
do
you
want
to
restart
it
until
it
succeeds,
or
if
it
fails,
it's
like
boom.
That's
it.
If
the
underlying
infrastructure
fails,
it'll,
try
and
run
it
someplace
else,
and
so
there
are
some
sort
of
quasi
sort
of
like
like
in
distributed
systems
in
general.
This
idea
of
like
Oh
suresh's,
the
unit
container,
is
pause,
not
finished.
A
B
A
So
what
so,
so
the
ideal
thing
here
is
that
you'd
want
to
like.
If
you,
if
you
do
a
one
shot,
you
want
to
run
once
and
only
once,
but
it
turns
out
that
in
distributed
systems,
doing
something
just
once
is
actually
really
hard.
When
you
look
at
systems
like
Kafka,
they
want
to
provide
some
sort
of
like
guaranteed
ones
delivery,
but
when
components
can
actually
crash
or
you
can
have
network
partitions,
you
may
say
hey
run
and
then
you
you
know,
and
then
you
never
hear
anything
back
well.
A
You
know
the
pattern
here
is
to
build
stuff
to
be
as
idempotent
as
possible,
which
means
that
you
can
run
it
multiple
times
and
you
can
get
the
same
impact
and
that
really
is
sort
of
like
one
of
the
main
sort
of
like
design
criteria
for
kubernetes
is
we
want
to
talk
about
what
we
want
not
how
to
get
there?
So
so,
let's
do
a
quick
example
of
how
this
works,
and
so
here
I,
so
I
have
a
cluster.
A
B
A
Up
this
kubernetes
roll
master,
and
so
we
have
a
convention
here
in
terms
of
in
terms
of
roles
and
then
node
role,
master,
and
so
this
stuff
is
built
into
into
cube
admin,
and
then
the
cubelet
knows
how
to
actually
call
this
stuff
out
and
and
expose
it.
So
that's
something
that
we're
gonna
have
to
work
on
all
right.
So
a
little
tangent
there,
okay,
so
I
have
a
cluster
here
with,
with,
with
with
the
control,
plane,
node
the
infrastructure
node.
A
So
we
call
it
that
I,
don't
know
the
control,
plane,
node,
plus
plus
three
worker
nodes
and
what
we're
gonna
do
and
I
have
a
cheat
sheet
document
here.
Is
we're
gonna
call
this
thing
here,
and
so
actually,
let's
let's
go
back
here
and
we'll
look
at
what
this
thing
does
so
this
is
I
want
to
do
a
run.
The
I
says
that
I
want
to
actually
get
the
output
stream
to
me,
so
the
image
that
I
want
to
run
is
this
thing
restart
on
failure.
A
So
this
is
the
thing
that
makes
it
a
job
versus
not
and
and
there's
I'll
go
into
a
little
bit
of
detail
there.
It's
because
this
is
cute
control
run
is
a
little
bit
confusing
here
and
I
want
to
I
want
to
dig
into
that,
and
then
everything
after
the
is
our
commands
to
the
underlying
image
that
we're
running,
and
here
we're
running
this
with
keygen
and
able
exit
uncomplete.
And
then
we
want
to
generate
number
10
key,
so
I'm
creating
private
keys
using
a
cryptographic
algorithm.
They
you
know
they
take.
A
You
know
a
couple
seconds
each
to
generate,
and
so
it's
a
good
sort
of
like
short-lived
worker
type
of
thing
for
being
able
to
generate
stuff.
So
so
the
workload
here
that
we're
doing
is
generating
a
key.
So
this
could
be
rendering
a
frame
in
a
movie
or
it
could
be
or
it
could
be,
you
know
resizing
an
image
or
it
could
be
encoding
a
video
or
it
could
be.
You
know,
I,
don't
know
so
the
any
number
of
sort
of
work
queue
items
you
could
think
about
doing
there.
A
So
let's
go
ahead
and
run
this
and
see
what
happens
and
what
I'm
gonna
do
is
down
here.
I'm
gonna
do
a
cube,
controlled,
get
pause,
W,
so
we'll
actually
see
this
thing
running.
The
W
means
we'll
watch
and
so,
as
things
come
and
go,
we'll
actually
see
them
come
and
go
here
or
actually
no,
let's
do
this.
A
Instead,
we're
going
to
do
kit
pods
we're
going
to
do
watch
every
five,
every
half
a
second
we're
gonna
call
queue,
control,
get
pods
and
we're
going
to
output
that
to
the
screen,
and
so
we
just
got
no
resources
found
and
now,
if
I
go
through
and
open
its
my
paste
buffer,
there
I'm
gonna
go
ahead
and
run
this,
and
here
you
can
see
container
creating.
So
it's
downloading
stuff
now.
B
A
We
go
and
it's
done
and
every
time
it
generates
one
of
these
things,
it's
jumping
a
shot,
256
of
the
private
private
key
that
it
actually
made
or
the
public
key
I
don't
know.
So
you
can
look
at
the
Cordy
code.
If
you
want
to
see
how
this
stuff
is
working,
and
so
you
can,
this
takes
a
couple
seconds
each
to
run,
and
then,
when
this
runs
it's
going
to
exit
with
success
and
then
the
whole
thing's
gonna,
actually
shut
down
and
and
and
the
pod
will
no
longer
be
running.
A
A
Okay,
the
use
cases
for
something
like
this
is
when
you
just
want
to
like
run
something
and
then
have
it
exit
so
so
sometimes
like
people
might
use
this
to
say
to
a
database.
Migration
might
be
something
that
you'd
want
to
do
or
if
you
have
sort
of
a
one-off
task
of
something
that
you
need
to
do
so
like
you
could
do
this
of,
like
hey
I,
want
to
copy.
A
You
know
you
know,
okay,
so
let's
say
that
you
want
to
copy
some
data
from
you
know:
Google
Cloud
Storage
to
s3
right,
and
so
you
have
a
command
line
tool
to
go
ahead
and
do
that
you
could
on
your
kubernetes
cluster,
you
could
run
a
one-shot
thing
that
actually
goes
through
and
copies
the
data
from
one
to
the
other
you're
running
that
in
the
cloud
once
it's
done,
you're
done,
but
but
after
but
after
that,
you
you
want
to.
You
want
to
be
done.
A
So,
in
some
ways
like
like,
we
look
we're
looking
at
the
workflow
systems
before
this
is
like
a
super
duper
simple
sort
of
single
stage
for
the
workflow
right.
So
this
is
you
know
this
ends
up
being
something
smaller
than
the
than
the
in
the
workflow
system,
so
david
says:
do
jobs
have
any
facilities
to
help
job
writers
paralyze
their
job
iet
metadata,
like
your
shard,
five
of
one,
fifty?
A
No,
there
is
no
assignment
there,
and
I
think
this
is
something
that
in
general,
kubernetes
has
avoided
for
a
variety
of
reasons.
I
think
one
way
to
think
about
doing
this
is
it's
not
uncommon.
You
know
to
have
a
bunch
of
things
join.
You
know,
communicate
with
each
other
and
sort
of
do
like
ringg
hash
so
that
they
can
actually
figure
out
which
part
of
a
hash
space
that
they
might
actually
deal
with
yeah
I'm,
trying
to
think
I.
You
know,
there's
other
techniques
that
you
can
use
there
they're.
A
The
reason
why
I
think
people
are
a
little
bit
allergic
to
sort
of
citing
like
shard
numbers
to
individual.
You
know,
tasks
and
we're
doing
parallelism
we'll
get
to
the
parallelism
side
of
the
house
is
because-
and
it
doesn't
apply
as
much
to
the
job
stuff
as
it
applies
to
the
to
the
for
the
long
running
tasks.
A
Is
that
when
you
have
long
running
tasks
and
and
like,
and
you
have
to
shrink
them
or
or
or
increase
them,
if
you
actually
give
them
indexes-
and
this
is
the
way
that
Borg
and
something
like
marathon
works,
then
when
you
shrink,
you
have
to
always
shrink
from
the
end.
So
that's
very
deterministic,
which
things
you
actually
need
to
remove,
which
gives
the
system
less
flexibility
around
that
stuff
and
then
also
with
respect
to
upgrades.
It's
you
have
to
be
very
for
doing
a
rolling
update.
A
You
have
to
be
very,
very
disciplined
about
sort
of
how
you
do
that
and
what
you
end
up
doing
is
building
that
rolling
update
state
machine
into
the
core
of
the
system.
Whereas
right
now
you
know,
replica
sets
and
pods
themselves
have
no
idea
about
how
deployment
does
a
rolling
update
and
keeping
those
separation
of
concerns
is
enabled,
because
we
don't
have
any
guarantees
around
indexes.
A
A
A
I
need
to
do
a
lot
of
stuff
like
traditionally,
you
think
about,
like
hey,
I'm
gonna
like
introduce
some
threads
and
and
then
I
can
use
multi-threaded
right,
but
you
can
only
do
multi-threaded
up
to
the
number
of
you
know,
CPUs
on
your
machine
before
you
get
diminishing
returns.
If
you
do
it
right,
you
can
launch
two
jobs
and
then
those
jobs
actually
essentially
become
sort
of
a
multi
processing.
A
So
now
you're
thinking
about
extending
your
stuff
across
the
cluster,
instead
of
extending
it
just
across
the
the
machine-
and
so
oh
hell
does
have
hooks
that,
let
you
run
a
job.
Okay,
so
I'm
wrong.
Bismarck
is
calling
me
out
there
I'm
wrong
okay,
and
then
it
says
at
this
point:
can
you
run
the
job
again
where
you
need
to
clean
up
the
completed
job?
A
B
A
Okay,
so
it's
hard
to
get
this
with
cute
control
run,
but
when
we
talked
about
this
in
a
previous
TGI
K,
so
let's
go
ahead
and
we'll
switch
to
to
the
ammo
equivalent
of
this.
So
so
the
API
version
is
batch,
its
job,
it's
one
shop
again.
This
was
directly
out
of
the
book
here,
where
we're
doing
some
metadata
and
labels
and
stuff
like
that
restart
policy
on
failure.
But
one
of
the
things
that
I
can
do
here
is
and
I
think
yeah.
A
At
the
API,
so
what
happens
if
I
run
the
job
again
before
deleting
the
results?
I
believe
I
would
get
an
error
if
I
try
to
run
the
thing
again,
because
it's
gonna
try
and
do
a
create
if
you
do
an
apply
and
the
things
already
committed
I,
don't
think
it
does
like
a
reset
back
to
sort
of
like
the
on
run
State.
So
so
at
that
point
it's
kind
of
it's
done.
A
So
let
me
pull
up
the
API
reference,
but
but
one
of
the
things-
and
this
works
really
well
with
jobs.
Is
that-
and
we
talked
about
this
in
a
previous
episode-
is
that
you
can
give
a
prefix
and
then
have
kubernetes
actually
decide
to
what
name
that
thing
is
going
to
get
so
you
can
either
name
your
objects
or
you
can
have
kubernetes
decide
the
name
of
the
object.
Why
is
this
thing
not
come
on?
Man
is
the.
A
Sorry
I'm
looking
this
up
because
I
can't
remember
the
name
off
the
the
top
of
my
head,
so
I'm
digging
deep
here,
I'm
gonna.
We
need
better
API
documentation,
better
sort
of
like
e
ammo
descriptions
in
general
here
so
generate
name.
That's
the
name
of
the
thing,
and
so
here's
the
this
is
the
sort
of
the
canonical
place
where
this
stuff
is,
but
this
flows
into
the
other
documentation
around
the
API.
So
if
we
do
instead
of
name
here,
we
do
generate
name.
A
A
B
B
Will
be
made
unique
and
the
name
return
client
will
be
different
than
pass.
The
values
field
will
be
combined
with
the
unique
suffix
and
the
server
bubble
blah.
If
this
field
is
specified
and
name
is
not
present,
server
will
not
return
to
for
nine;
instead,
it
will
return
to
and
created
or
if
I,
when
the
server
timeout.
So
why,
in
this
orchid
item,
is
it
what
the
heck
that
should
work.
A
A
A
A
And
so
you
can
see
and
then
it
actually
there's
there's
you
know
when
it
completed
how
many
completions
what
happened
there
get
pods.
If
I
look
at
all
the
pause,
you
can
see,
here's
you
know
and
prefix
is
predictable.
So
you
can
try
and
figure
stuff
out,
but
you
can
also
go
through
and
use
use
labels
to
actually
relate
these
things
back
and
forth
to
and
then
we
can
go
through
and
we
can
keep
cube
control
logs
and
you
can
see.
B
A
Now
one
of
the
things
that
we
can
do
here
is:
let's
simulate
a
failure
right,
so
let's
go
through
and
we'll
do
it's
key
gin
exit
code,
one
so
now,
when
this
this
exits
that
connect
going
to
exit
with
a
1
instead
of
a
zero.
So
those
who
are
not
super
familiar
with
sort
of
UNIX
e
stuff
when
a
process
exits,
it
actually
returns
an
exit
code
back
to
the
calling
process
or
your
shell
or
whatever
and
unintuitive
ly.
Zero
means
success
and
any
other
number
actually
means
failure
of
some
sort.
A
There's
also
like
if
it's
killed
by
a
signal,
it's
a
negative
number.
If
it's
a
positive
number
and
there's
conventions-
and
sometimes
people
will
document
this,
but
for
the
purposes
of
kubernetes
it's
really
sort
of
like
zero
success.
Anything
else
is
failure.
So
now
we're
gonna
have
this
thing
exit
with
one
which
says
that
it's
a
failure
and
we
have
the
restart
policy
on
failure
here
and
I'm,
going
to
change
this
back
to
just
plain
old
name,
because
that'll
save
me
some
typing
and
copy
and
pasting.
A
B
A
Do
the
dash
F
I
think
that
doesn't
follow,
so
that's
essentially
saying
read
the
logs
and
then
stream
any
logs
afterwards.
So
we
can
actually
see
that
this
thing's
starting
up
we're
going
to
generate
two
of
these
keys
and
then
it's
going
to
exit,
but
it
exits
with
an
error
and
then
the
job
controller
is
going
to
go
ahead
and,
if
all
goes
as
planned,
it
should
actually
go
ahead
and
restart
the
thing
so
restarts
one.
A
C
A
A
B
A
A
Here
this
is
what's
so
confusing.
This
chart
is
beautiful,
okay,
so
essentially,
what
this
saying
is
is
there's
a
whole
bunch
of
options
to
cube
control
run
and
the
type
of
object
that
it
creates
under
the
covers
really
depends
on
the
options
that
it
says,
and
it
also
depends
on
the
cluster
that
you're
running
to
so
I
think
this
is
like
some
hidden
complexity.
A
I
think
this
is
trying
to
hide
some
of
how
kubernetes
works
to
users,
but
in
reality
it
means
that
you
have
to
sort
of
have
this
chart
decode
what
the
thing
is
really
doing.
So
the
default
is
that
you
have
this
like
restart
equals
always,
and
that
creates
a
deployment.
So
most
of
the
time
when
you
do
Cube
control
run,
you'll
get
the
restored,
equals,
always
and
then
you're
up
and
running
so
Pat
says
so.
Do
we
always
have
to
have
our
own
job
cleanup
process?
A
A
A
You
have
the
overall
cron
job
and
then
you
have
all
the
runs
of
the
cron
job,
but
each
job
itself
is
kind
of
like
a
one-shot
e
type
of
thing
right,
so
that
the
idea
is
that,
like,
if
you
have
your
own
process
that
say
triggering
jobs
based
on
some
sort
of
like
Jenkins
thing
or
whatever
you
want
to
say
hey.
If
this
thing's
older
than
a
day
then
go
ahead
and
delete
that
and
so
I
think
it's
totally
like
it's
it's
you
know
yeah
and
so
Pat's,
like
you,
do
something
in
a
cron
job.
A
What
I
would
do
is
have
something
like
a
label
or
an
annotation
that
says:
hey.
You
know
after
a
day
clean
this
thing
up
with
me,
be
a
default
process,
for
you
know
for
something:
that's
not
there
and
then
you
can
do
a
query
and
just
delete
those
things,
and
so
do
that
sort
of
you
know
maybe
on
your
own
cron
job
or
have
a
controller
that
keeps
track
of
that
stuff.
I.
Don't
think
anybody's
built
that
in
yet
to
be
honest,
so
container
v1
course
this
is
so.
Let's
see
it
would
probably
be
delecious.
A
This
is
all
so
stuff
like
the
deletion
grades.
That's
that's
for
graceful
deletion
when
it
takes
time
for
things
to
actually
be
deleted,
especially
namespaces
yeah.
So
it's
nothing
that's
sort
of
a
generic
built-in
thing
because
it
would
probably
show
up
I'm
guessing
here
in
the
metadata
thing,
because
this
is
where
you
see
things
like,
like
your
owner
references.
A
So
yeah
that'd
be
a
fun
little
add-on
for
somebody
to
write
there.
That'd
be
that'd,
be
pretty
cool,
okay,
so
cube
control,
so
it
so
is
interesting.
If
you
do
restart
on
failure,
then
it
becomes
a
job.
If
you
do
restart
equals,
never
then
it
creates
a
pod
for
you,
a
plain
old
pod
or
if
you
like,
passing
the
scheduled
thing,
then
it'll
go
through
and
actually
create
a
cron
job
for
you,
and
so
the
fact
that,
like
cute
control
run,
is
kind
of
modal
like
that
can
be
a
little
bit
confusing
for
folks.
A
Okay,
let's
see
what
do
I
just
that's
pod
failure
all
right!
So
now,
let's
talk
about
parallelism,
so
there's
two
there's
two
properties
on
the
job,
and
if
we
pull
up
this
here,
there's
one
called
parallelism
and
there's
one
called
completion,
and
what
this
says
here
is
it
says
that
I
want
to
run
ten
copies
of
this.
You
know
ten
instances
of
this
thing
to
completion.
That's
what
completions
is
and
I
want
to
have
it
most
five
running
at
any
one
time,
Wow
so
yeah,
so
research,
a
failure
gives
you
a
job.
A
That
seems
like
a
really
weird
wording
for
it.
I
get
it
now,
but
it's
not
way
clear
to
me
yeah.
I
think
I
think
that
cube
control
run
semantics
is
super
unclear
to
folks.
I
think
most
people
just
use
it
for
deployments
and
don't
really
look
that
much
deeper
for
it
yeah.
But
I
think
this
is
a
great
example
of
of
by
trying
to
make
things
simpler.
You
kind
of
introduce
complexity
and
so
you're
kind
of
moving
the
complexity
around
right.
B
A
All
right
so,
as
we
see
five
of
these
things,
creating
a
couple
of
them
are
running
right
away,
because
probably
those
landed
on
notes
that
had
already
downloaded
the
image
now
they're
all
running
I
can
pick
one
of
these
things
and
do
do
a
cube
control
log.
Oh,
if
I
do
kale
that
should
tail
all
of
them.
That's
cool
utility
called
kale.
Okay,
so
we
saw
one
exiting
and
then
we
see
some
new
stuff
starting.
We
can
see
you
had
one
successful
and
this
should
updates.
A
Now
we
have
three
successful
and
when
they're
in
this
completed
sort
of
cleaning
up
state,
that's
enough
to
actually
said
hey,
let's
start
the
new
ones
off
and
running,
oh
and
since
I
give
the
dash
a
it's,
including
the
completed
ones
right,
and
so
now
we
have
the
next
five
running
or
for
running
one.
Two,
three
four
and
we've
completed
one
two,
three
four.
So
it's
actually
the
job
is
catching
up
with
stuff
yeah,
so
kale
is
really
cool.
A
A
Love
to
get
to
the
way
world
where
these
things
can
be
more
integrated
with
cube
control
or
people
feel
comfortable,
adding
this
stuff
to
keep
control.
But
right
now
the
the
mechanics
of
actually
getting
stuff
into
cube
control
is
a
little
bit
harder
than
I.
Think
anybody
would
like
it
to
be
so
we
have
one
that
we're
waiting
for
here
boom
it's
done
now.
A
The
whole
thing
is
done
and
those
things
are
all
completed,
and
so
you
can
imagine
that
if
you,
if,
if
you
just
want
to
do
some
work-
and
you
want
to
say,
generate
some
stuff-
you
can
go
ahead
and
do
that
the
fact
that
you
can't
prepare
ammeter
eyes
these
things.
The
fact
that
you
can't
do
something
along
the
lines
of
like
you
know,
say:
hey
give
it
the
index
of
which
number
it
is
in
terms
of
things
running.
Definitely
I
think
limits
the
usefulness
to
some
degree
of
of
this.
A
This
scenario
that
we're
running
with
here,
let
me
see
if
and
I,
don't
think
anybody
so
that
that
might
be
I
could
see,
relaxing
that
rules
of
actually
giving
an
identifier
with
respect
to
jobs,
because
I
don't
think
some
of
the
same
concerns
that
we
had
before.
But
if
something
fails
hard
right,
you
want
to
replace
it.
You
replace
with
the
same
index.
So
now
you
have
me
have
two
things
running
with
the
same
index:
people
don't
expect
that
stuff
to
actually
to.
A
Yeah
Dave
yeah-
this
is
saying
the
same
thing:
I
did
work
naive
tasks,
numbers
would
actually
so
parallelism.
Plus
completion
is
a
good
illustration
of
where
naive
tasks
numbers
would
fall
down.
If
not
all
the
pods
are
running
at
once,
yeah
exactly
all
right.
So
do
you
want
like
here's,
your
run
number
or
here's
your
instance
yeah
number.
That's
that's
the
type
of
thing
that
falls
down,
so
that's
really
interesting
there.
A
Now
one
of
the
things
okay,
so
yeah
now
I
want
to
keep
moving
here.
So
the
next
thing
we're
going
to
do
here
is
we're
gonna
go.
This
is
probably
the
most
complicated
example.
Is
we're
going
to
do
a
producer-consumer,
Q
and
so
I
had
an
image
in
the
book
that
I
grew
up?
Let
me
see
if
I
can
hold
that
up
here.
A
It's
very
simple:
we're
gonna
actually
build
it
a
little
like
in
memory
worker
Q
and
then
we're
gonna
bring
up
a
bunch
of
tasks
that
go
ahead
and
pull
from
that
worker
Q
and
when
the
queue
is
empty,
everything's
gonna,
shut
itself
down
excuses.
I
wish
I
could
give
a
portion
of
a
naming
cube
control
so
annoying
copying
and
pasting
everywhere,
yeah
yeah
for
sure
for
sure,
especially
with
the
port
forward
and
we're
gonna
get
to
that
pretty
quickly
here.
A
So
the
first
thing
that
we're
gonna
do
here
and
I,
don't
know
why
I
give
this
as
a
replica
set
before,
but
in
the
book.
But
let's
so,
if
I
look
here,
what
we
have
is
we're
gonna
have
we're
gonna
run
a
replica
set.
This
could
very
easily
be
a
deployment
also
where
we're
gonna
run
quar
D,
and
it's
going
to
do
nothing
special
and
we're
gonna
call
this
thing
this
is
going
to
be
called
our
it's
being
called.
A
A
C
A
Oh,
that's
fine!
So
let's
decode
this
a
little
bit
just
so
that
people
know
so
I'm
getting
get
pods.
There's
a
label
selector
here,
because
I
actually
had
you
know
a
P
equals
work
queue
and
component
equals
Q.
So
that's
like.
If
you
have
a
bunch
of
pods
running,
that's
a
way
that
you
can
narrow
it
down
and
then
this
thing
is
saying:
I
want
to
pull
out
the
name
of
the
first
thing.
So
dot
item
0
metadata
name
using
JSON
path,
and
so
it's
a
little
bit
complicated.
A
To
that
thing,
so
you
may
see
people
when
they're
doing
similar
stuff,
they'll
use
proxy
proxies
HTTP
and
it
it'll
actually
try
and
rewrite
some
of
the
return
content
if
it
has
a
URL,
ish
type
of
thing
in
it,
so
that
things
work
well
when
you're
proxying
you
eyes
so
I
generally
use
port
forward
when
I
can,
instead
of
proxy,
so
that
things
up
and
running
okay.
Now
the
next
thing
we're
going
to
do
here
is
we're
gonna
load.
A
Okay
and
I'll
show
you-
and
this
is
just
a
simple-
let's
see-
I
have
load
Q,
okay,
so
I
have
a
simple:
there's
got
to
be
a
bug
for
port
forwarding
of
POVs
already
there
is,
there
is
I.
Think
a
lot
of
folks
are
assuming
that
you're
going
to
do
this
stuff
server-side.
My
idea
is
just:
do
it
all
client-side
right.
A
So
that
way,
it's
like
just
pick
one
of
them,
but
I
want
to
talk
to
a
consistent
one,
which
is
different
from
the
way
that
services
work
inside
of
kubernetes.
Where
every
time
you
talk
to
something
generally
it'll
pick
a
new
one
to
talk
to
a
lot
of
times
when
you're,
using
something
like
port
forward
from
your
desktop.
You
want
to
have
that
stability
if
you're
running
a
group
of
things,
but
that's
my
side
project
that
I
have
no
time
to
work
on
okay.
A
So
here's
what
we're
going
to
do
here
so
there's
a
simple
restful
api
for
this
we're
going
to
use
curl
to
do
that.
We're
gonna
create
a
queue
called
key
key
gin
there
and
you
can
feel
free
to
look
at
the
go
code
for
this
stuff.
It's
all
up
there
and
then
I'm
gonna,
add
99
items
here.
Let's
actually
add,
let's
actually
add
1000
do,
let's
do
30,
30,
okay,
we're
30
items
and
for
each
of
those
I'm
going
to
do
a
post
to
that
to
the
Inc.
A
A
And
then,
as
it
returns
it,
it
actually
returns
a
unique
ID
for
that.
So
now,
if
I
go
back
to
core
D
and
if
we
actually
see
the
the
mem
Q
server,
here's
the
documentation
of
that
API
and
you
can
see
that
we
have
30
things
in
queued.
Okay
and
the
reason
we
have
30
s,
because
I
went
to
0
to
30
inclusive,
that's
just
the
way
that
bash
works.
This
is
a
this
is
a
bash
ism
there.
A
Then
it's
gonna
go
ahead
and
shut
itself
down.
So
now,
let's
bring
up
our
worker
worker
pool
the
consumer
job
and
let's
look
at
what
that
looks
like
so
we
have
that
job
consumers
here
and
so
here
we're
doing
parallelism
five,
but
we're
not
we're
not
actually
saying
we're
not
doing
completions
right.
So
so,
whatever
you
know,
we're
just
going
to
run
five
until
those
five
exit,
okay
and
and
then
what
we're
doing
is
key
Jenin
able
exit
on
complete.
A
So
you
can
imagine
you're
writing
code.
You
can
have
something
that
says:
here's
the
work
I
need
to.
Do
you
launch
a
bunch
of
workers.
You
have
them.
Do
that
work.
You
either
explicitly
shut
down
those
workers.
If
you're
doing
like
a
deployment
type
of
thing-
or
maybe
like
you
know
you,
you
want
to
have
them
shut
themselves
down,
in
which
case
the
the
jobs
actually
makes
sense.
A
A
I
forgot
to
create
the
service
so
yeah,
so
we
can't
talk
to
the
thing
because
we
haven't
created
a
service,
it's
just
a
pod,
you
know,
or
you
know
it's
just
a
replica
set
running
so
now
we
have
to
create
a
service
for
doing
that.
So
so
right
now,
each
of
these
things
is
retrying
in
a
loop
trying
to
talk
to
the
cue
server
and
it's
like
I
can't
talk
to
it.
So
this
is
another
interesting
pattern.
A
Is
that
when
you
start
up
a
job,
you
know
some
sort
of
like
some
sort
of
process
and
it's
trying
to
talk
to
something
else,
and
he
can't
reach
the
thing
that
it's
talking
to.
What
should
you
do?
Should
you
retry
like
this,
or
should
you
error
out
and
exit
and
there's
really
different
schools
of
thought?
I
prefer
to
actually
retry,
because
then,
when
you're
bringing
up
a
whole
bunch
of
things,
the
order
ends
up
not
mattering.
A
So
I
forgot
to
bring
up
the
service
and,
from
the
point
of
view
of
these
workers
that
that
work
queue
server
is,
you
know,
is,
is
not
there.
But,
but
what
will
happen
now
is,
as
I
bring
up
the
service
all
of
a
sudden.
These
things
will
actually
start
talking
to
it,
and
the
whole
sort
of
system
will
click
into
place.
But
right
now
the
fact
that
this
thing
isn't
making
progress
and
there's
no
errors
being
explicitly
thrown
outside
of
the
logs
is
something
that
that
you
have
to.
You
know
instrument
forward
directly
right.
A
So
if
I
were
building
a
real
production
system,
I'd
probably
do
something
like
point
Prometheus
at
this
and
I'd.
Have
it
report
errors
when
it's
trying
to
DQ
and
then
Prometheus
would
be
tracking
those
errors,
then
I
would
use
that
to
figure
out
that
something's
wrong
versus
doing
something
like
having
the
thing
exit
out
with
a
with
a
really
noisy
air.
A
Okay,
so
cube
control,
create
a
chef
and
I
believe
the
so
it's
service
queue
will
do
we're
going
to
do
that
and
then
we're
gonna
look
at
the
log
and
it
will
give
that
a
dash
of,
and
so
it's
gonna,
retry
and
now
I
think
it
started
queuing
stuff.
Maybe
if
we
go
back
here,
we
can
see
that
the
depth
of
the
queue
is
actually
and
the
number
of
things
DQ'd
is
actually
reducing
over
time.
Okay,
so
these
things
for
whatever
reason,
they're
not
actually
when
they
actually
do
a
run,
an
object,
they're,
not
Oh,.
A
Io
timeout,
interesting,
okay,
so
the
way
that
I'm
actually
doing
this
DQ
on
this
stuff.
It's
it's
hitting
this
IO
timeout
it.
This
one
got
in
a
weird
state:
okay,
now
it's
actually
D
queuing
stuff,
so
the
whole
thing
kind
of
recovered
after
the
the
service
came
back
up
and
you
can
see
that
these
things
are
all
d
queuing
over
time
and
eventually
this
thing
will
empty,
but
everything
else,
so
everything
will
still
be
still
be
working
and
we'll
give
that
a
few
more
seconds
to
go
ahead
and
do
that
and
so
you
can
go.
A
Is
this
all
available
on
your
github
to
try
and
duplicate
yeah?
So
all
the
stuff
is
in
Cordy
and
then
the
example
is
okay.
So
now,
okay,
we
see
these
things
are
moving
to
completed
because
they
tried
to
DQ.
They
got
that
everything
there's
nothing
left,
and
so
they
exited
cleanly
and
we
have
one-
that's
left
running
and
boom
he's
done
right,
and
so
this
is
a
really
useful
pattern.
A
A
There
we
go,
those
are
the
stats,
and
so
that's
the
same
stuff
that
we
were
seeing
on
the
page,
so
we
can
go
ahead
and
get
that
okay.
So
now,
if
I
go
through,
you
were
asking
if
this
stuff
is
available
online,
if
you
just
look
for
Cordy
here,
is
the
documentation
for
quark
II,
including,
like
all
these
command
line,
flags
that
I
have
here
in
the
mem,
cue
server
and
stuff
like
that.
A
A
C
A
But
I
hadn't
been
aware
of
this
issue
funny
enough.
The
person
who
filed
the
issue,
which
is
who
was
I,
worked
with
Alex
at
at
Google.
He
helped
work
on
kubernetes
and
and
some
other
GCE
stuff
DCP
stuff
earlier,
but
he's
over
at
cockroach
DB
now
so
he's
the
one
who
actually
hit
this
one
and
filed
the
issue
on
that
and
then
Nadeem
thanks
for
thanks
for
helping
out
and
then
Jay
Milliken
here
at
stripe,
who
I
think
is
also
next
Googler,
suggested
sort
of
a
workaround
for
this
issue.
A
A
But
the
use
case
here,
the
the
normal
use
case
is
really
one
of
a
main
thing.
That's
going
to
run
plus
some
helper
side
cars
where
those
side
cars
might
be
doing
some
sort
of
like
fancy
hat.
You
know,
proxying,
you
know
with
something
like
envoy
or
to
something
like
say
cloud
sequel
at
google.
Those
things
might
be
doing
logs
collection,
where
you're
actually
running
these
things
and
they're
uploading
to
the
logs
and
so
and
a
lot
of
times
those
side.
A
So
you
could
see
that
happening
with,
say,
maybe
a
shared
logs,
collector
right,
and
so
that
thing
says:
okay,
I'm
gonna,
actually
once
I
finish,
actually
making
sure
that
I
upload
all
my
logs
to
to
some
long-term
storage
if
it's
like
a
logs
collector,
whereas
if
it's
some
sort
of
proxy
and
that
proxy
may
want
to
do
a
clean
shutdown.
Also,
but
right
now,
most
sidecars
are
not
built
to
exit
cleanly
and
so
there's
no
way
to
say
hey.
A
A
A
There
are
some
proposals
here
in
terms
of
how
this
might
be
solved.
I
don't
know
if
anybody
seriously
is
working
on
it
now
so,
but
this
is
definitely.
This
is
definitely
some
pain
that
some
folks
are
feeling
so
dan
says:
I
have
the
same
problem,
which
it
was
easier
to
ask
the
cubelet
to
kill
the
pot
for
you.
I
have
built-in
cluster
services
to
help
do
this
yeah,
so
yeah,
so
I
think
I.
You
know
I,
don't
know
honestly
I'm,
not
sure
which
sig
this
stuff
rolls
up
to.
My
guess
would
be
sig
apps.
A
This
might
be
part
of
the
sort
of
workload
EP
is,
but
but
if
you're
going
to
be
using
this
with
some
sort
of
sidecar,
this
is
definitely
something
to
be
aware
of.
So
I
just
wanted
to
call
attention
to
that
all
right.
So
that's
all
the
stuff
that
I
had
planned
I'm.
Definitely
a
lot
shorter.
This
week
than
I
was
last
week,
so
I'm
feeling
good
about
that.
I
think
I
have
a
better
idea
of
how
much
stuff
I
can
fit
into
one
of
these
episodes.
A
Are
there
any
other
questions
before
I
sign
off
about
this
stuff
or
about
other
stuff?
That's
happening
in
kubernetes,
because
I'd
love
to
be
able
to
answer
that
while
I'm
here,
no,
oh
so
apartment
said,
missed
and
all
sidecars
are
just
for
the
job
kind.
So
so
this
is
really
only
most
critical
for
jobs
where
you
want
to
run
a
pod
to
completion.
The
pod
only
exits
when
all
the
containers
in
the
pod
exit
and
there's
no
way
to
say
hey.
This
container
is
the
primary
thing
when
it's
exits
kill
all
the
other
containers.
A
If
that
was
create
that
existed,
then
that
would
actually
fit
really
well
with
with
pots
I
mean
with
with
the
job
object
so
yeah.
So
hopefully
you
guys
all
got
something
out
of
that.
I
I'm
not
quite
sure
what
I'm
gonna
do
next
week
for
a
topic.
One
of
the
things
I
do
want
to
do
is
figure
out
and
I
have
to
find
some
time
to
to
figure
this
out.
A
You
know,
have
people
share
their
desktops
and
actually
you
know,
do
their
own
sort
of
demos
and
stuff
like
that,
and
as
we
do,
that
I
think
it'll
be
a
lot
easier
for
me
to
bring
people
in
to
really
show
their
stuff
off
go
into
some
depth.
There
definitely
want
to
show
off
some
of
the
the
other
stuff
that
we've
been
working
on
here
at
hefty
Oh.
A
Things
like
Ark
and
sauna
boy
have
been
on
my
list
for
a
while
too
so
yeah,
so
not
sure
what
I'm
gonna
do,
but
hopefully
I
can
get
the
the
mechanics
of
doing
that
done.
Assuming
so
yeah
well,
hangouts
on
air
is
deprecated
and
hangouts
on
on
air
has
huge
latency,
right
and
so
I
believe
hangouts
on
air
is
deprecated,
so
I'm
still
using
YouTube
for
this
stuff
and
in
YouTube
has
sort
of
lower
latency
modes.
A
But
it's
it's
a
little
bit
tricky
I
do
like
hangouts
on
air,
though
when
it
was
when
I
was
well
supported.
So
thank
you,
everybody
and
have
a
great
week.
If
you
sat
through
it.
This
long
please
go
through
and
and
click
to
subscribe
and
give
it
a
thumbs
up,
and
all
that
I
appreciate
that
and
I'll
see
you
all
next
week.
All
right
have
a
good
one.