►
From YouTube: (k)Native K8s CI/CD platform - Shash Reddy, Pivotal
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
(k)Native K8s CI/CD platform - Shash Reddy, Pivotal
In this talk we will discuss how to use the Knative Pipeline project to configure and run CI/CD pipelines for your Kubernetes applications. You’ll be able to manage and observe CI/CD objects with kubectl the same way that you would interact with Kubernetes resources. We will dive into the major features Knative Pipeline supports and how to use them. This talk will include - How to build pipelines. - How to run one-off tasks. - How to leverage the K8s ecosystem while running Pipelines for diagnosis and observability. - Demonstrate a couple of use cases like an application developer pushing source code building images deploying application on their cluster without any external CI/CD resource. This talk proposal is also submitted to Kubecon Europe 2019.
https://sched.co/NrmR
A
A
So
I
appreciate
everyone
being
here
understand
we're
at
the
tail
end
of
the
conference
and
we
all
want
to
go
home
and
relax
so
bear
with
me
for
the
next
30
minutes
and
we'll
be
a
step
closer
to
that.
Let's
jump
into
the
agenda
of
the
talk
today
today.
The
first
thing
I'm
going
to
talk
about
is
the
terminology.
So,
let's
make
sure
we're
on
the
same
page
about
what
the
CI,
what
is
CB
and
so
because
I'm
going
to
be
using
this
throughout
this
presentation.
A
The
next
section
is
why
kubernetes
fits
the
bill
for
building
CI
CD
platform
on
then
very
briefly,
I'm
going
to
touch.
How
are
we
accomplishing
this
and
for
the
majority
of
the
presentation
I'm
going
to
be
concentrating
on
different
resources
and
the
product
itself?
I
do
have
a
demo
plan.
However,
the
live
demo
did
not
work
out
as
well
as
I
expected,
so
I
have
a
recorded
of
the
demo,
and
I
will
play
that
I
hope.
I
will
end
this
presentation
in
time.
A
A
A
Kubernetes
actually
provides
this
for
free.
If
you
use
this
because
kubernetes
has
battle-tested
over
the
last
few
years
to
come
up
with
a
good
scheduling,
algorithms
and
multiple
ways
on
how
you
can
tune
the
scheduling
algorithm
as
well
observer
booty.
So
there
are
two
aspects
of
observability.
The
first
one
is
as
a
platform
operator.
You
want
to
ensure
that
your
CI
software
is
up
and
running
at
any
given
point,
and
you
want
to
manage
that.
The
second
is,
as
a
user
of
a
CI
CI
platform
you
want
to
access.
A
The
logs
of
your
built
kubernetes
by
itself
does
not
provide
both
of
these
options,
but
it
has
been
around
long
enough
that
there
are
ton
of
Suites
of
products
which
help
you
with
the
logging
solution,
tracing
metrics
alerts.
So
as
a
platform
operator
you
can
plug
in
and
play
any
of
these
vendor
software's
out
there
to
get
the
observer
booty
needs
that
would
have
cloud
platform
agnostic.
This
applies
to
any
piece
of
software.
A
Nobody
wants
to
be
tied
down
to
a
cloud
vendor
and
kubernetes
provides
a
great
abstraction
layer
on
top
of
cloud
vendors,
where
you
can
move
your
platform
from
one
kubernetes
to
the
other
without
being
tied
to
in
wonders,
identity
management.
A
lot
of
the
CICE
builds
have
contained
very
sensitive
information,
and
you
want
to
ensure,
as
an
operator
that
right
people
have
access
to
the
right
resources
and
kubernetes
provides
great.
A
What
does
this
mean
in
the
case
of
CI
CD
world?
The
desired
state
of
the
object
is
I
want
my
CI
job
to
be
completed.
What
is
the
current
state?
Current
state
represents.
What
is
the
list
of
actions
that
I
want
to
perform
and
the
reconciliation
loop
or
the
controller's
here
continuously
try
to
reconcile
your
given
set
of
actions
to
ensure
it
is
completed,
whether
it's
successful
or
failed?
It
depends
on
what
the
actions
itself
are
so,
but
the
goal
is
to
reach
the
completion
stage.
A
It
originated
in
a
native
organization
as
Canada
built,
and
then
it
transformed
to
Canada
build
pipeline
and
very
soon
the
scope
of
the
project,
outgrew
Canada
and
it
spun
off
into
a
separate
organization
and
has
now
found
home
in
CDF,
continuous
delivery
foundation,
and
it's
actually
part
of
the
Linux
Foundation
umbrella,
all
right
so
I'm
going
to
dive
into
individual
CR
IDs
that
are
defined
in
this
project
and
what
they
mean.
How
can
you
configure
them,
and
hopefully
you
can
get
information
on?
How
can
you
construct
a
basic
pipeline
using
protect
on
CD
resources?
A
These
are
the
currently
supported
types
of
pipeline
resources
intact
on
City
project
today,
I
think
probably
I'm
going
to
just
briefly
mention
github,
as
this
is
the
most
common
resource
that
we
have
all
come
across.
So
we
can
relate
much
easily
so,
as
you
define
a
github
pipeline
resource
type,
let's
look
at
sample
yamo
for
clarification,
so
the
type
here
refers
to
what
the
type
of
the
pipeline
resource
itself
is
and
then
that
next
section
is
the
parameters.
A
Parameters
are
the
defining
characteristics
of
the
pipeline
resource
itself
and
in
the
context
of
get
this
refers
to
the
revision,
which
corresponds
to
what
is
the
branch
name
or
it
could
be
a
comet
shell
or
it
could
even
be
a
version
tag
and
the
second
one
is
the
URL.
Where
is
the
project
actually
present?
A
So
it
can
actually
fetch
that
and
the
parameters
are
going
to
change
depending
on
what
resource
you're
talking
about,
for
example,
in
the
context
of
docker
image,
this
is
going
to
be
an
image
digest
and
this
is
going
to
be
a
URL
which
corresponds
to
where
is
the
registry
in
which
the
docker
image
is
hosted
all
right
tasks
early.
This
is
the
next
CRD.
A
So,
like
I
mentioned,
it
can
have
number
of
input
resources
and
all
these
steps
are
executed,
serially
and
as
an
artefact
or
an
output.
You
can
have
a
number
of
output
resources
that
are
produced
as
a
result
of
execution
of
these
steps,
I
like
to
think
of
tasks,
er
D
as
a
template,
where
you
dictate
what
the
input
parameters
and
the
resources
and
addition
of
the
task
resource
itself
does
not
actually
create
any
kubernetes
resources.
There
is
no
custom
controller,
which
is
watching
the
task,
resources
and
act
upon
it.
A
So
I'm
going
to
explain
the
next
next
part
of
the
presentation
how
this
template
is
being
used.
So
let's
take
a
sample
task
and
let's
try
to
see
how
does
the
yama
look?
So
in
this
example,
the
input
is
a
github
repository
and
it's
expecting
an
input
get
resource.
The
task
itself
is
building
an
image
to
take.
The
source
code
builds
an
image
and
the
output
artifact
that's
produced
out
of
this
task
is
an
output,
a
docker
image.
So
this
is
the
example
llamo.
A
So
the
first
section
is
inputs,
so
inputs
by
itself
is
there
in
two
parameters
and
the
resources
like
I
mentioned.
This
is
a
template,
so
this
dictates
what
are
the
input
parameters
and
as
someone
who's
trying
to
configure
this
template
if
the
default
value
is,
if
the
value
is
different,
they
can
override
the
default
value.
But
if
they
are
following
this
pattern,
where
the
docker
file
is
present
at
any
mentioned
location,
they
don't
have
to
configure
the
parameters.
A
The
second
section
talks
about
the
resources
and,
if
you
see
here,
it
only
mentions
what
is
the
expected
type,
it
does
not
actually
bind
the
pipeline
resource
itself
very
similar
in
the
output
resources.
It
mentions
what's
the
type
here,
but
it
does
not
bind
the
pipeline
resource.
The
next
section
we
have
steps
and
each
step
corresponds
to
a
container
definition.
You
can
have
as
many
steps
as
you
want,
but
in
this
example,
I
only
have
one,
and
in
this
example,
I
am
building
docker
image.
A
Using
koneko
koneko
is
an
open
source
tool
by
Google
for
building
docker
images
from
source,
and,
if
you
notice
here,
the
docker
file
actually
reads
the
parameters
using
this
template
eyes
duay.
So
if
you
actually
override
the
value,
you
don't
have
to
change
these
steps
because
it
picks
up
the
change
using
the
value
all
right.
Let's
look
at
tasks
on
CRD,
so
this
is
where
the
meat
or
the
custom
controllers
are
continuously
watching,
because
task
template
is
used
here
and
task
runs
here.
A
Are
the
binds
being
put
pipeline
resources
and
if
there's
any
power
output
pipeline
resources,
it
catches
them,
and
this
result,
oh,
and
also
if
the
pipeline
resources
requires
any
identification
like.
If
you
have
a
github
repository,
which
is
private,
it
attaches
the
service
account
to
provide
identity.
And
what
does
this
end
up?
Creating
it
ends
up
creating
a
pod.
A
So
this
translates
to
a
pod
service
account
and
all
the
input
pipeline
resources
are
mounted
on
a
shared
volume
under
workspace
and
all
the
steps
defined
in
the
tasks
will
be
executed,
serially
to
ensure
the
workspace
resources
and
once
the
steps
have
finished,
the
execution
output
pipeline
resource
will
be
executed.
So
let's
take
an
example
where
you
have
zip
artifact
generated
as
an
output
of
the
steps.
So
at
the
end
of
all
the
steps
execution,
your
output
pipeline
resource
will
update
or
upload
the
artifact
to
a
cloud
storage.
A
This
is
a
sample
llamó,
where
we
are
binding,
actual
resources
and
creating
kubernetes
resources.
So
the
all
right,
sorry,
the
first
section
refers
to
the
task
reference.
So
this
is
the
same
Kaneko
task
we
saw
previously,
which
is
a
task.
The
next
section
is
where
it
binds
so
I
mentioned
here.
I
have
the
template
on
my
right
hand
and
I?
Have
the
runtime
object
on
my
left
hand?
So
if
you
notice
here
this
is
what
the
template
looks
like
it
mentions
that
it
needs
a
pipe
get.
A
The
runtime
object
actually
binds
the
my
app
source,
which
is
a
pipeline
resource
as
an
input
very
similar
fashion,
output
resources
where
the
output
is
of
type
image
and
the
output
resource
runtime
configuration
actually
binds
the
actual
image
if
I'm
lost
in
the
mo
so
far,
I
have
a
summary
which
probably
provides
a
bigger
picture.
So,
as
a
user,
when
you
submit
the
tasks,
run
Yama
file,
kazran
controller,
which
is
continuously
watching
these
resources.
A
Well,
that's
the
task
CRD,
which
is
your
template,
object
and
validates
all
the
requested
parameters
and
the
resources
from
the
template
is
provided
in
the
runtime
configuration
and
if
that
matches
perfectly
tas
room
controller
goes
ahead
and
fetches
all
the
input
pipeline
resources
builds
a
pod
executes
all
the
steps
and
once
the
step
has
finished
the
execution,
it's
going
to
upload
your
pipeline
resources,
whatever
you
have
defined
I,
do
have
a
demo
to
show
this
in
action,
and
the
demo
uses
all
the
3cr
these
I've
described
so
far.
So
I'm
going
to
play
the
video.
A
So
for
context,
this
demo
is
using
kiiis
cluster
kia
trees
from
the
Rancher
labs
and
I
have
installed
it
on
my
Mac
and
I
have
not
sped
up
the
video.
So
hopefully
this
shows
how
lightweight
the
CI
CD
layer
is
on
top
of
kubernetes,
and
this
can
be
can
perform
powerful
actions
using
something
very
simple.
So
I
have
a
hello
world
application
very
simple
server,
which
is
has
a
handler
for
printing
hello
world.
A
A
Don't
have
anything
installed
on
my
Cooke
kubernetes
cluster,
no
pods,
nothing
in
any
of
your
namespaces.
This
is
the
installation
of
the
Tecton
city
resources,
I'm,
applying
the
resource
and
the
task
camel
itself
so
for
context.
I'm,
using
the
same
example
that
I've
described
so
far,
where
I
have
a
github
resource
and
I
have
an
output
which
is
docker
image,
resource
and
I
have
tasks,
which
is
only
one
task
which
is
referring
to
the
Kaneko
and
applying
the
tasks
run,
which
is
actually
going
to
bind
all
of
these
resources
together
and
watching
the
pot.
A
So
this
is
where
I've
actually
edited
the
video,
because
building
an
image
and
uploading
takes
about
two
and
a
half
minutes.
So
I
have
cut
that
part
of
the
video,
and
you
can
clearly
see
that
it's
just
two
and
two
enough
minutes,
and
it
didn't
take
that
long,
alright.
So
here
voila
we
have
any
much
and
that's
the
end
of
my
demo.
A
All
right
so
here
we
talked
about
pipeline
CRD
tasks,
Urd
and
tasks
run
CR
D.
So
this
is
a
very
small
execution.
You
need
a
in
enterprise
building
pipelines.
You
need
something
a
much
more
because
the
pipelines
are
going
to
be
complicated.
You
need
to
have
a
fun
in
fan-out
structures,
you're
going
to
have
multiple,
parallel
suites
that
you
want
to
run
so
pipeline
security
sits
at
a
layer
above
the
task
and
tasks
run
so
I.
Think
of
pipeline
as
a
template,
which
is
orchestrating
multiple
tasks.
A
Let's
take
a
simple
pipeline
where
we
have
a
first
task,
which
is
running
tests
and
the
second
task,
which
is
actually
building
the
image
that
we
rally
familiar
with,
and
the
output
of
this
pipeline
is
going
to
be
building
the
docker
image.
Let's
look
at
some
llamo.
The
first
part
is
defining
the
resources
very
similar
to
how
tasks
here
it
is
a
template
pipelines
here.
It
is
also
a
template,
it
does
not
actually
bind
any
pipeline
resources,
so
here
it
talks
about
two
types.
One
is
get
an
image,
because
those
are
the
same.
A
Two
resources
were
still
using
in
our
pipeline.
The
second
section
is
where
we
define
the
tasks,
so
the
first
one
is
where
we
want
around
the
unit
test,
and
we
refer
to
a
task
called
as
unit
test
which
dictates
what
the
step
is
and
we
define
what
the
input
resources
are,
and
next
section
is
building
the
app
image
itself,
and
here
is
the
important
part
which
is
run
after
which
dictates
the
flow.
So
once
this
pipeline
see
are
these
presented,
it
actually
construct
a
direct
acyclic
graph.
A
So
the
dag
is
constructed
based
on
the
keyword,
so
it
decides
the
flow
of
the
task
or
dependent
on
the
run
after
so,
the
first
unit
test
did
not
have
any
dependencies
that
will
be
executed.
First,
all
right
so
pipeline
runs.
Grd
very
similar
has
is
a
run,
configuration
that
uses
pipeline
and
bind
the
pipeline
resources
and
the
pipeline
run
controller
creates
a
number
of
tasks,
runs
depending
on
what
the
flow
is
expected
again.
Going
through
some
yeah
mole
refers,
what
the
pipeline
is
service
account
is
how
you
attach
the
identity
for
the
pot.
A
In
next
section,
you
actually
bind
the
resources
again
very
similar
to
what
we
see
in
tasks
run
where
this
actually
references
the
actual
pipeline
resources.
If
I
have
lost
you
already
in
the
yamo
again,
I
have
a
summary
slide,
which
hopefully
puts
them
all
together
in
a
good
picture.
As
a
user,
when
you
submit
the
pipeline
running
amyl
pipeline,
run
controller
watches
for
these
resources
and
very
similarly,
it
takes
in
the
pipeline
ERD
checks.
What
is
the
expected
resources
and
checks
whether
the
run
time
object
has
provided
all
the
resources
if
at
all
matches?
A
Well,
the
pipeline
controller
now
tries
to
find
out
series
of
eligible
tasks
runs
which
can
be
spun
off
now
because
it
does
not
have
any
dependencies,
so
it
is
responsible
for
creating
task
runs
and
once
the
task
runs
go
through
their
life
cycle.
The
pipeline
run
controller
updates.
The
status
itself
to
ensure
the
pipeline
run
is
whether
it's
completed
or
it's
not
completed,
so
it
continuously
watches
for
the
pipeline
run
sales.
Alright,
I,
don't
know.
A
If
you
have
actually
got
this
question
so
far,
but
each
task
run
corresponds
to
a
pod,
so
how's
the
information
shared
between
multiple
pots.
The
answer
is
PVC
PVC
stands
for
a
persistent
volume
claim
and
pipeline
run
manages
the
lifecycle
of
PVC,
where,
after
execution
of
each
task
run,
the
state
of
the
world
of
the
pod
is
transferred
to
a
PVC
and
before
the
next
task
starts,
the
world
is
transferred
from
the
PVC
to
the
pod
before
the
execution
so
and
very
soon
we
realize
not.
A
Alright,
how
are
we
doing
at
the
time?
I
think
we
have
about
10
minutes.
So
if
you
have
any
questions,
I'm
happy
to
take
I
will
leave
the
references
slide
up
here.
I
have
the
link
for
the
project,
documentation
and
there's
a
catalog
of
tasks.
If
you
want
to
build
something
new
and
you
want
to
use
as
reference
there's,
a
list
of
tasks
that
you
can
reference
from
a
fun
like
I
learned
very
recently
that
Tecton
project
releases
using
Tecton
CD
resources
itself.
A
A
A
C
Hi,
so
we
have
in
Jenkins
X
a
similar.
We
have
a
simple
syntax.
The
Jenkins
users
are
more
familiar
with,
but
it
all
generates
pipeline
sorry,
Tecton
resources
underneath.
So
all
the
Tecton
tasks
task
runs
pipeline
resources,
everything
we
generate
from
Jenkins
X
camera.
So
it's
really
up
to
you
for
whether
you
want
to
use
Tecton
resources,
vanilla,
either
Tecton
project
using
themselves
and
many
other
people
are
using
themselves
or
but
for
Jenkins.
At
the
point
of
view,
we
wanted
to
create
a
syntax
that
was
familiar
to
existing
Jenkins
users.
C
D
A
B
B
A
Okay,
so
I
think
the
question
is
more
about
what
level
should
be
considered
using
this
we
want
to
as
a
user,
should
we
build
pipelines
at
the
level
of
Tecton
CD,
or
should
we
use
something
like
junk
in
X,
which
is
at
a
layer
above
it
generates
Tecton
city
resources
itself.
It
is
completely
up
to
you
because
as
a
user,
if
you're
already
familiar
with
Jenkins
X
style
of
writing
pipelines,
then
you
can
continue
to
do
so
using
Jenkins
X.
A
E
A
Sure,
if
I'm
understanding
a
question
correctly
or
it
is
going
to
be
like
an
ecosystem
of
integrations
that
are
going
to
be
using
Tecton
project
goal
is
yes
definitely
to
have
pectin
CD
as
like
the
underlying
running
engine
for
the
CI
CD
resources
and
other
CI
CD
products
can
translate
its
yellow
into
pectin
seeding
pipelines.
That
will
be
the
end
goal.
E
F
Hey
so
I
think
the
question
you're
asking
is
about
will
Tecton,
add
features
and
tools
which
are
also
provided
by
Jenkins
X,
and
then
people
maybe
will
have
to
choose
between
using
one
or
the
other,
or
maybe
it
would
even
obsolete.
Some
of
the
things
that
are
offered
by
Jenkins
X
is
that
sort
of
what
you're
asking
yeah
so
I
think
the
goal
is
that
Tecton
would
be
building
blocks,
that
Jenkins
X
would
be
consuming
so
and
James
is
nodding
vigorously.
So.
F
Is
we
have
this
catalog
that
shash
linked
up
there
a
catalog
of
tasks,
and
we
really
want
to
get
to
a
point
where
people
can
choose
items
from
that
catalog
and
they
can
use
it
with
Jenkins
X
or
with
any
other
system.
That
is
a
using
Tecton
underneath.
So
hopefully
you
wouldn't
be
choosing
between
them.
It
would
be
more
like
you
can
use
the
features
of
Tecton
with
Jenkins
X
or
whichever
CI
CD
product
you're,
using.
D
A
So
the
first
question
is
sorry.
The
first
question
was
about
the
performance
of
the
pipeline,
so
performance
of
the
pipeline
totally
depends
on
what
the
pipeline
itself
is,
because
if
you
have
a
series
of
parallel
tasks,
then
you
will
have
all
the
task
runs
that
are
spin
up
parallely,
so
obviously
you're
going
to
get
the
most
performance.
So
there's
no
way
to
measure
the
performance,
because
there
is
no
single
pipeline
out
there,
which
can
say
this.
It
can
execute
within
10
seconds
and
we
can
measure
against
different
CI
CD
products.
A
A
A
Yeah
I
understand
your
question
tasks,
so
technology
D
projects
provides
like
resources
to
build
the
pipeline
and
what
you're
asking
is
like
the
next
level
of
intelligent
deployments
of
the
application.
So
this
is
another
problem
or
another
use
case,
which
is
solved
by
spinnaker
and
again.
Spinnaker
actually
is
under
the
same
foundation,
which
is
targeting
a
different
set
of
the
problems
and
tech.
Con
city
is
not
solving
that
problem
as
address.
H
Right
so
you're,
using
like
multiple
CRT
to
light
for
fat
as
in
pipeline.
How
do
you
get
a
get?
Only
one
place
to
find
a
law
find
every
cell
of
the
things
to
make
sure
that,
like
if
something
fell,
fell
in
the
middle
of
the
pipeline.
How
do
you
find
if
I
were
where
they
said,
because
it's
like
different
container
different
logs
yeah.
A
So
each
pipeline,
when
it
creates
tasks,
runs
it
attaches
using
the
owner
reference,
so
you
can
actually
track
which
tasks
runs
are
created
by
individual
pipeline
and
individual
task
runs
create
pots
and
it
attaches
its
own
on
a
reference
to
the
pot.
So
if
something
fails,
you
can
actually
track
back
using
the
parent
references
which
pod
failed
in
which
container
filled
and
the
status
of
the
object
itself
will
hold
information
on
what
is
the
pod
and
what
is
the
container
which
actually
failed
for
you.
H
A
Okay,
because
if
you
have
a
pipeline
which
is
going
to
spin
off
multiple
pots,
then
multiple
pots
are
this?
Give
any
given
point
want
to
access
the
PVC,
then
yes,
but
read/write
is
required,
and
there
wasn't
other
reasons
why
the
added
support
for
cloud
storage,
because
it
was
much
more
easily
solved
at
those
storage
bucket
level.