►
From YouTube: Lightning Talk: Leveraging GitOps - How Red Hat Delivers - Yashvardhan Kukreja & Ashish Narayanan
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Lightning Talk: Leveraging GitOps - How Red Hat Delivers Managed Layered Services on Kubernetes - Yashvardhan Kukreja & Ashish Anand Narayanan, Red Hat
Red Hat's Managed Services ecosystem is heavily backed by GitOps. All the way from defining the core bundles and manifests of those services, to validating them, to propagating them through the upstream Openshift Cluster Manager's Cluster Service, to delivering and deploying them across the entire fleet of all of the customer's Openshift Dedicated clusters, it's all governed and lifecycled by airtight GitOps tooling. By leveraging technologies like Jenkins, TektonCD, Terraform, handwritten GitLab bots and many more, Red Hat has carved this seemingly complicated yet beautiful at its core ecosystem of delivering its Managed Services in an extremely resilient, majorly self-serviceable and reconciliable pattern to its customers. In this talk, Yashvardhan and Ashish would like to share the architectural insights and learnings about this GitOps-powered ecosystem, how was it implemented and how do all these pieces fit together to deliver Red Hat's Managed Layered Services to the customers seamlessly.
B
Hello,
everyone,
I'm
Ashish
I'm,
also
a
site
reliability
engineer
at
Red
Hat
like
yes,
I,
also
love
kubernetes
and
in
my
free
time,
I
love,
learning
new
programming
languages
when
I'm
not
working
I
love
to
go
out
hiking
in
the
mountains.
So
enough
about
us.
Let's
talk
about
managed
services,
so
the
core
of
managed
services
at
Red
Hat
revolves
around
our
platform,
which
is
called
the
openshift
dedicated,
so
operative
dedicated
is
Bangla,
kubernetes,
plus
a
bunch
of
operators
and
web
books,
which
makes
this
version
of
kubernetes
is
called
as
openshift
right.
B
So
now,
the
unique
USB
of
this
platform
is
that
customers
can
install
additional
software
on
top
of
their
clusters.
So
we
call
these
additional
software's
patterns
and
we,
the
mtsra
team,
is
on
the
hook
for
delivering
and
supporting
these
add-ons
under
the
hood.
These
add-ons
are
nothing
but
an
opiniated
way
of
packaging.
Kubernetes
controllers
go
to
the
mixer
yeah
managing
the
add-ons
available,
so
as
a
necessary
team
responsible
for
add-ons,
we
use
githubs
to
smoothly
deliver
our
add-ons
end-to-end,
that
is,
from
the
product
developers
right
to
the
customers.
B
So
how
do
we
do
that?
We
have
a
couple
of
repositories
which
include
the
management
repository
and
the
management
together.
These
two
repositories
store
the
entire
state
of
all
the
add-ons
talking
about
the
manage
tenants,
boundaries
repositive,
this
repository
houses,
all
the
artifacts
of
all
the
add-ons.
So
what
I
mean
by
artifacts
is
that
the
deployment
of
the
add-ons
operator,
the
crds
that
come
with
the
operator,
the
stateful,
sets
and
other
kubernetes
resources.
It
can
also
have
dependencies
with
which
the
add-on
depends
on
this
operator.
You
can
also
specify
that
here.
B
One
other
thing
about
this
repository
is
that
each
add-on
artifact
this
version
in
that
a
particular
version
of
the
add-on
has
its
own
directory
and
its
artifacts
it
inside
that
Direct.
Now
talking
about
the
managed
tenants,
the
first
three,
this
the
first
three
houses,
all
the
configuration
and
metadata
of
an
add-on.
So
what
do
I
mean
by
metadata
things
like
the
name
of
the
add-on,
the
image
of
the
add-on,
the
namespace,
where
the
add-on
has
to
be
installed
and
also
it
can
have
constraints
the
runtime
constraints.
B
Like
you
know,
this
add-on
can
also
be
installed
on
clusters,
where
this
amount
of
RAM
is
present,
and
things
like
that.
Finally,
we
have
pipelines
which
do
the
Plumping
and
communication
between
these
two
reposites
over
to
the
next
slide.
Yeah.
Now,
let's
take
a
look
at
the
UI
which
the
customer
sees
when
they
have
to
go
and
install
an
add-on
in
the
cluster.
A
Oh
thanks
Ashish,
sir,
this
screenshot,
which
folks
are
seeing
in
front
of
you.
This
is
how
a
cluster
looks
to
a
customer
when
they
install
an
open
shift
dedicated
cluster
on
the
radar
Cloud.
So
this
UI
offers
a
lot
of
capabilities
to
fiddle
with
the
cluster
without
even
opening
up
your
terminal
and,
as
you
can
see
as
a
part
of
this
UI,
there
is
an
add-ons
tab
which
is
selected,
and
this
tab
essentially
shows
the
entire
catalog
of
all
the
add-ons
which
a
customer
can
install.
A
Finally,
installing
an
add-on
is
as
simple
as
just
clicking
on
one
of
these
thumbnails
and
clicking
on
the
install
button,
and
you
are
done.
You
can
install
an
add-on,
a
fully
pledged
add-on
and,
if
there's
a
complicated
add-on
to
install,
which
requires
a
bunch
of
input
parameters.
Well,
the
UI
would
prompt
you
to
provide
those
parameters
through
a
through
an
interactive
form,
so
very
simple
process
to
install
an
add-on,
but
the
main
stuff
happens
behind
the
scenes
right.
So
let's
dive
right
into
it,
so
so
don't
get
feed
out
with
this
diagram.
A
This
is
much
more
simpler
than
it
looks,
so
you
might
be
noticing
a
lot
of
pillars
flying
around
here
in
this
diagram,
but
they
are
essentially
meant
to
emphasize
on
how
abstracted
and
automated
things
are
from
the
eyes
of
both
the
customer
and
the
core
add-on
developer
with
the
power
of
githubs.
So,
first
of
all,
let's
dive
into
the
bottom
part
of
this
diagram,
which
represents
the
add-on
release,
lifecycle
and
obviously
the
customer
has
no
visibility
into
into
this
thing.
A
Only
the
add-on
developers,
the
internal
Red
Hat
add-on
developers
have
visibility
into
the
add-on
release
life
cycle.
So
the
only
step
they
take,
the
only
manual
step
they
take
the
add-on
developers.
They
only
raise
one
pull
request
containing
the
add-on
artifacts
like
the
deployment
animals,
the
custom
resource
definitions
and
whatnot
to
the
managed
tenants,
bundles
repository,
which
actually
shocked
about,
and
we
have
continuous
integration
jobs
to
ensure
that
these
bundles.
This
pull
request
is
absolutely
valid
and
compliant
with
our
ecosystem
before
it's
merged.
A
Then
the
next
step
is
to
propagate
the
link
to
this
newly
pushed
Docker
image
to
the
managed
tenants
repository
so
just
a
quick
revision,
the
managed
sentence
repository
as
Ashish
explained
right.
It
houses
the
configuration
and
metadata
of
all
the
add-ons
which
includes
the
name
of
the
add-on,
the
display
pick
of
the
add-on
parameters
of
the
add-on
Etc.
So,
as
a
part
of
all
this
configuration,
there
is
also
supposed
to
be
a
reference
to
the
artifact
image
of
the
add-on,
and
that's
exactly
what's
going
to
happen
in
this
step.
A
The
the
link
of
the
newly
pushed
image
will
be
propagated
to
be
to
this
managed
sense,
repository
and
once
this
propagation
is
successful,
are
again
our
automation,
our
gitlab
Bots.
They
synchronize
the
entire
state
of
the
managed
tenants
repository
with
the
ocm
now.
What
is
ocm
ocm
is
the
openshift
cluster
manager,
and
what
does
it
do?
Well,
it
serves
our
customer
and
well.
How
does
it
serve
our
customer?
Well,
the
UI
is
for
it.
This
is
ocm.
Well,
this
UI
plus
the
back
end,
which
Powers
it
that
is
OC
so
yeah.
A
So
whenever
a
customer
clicks
on
the
install
add-on
button
in
the
UI
ocm
captures
that
event
and
it's
like
okay,
cool
I'll,
install
the
add-on
100
plus
on
your
cluster,
but
it
doesn't
do
that
directly.
It
actually
offloads
this
job
of
add-on,
installation
to
one
of
our
another
internal
component
called
Hive.
And
Hive
is
basically
the
component
which
is
like
a
grandparent
of
all
the
customer
clusters
in
the
world
and
it
essentially
orchestrates
the
add-on
installation
for
all
of
our
customers.
A
So
Hive
is
essentially
Cube
CTL,
applying
all
those
all
those
artifacts.
All
those
add-on
manifests
on
the
customer
cluster,
and
not
only
that.
It
also
ensures
that
if
the
add-on
is
fiddled
around
with
the
customer
cluster,
it's
reconciled
back
to
the
desired
sales
matching
exactly
what
is
present
in
our
repositories
and
what
ocm
is
aware
of.
So
that's
the
beauty
of
this
entire
ecosystem,
powered
by
githubs
right
I
mean
things
are
abstracted
as
much
as
possible.
Most
of
the
things
are
automated
I
mean
you
can
look
in
the
diagram.
A
Most
of
the
text
is
green,
colored
and
well.
Green
color
in
our
case
represents
automated
steps
and
the
only
manual
steps
are
like
one
step
per
person.
So
a
customer
takes
the
one
manual
step
of
install
clicking
on
the
install
add-on
button
and
the
add-on
developers.
They
only
take
one
manual
step
of
raising
the
pull
request.
That's
it.
Everything
else
is
automated
and
abstractable
so
I
believe
that's
the
power
of
git
Ops
and
shedding
more
light
upon
how
github's
Appliance
compliance
is
achieved.
B
Yeah,
let's
talk
about
how
we
achieve
the
job,
compliance
with
our
add-ons
delivery
flow.
As
we've
talked
about
earlier,
the
entire
declarative
state
of
all
the
add-ons
is
kept
in
the
managed
tenants
and
management
is
Repository
and
as
again,
we
talked
about
this
earlier
version.
It's
all
the
add-on
artifacts
our
version
and
it's
immutable.
B
Add-On
developers
are
forced
to
have
new
tax
releases
or
major
releases
if
they
want
to
push
new
releases
right.
So
it's
immutable
there.
Unfortunately,
it's
not
pulled
automatically
at
the
moment.
We
need
a
manual
trigger
which
in
turn
invokes
our
Bots,
which
would
would
the
push
all
or
add-on
State
into
ocm
and
procure
continuous
reconciliation.
We.
What
we
have
right
now
is
sort
of
like
a
lazy
reconciliation.
B
Whenever
a
change
happens
in
a
repository,
we
take
the
entire
state
in
the
managed,
tenants
and
management
repository
and
push
it
into
a
platform
instead
of
just
isolating
that
change
to
a
particular
alarm.
So
in
that
way
we
have
continuous
reconciliation.
There
is
already
work
in
progress
to
make
this
a
true,
truly
continuous
reconciliation,
with
periodic
reconciliation
against
the
flux.