►
From YouTube: Building Catalogs of Operators for OLM the Declarative Way - Joe Lanford & Anik Bhattacharjee
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Building Catalogs of Operators for OLM the Declarative Way - Joe Lanford & Anik Bhattacharjee, Red Hat
This presentation will cover an update from the Operator-lifecycle-Manager project - a suite of production-ready operators and tools that help manage the complexity of installing and upgrading operators on a Kubernetes cluster. This talk will focus on architectural updates to the set of tools that are used to build and maintain Catalogs of Operators, the goals and motivation behind the changes, the lessons learned using the old tools in production, and the future of OLM. Interested in joining the OLM community? We have weekly meetings every Thursday — check out https://github.com/operator-framework/community/blob/master/wg-olm/README.md to get started!
A
A
In
today's
presentation
I
will
be
going
over
a
short
refresher
on
what
an
operator
is
then
I'll
be
covering
a
short
refresher
on
what
the
operator
lifecycle
manager
project
is
about.
I
will
then
talk
about
what
oem
catalogs
are
and
then
discuss
what
the
current
catalog
building
process
looks
like
I'll,
also
be
showcasing.
Some
of
the
pain
points
involved
in
building
catalogs
in
the
current
imperative
way.
A
A
A
These
features
are
enabled
by
allowing
operators
to
express
dependencies
on
specific
platforms
and
on
other
operators
together,
together,
with
the
help
of
rich
update
mechanisms,
to
keep
operators
up
to
date
automatically
via
over-the-air
updates
to
catalogs
so
catalogs.
What
are
they
to
discuss?
Catalogs?
We
first
have
to
go
over
what
an
operator
bundle
is
an
operator
bundle
is
essentially
all
the
that
defines
the
bundle
that
defines
the
operator
version
packaged
in
a
directory.
So
what
do
I
mean
by
the
manifest
that
defined
the
bundle?
A
The
metadata
directory
contains
application
metadata,
including
the
name
of
the
opera
version,
the
version
and
and
package
information,
including
dependencies,
so
the
dependencies
that
I
mentioned
on
on
on
platforms
and
other
other
operators.
They
are
expressed
through
the
dependencies
on
the
aml
file.
A
A
So,
like
I
mentioned
in
in
olm,
you
can,
you
can
have.
You
can
mention
upgrade
graphs
for
your
operator.
For
example,
the
lcd
v
dot
0.6.1
can
upgrade
to
v.0.6.2,
and
so
in
the
operator
bundle
table.
You
would
have
these
two
bundles
stored
and
in
the
channel,
for
example,
you
can
mention
that
both
those
bundles
belong
in
the
alpha
channel.
A
So
in
the
channel
table
we'll
have
the
information
about
the
alpha
channel
in
the
channel
entry
table
will
have
information
about
the
both
the
versions
belonging
to
the
alpha
channel.
So,
basically,
all
of
the
information
that
is
required
to
define
each
bundle
and
the
relation
between
other
bundle
versions
for
your
operator.
A
This
database,
the
sqli
database,
is
then
packaged
into
a
runnable
container
image.
So
you
can
see
an
example
of
the
docker
file
here.
It's
basically
this
index
star
database
that
is
being
copied
and
pasted
into
the
container
image
and
and
the
contents
of
this
database
is
then
served
with
a
with
a
command
that
we'll
talk
about
more
in
the
next
few
slides
over
grpc
api.
A
A
Now,
how
is
this
database
created?
So
the
operator
registry
project
also
component
of
the
operator
framework,
is
where
we
originally
house
all
the
tools
that
were
used
to
create
and
manipulate
this
database,
so
the
project
started
off
with
and
it
with
a
array
of
binaries.
It
had
the
initializer
binary,
the
register,
server,
configmap
server
and
all
of
this
did
their
own
job.
A
A
So
the
way
opm
started
off
was
it
was
pretty
simple
in
the
beginning:
it
had
the
registry
sub
command
and
the
index
up
command
and
both
of
those
had
the
add,
rm
and
serve
command.
The
ad
would
just
add
the
operator
bundle.
So
once
you
have
it
in
a
container
registry,
an
image
reference,
you
would
pass
that
on
to
opm
register
your
opium
index
ad,
and
that
would
extract
the
information
from
the
from
the
bundle,
image
and
store
that
in
the
database
the
rm
would
remove
an
entire
operator.
A
A
A
So,
with
the
introduction
of
the
new
command,
we
had
to
build
out
processes
for
building
these
catalogs.
So
imagine
you
have
a
catalog
called
the
cool
catalog.
Obviously
you
have
a
team
responsible
for
building
this
catalog
and
maintaining
this
catalog,
and
then
you
have
the
xcd
operator
team,
prometheus
operator
team
and
a
lot
of
other
operator
teams
who
wants
to
include
their
operator,
who
or
who
wants
to
make
their
operator
available
on
cluster
for
users
through
your
catalog.
A
So
they
would
submit.
They
would
first
build
the
bundle,
each
bundle
that
they
have
in
their
operator
upgrade
graph
and
they
would
submit
that
to
the
team
responsible
for
creating
the
cool
catalog
and
the
team.
The
cool
catalog
team
would
then
use
opm
index
ad
over
the
bundle
images
that
were
submitted
to
them
to
create
the
cool
catalog.
A
So
so
suppose
I
make
a
mistake,
etc.
Team
makes
a
mistake
on
v.0.6.1
and
wants
to
replace
that
the
pipeline
team
then
had
to
opm
index
rm
the
lcd
operator
entirely
from
the
catalog
and
then
rebuild
the
bundle,
and
the
reason
that
we
have
to
rebuild
all
the
bundles
is
because
v.6.2
would
have
a
reference
to
v.6.1
of
that
city
operator.
Think
about
it
as
a
as
a
linked
list
and
v
dot
series
dots
two
is
where
it
would
say.
A
A
That
was
still
fine,
but
now
imagine
a
scenario
where
an
operator
team
wants
to
include
a
cv
for
an
existing
bundle
or
wants
to
add
yeah.
I
want
to
replace
the
bundle
in
the
catalog,
so
the
like,
I
mentioned
the
pipeline
team,
had
to
rebuild
all
of
the
bundle
images
remove
the
operator
from
the
catalog
and
then
re-add
all
the
bundles.
A
So
this
is
when
a
request
came
in
for
opm
for
adding
a
new
sub
command
called
prune
that
would
prune
all
of
the
all
of
the
that
would
get
rid
of
all
of
the
packages
or
all
of
the
operators
in
the
catalog,
except
for
the
ones
specified
that
only
helped
a
little
bit.
It
didn't
help
all
of
the
pain
points
that
I
described
earlier,
though,
now
imagine
another
scenario
where
operating
wants
to
add
a
bundle
in
the
middle
of
the
upgrade
graph.
A
So
this
is
when
another
request
came
in
to
add
a
substitute
for
field
in
the
cluster
service
version
api
that
I
mentioned
earlier.
Essentially,
this
would
add
a
new
field
in
your
database,
and
you
could
mention,
through
this
cluster
service
version,
the
new
bundle
that
would
be
a
substitute
for
the
old
bundle.
A
Now
imagine
another
requirement
coming
in
from
another
team
where
they
said
that
okay,
we
released
a
bundle
earlier,
but
now
we
want
to
deprecate
that
bundle
and
therefore
all
of
the
bundles
that
came
in
later
in
the
upgrade
graph
after
that
bundle
would
have
to
be
removed
so
that
users
cannot
ins
upgrade
to
that
bundle.
In
the
current
scenario
that
I
mentioned
opera,
the
cool
catalog
team
had
no
way
of
doing
that.
A
A
So,
as
you
can
see,
imagine
imagine
I
was
just
talking
about
cool
catalog,
but
imagine
three
four
catalogs
being
built
and
then
all
each
catalog
being
the
responsibility
of
each
team
of
an
individual
team.
Each
team
would
then
have
come
up
with
their
unique
way
of
building
and
maintaining
these
catalogs
and
therefore
they
would
have
their
unique
requirements
for
how
opm
should
behave
or
for
how
that
tool
should
let
them
handle
the
or
maintain
the
catalog.
A
B
That
you
know
is
difficult
to
maintain
and
kind
of
keep
everything
on
track.
So
what
we
came
up
with
is
a
brand
new
format
for
the
underlying
index,
which
is
just
declarative,
plain
text
files
stored
in
a
directory
structure.
So
now
every
every
package,
so
every
operator
team,
so
you
might
have
an
scd
operator.
You
might
have
prometheus
operator.
B
All
of
those
operators
can
store
their
own
package
level
metadata
for
their
index
now,
just
in
a
plain
text,
yaml
or
json
file-
and
this
is
so-
the
opm
tool
has
now
been
updated
to
to
support
this
new
plain
text
format.
The
sqlite
database
format
has
been
deprecated
and
we're
going
to
try
to
migrate
a
lot
of
the
catalog
usage
over
to
this
new
declarative
format.
B
So
so
this
is
just
a
short
example
of
what
something
looks
like
you
have
a
couple
of
like
root
level
schemas.
Here
we
see
like
an
olm
dot
package
which
has
package
level
metadata,
the
name
of
the
package,
the
default
channel
for
the
package,
a
description
for
it,
and
then
you
can
start
listing
other
things
like
an
olm.bundle
which
is
just
an
index
representation
of
that
bundle
image.
So
we
have
the
the
name
of
the
csv.
That
was
in
the
bundle.
B
A
B
That
kind
of
thing,
so
now
all
of
the
stuff
is
stored
in
plain
text
and
operator.
Authors
can
now,
you
know,
have
the
control
to
to
make
changes
declaratively
based
on
what
they
need
their
index
to
look
like,
so
the
issues
that
onik
was
explaining
about
you
know
inserting
into
a
linked
list
of
an
upgrade
graph.
Now
is
just
cracking
open
this
index
file
and
inserting
the
changes
in
the
right
way,
rather
than
having
to
do
this
imperative,
workflow
of
removing
the
whole
thing
and
then
re-adding
everything
in
order.
B
B
B
B
So
now
what
we
can
do,
because
this
is
organized
as
like,
a
an
arbitrary
directory
structure,
there's
not
a
single
source
of
truth
for,
like
the
entire
contents
of
the
catalog.
Now
what
happens
is
operator?
Authors
can
build
just
just
an
index
that
contains
just
their
package
and
then
there's
kind
of
two
ways.
You
could
do
this.
You
could
have
a
catalog
that
says
like
oh
well.
I
want
to
maintain
a
list
of
references
to
other
indexes
that
already
exist
and
I'll
just
copy
those
into
my
larger
catalog.
B
So
now,
basically,
we
can
have
this
composability
feature
where
you
can
build
indexes
of
indexes
of
indexes
just
by
copying
more
sub-indexes
into
your
index
and
then,
lastly,
with
extensibility,
because
this
is
a
plain
text:
format,
it's
json
or
yaml.
There's
lots
of
programming
languages
out
there,
there's
lots
of
tools
out
there
that
understand
json
and
yaml.
B
So
it's
it's
really
easy
for
both
operator
authors
and
catalog
maintainers
to
build
their
own
tooling
on
top
of
the
schema
and
format
and
come
up
with
things
that
the
olm
team
either
hasn't
come
up
with
or
has
decided,
is
too
niche
of
a
concern
for
the
larger
opm
tooling
to
to
maintain
and
use.
But
it
gives
all
of
these
other
users
an
escape
hatch
to
build
their
own
kind
of
workflows
around
this
low-lying
schema.
B
So
so
we've
got
looks
like
maybe
five
or
six
commands
new
for
opm
that
are
supporting
this
new
file
based
catalog
format.
So
a
couple
are
just
kind
of
really
straightforward,
so
like
opm
init,
all
it
does.
Is
it
just
generates
that
olim.packageblob
for
package
you
give
it
a
package
name,
and
you
can
specify
some
flags
for
specifying
the
default
channel,
an
icon
or
a
description
and
it'll
just
generate
an
output
on
standard
out
just
the
olin.package
blob,
and
you
can.
Then
you
know
pipe
that
into
a
yama
file.
B
You
can
pipe
that
through
yq
or
jq,
to
make
some
edits.
It's
kind
of
this
unix
way
of
doing
things
where
you
know
it'll
print
it
to
stand
it
out,
and
then
it's
up
to
the
user
to
decide
how
to
use
that
render
is
the
same
sort
of
thing,
except
that
it
takes
an
existing
index
image
or
a
bundle,
image
or
a
an
old
sqlite
file
and
produces
the
equivalent
file
based
catalog
for
that
content.
B
So
this
is
nice
because
all
of
that
existing
content
out
there,
that's
in
an
sqlite
based
index
image
or
an
sqlite
file,
can
just
be
readily
converted
over
to
this
new
filebase
catalog
format.
Just
by
running
opm
render
the
index
image
that
you
want
to
convert
and
then
again
this
will
just
spit
that
entire
thing
out
on
standard
out
and
then
and
then
you
can
write.
B
B
Opm
could
guarantee
that
any
changes
made
to
the
database
were
valid
based
on
opm's
code
or
based
on
the
database
schemas
that
were
underlying
those
commands.
So
with
file
based
catalogs,
we
needed
a
validate
command
that
that
users
can
use
such
that
when
they
make
changes
to
their
file
base
config,
they
can
run
this
opium
validate
command.
That
will
tell
them
if
what
they
have
built
is
actually
a
valid
catalog
opm
server.
B
This
is
basically
the
equivalent
of
the
opium
registry,
serve
command
for
sqlite
databases,
except
this
knows
how
to
read
a
source
directory
that
contains
file
based
catalogs.
It
serves
the
exact
same
grpc
api.
It's
fully
backward
compatible,
so
you
don't
have
to
use
a
new
version
of
olm
to
use
this
any
version
of
olm
that
works
with
the
grpc
api
will
work
with
file
based
catalogs
as
well
via
this
opm
surf
command.
Then
we
have
a
couple
of
alpha
commands.
These
are
still
subject
to
change.
B
This
kind
of
takes
the
place
a
little
bit
of
opm
index
ad,
which,
in
the
past
opm,
would
try
to
manage
your
images
for
you
and
it
had
underlying
tooling
and
configuration
to
let
you
decide
to
push
and
pull
images
with
certain
container
tools.
So,
for
instance,
opm
index
ad
has
a
flag
that
says:
push
like
pull
tool,
equals
docker
and
push
tool,
equals
podman,
and
then
opm
would
actually
invoke
docker
and
podman
for
those
pull
and
push
actions.
B
We've
decided
based
on
lots
of
feedback
and
lots
of
real
world
usage,
that
trying
to
maintain
these.
These
image
push
and
pull
kind
of
workflows
directly
in
opm
is
not
very
maintainable,
so
the
alternative
that
we've
come
up
with
is
basically
like.
Let's
just
have
a
command
that
generates
a
docker
file.
That
knows
how
to
publish
your
catalog
for
you
and
then
it's
up
to
you
then,
to
to
go
to
the
next
step,
basically
and
invoke
whatever
your
tool
of
choice
is
rather
whether
it's
docker
build
or
podman.
B
So,
let's
look
at
kind
of
what
this
new
catalog
building
workflow
is
so
so
right
now
we're
seeing
kind
of
how
this
used
to
work
right.
This
is
what
onik
was
explaining
earlier
operator.
Teams
would
submit
bundle
references
and
they
would
only
submit
button
references
so
so
a
bundle
would
basically
have
to
fully
explain
every
change
that
that
would
possibly
want
to
be
made
to
a
catalog,
and
then
the
cool
catalog
maintainers
would
run
opm
index
ad
and
opm
index
ad
would
have
to
know
how
to
interpret
that
bundle
and
update
the
catalog
appropriately.
B
B
They
can
well,
they
need
to
push
their
bundle,
images
like
they
always
have
to
a
container
registry
and
then,
when
they
go
and
talk
to
the
the
catalog,
they
can
just
basically
make
declarative
changes
to
their
operators
in
their
package
based
index
and
submit
pull
requests
potentially
to
the
catalog.
That
says,
you
know
here's
the
changes
I
want
to
make
to
my
index
there's
another
option
where
an
operator
team
could
say
we
want
to
maintain
our
own
index
repository,
and
maybe
the
catalog
says.
B
B
A
B
Okay,
yeah
yeah
I'll
get
to
the
next
slide.
I
just
want
to
finish
my
thought
here.
So
the
last
thing
is
that
once
the
entire
ci
has
passed
and
we
merged
that
pr
into
the
main
branch,
we
can
actually
build
a
new
image
for
that.
So
we
have
some
examples
of
what
this
looks
like
right.
So
we
have
this
cool,
catalog
repo,
it's
just
an
example,
but
it
kind
of
demonstrates
how
all
of
this
could
work.
B
So
in
this
case
we've
got
just
a
single
operator:
we've
got
this
etc
folder
we've
got
a
docker
file
and
we've
got
some
github
workflows
right.
So
the
the
current
scd
index
has
just
got
a
single
version
in
it
it
it.
We
declare
our
olin.package
name
it
at
cd,
we've
got
0.6.1,
so
now
we
decide.
Okay,
we've
got
an
scd09.4
bundle
that
we
want
to
add
to
our
index.
So
we
render
that
we
can
see
what
it
looks
like
on
our
output.
B
We
call
render
again
we
redirect
it
into.
You
know
just
cut
it
to
the
end
of
our
index
file,
and
then
we
submit
a
pull
request
right.
So
we've
just
added
some
lines
to
our
index
file.
This
is
what
the
rendered
output
looks
like
there.
We've
submitted
this
pull
request
to
the
cool
catalog
and
we
say
like
hey.
We
want
the
index
to
be
updated,
such
that
this
new
0.9.4
version
is
available
to
users
who
are
using
this
catalog
in
the
cluster.
B
So
at
that
point,
then
users
of
of
this
catalog
will
potentially
get
an
automatic
update.
If
they've
been
to
a
tag
that
we're
updating
or
they
can
go
and
add
it,
you
know
if
they
don't
have
this
catalog
yet
in
the
cluster
they
can
add
a
catalog
via
catalog
source
and
and
now
they'll
be
able
to
install
the
sd
0.9.4
version.
B
So
there's
lots
of
resources
here,
we're
constantly
updating
our
documentation.
If
you
go
to
the
olim
dot
operator,
framework,
dot,
io
website,
we've
got
all
the
docs
and
and
there's
lots
of
docs
about
this
new
file
base.
Catalog.
If
you
want
to
check
out
the
this
example:
cool,
catalog
workflow,
here's
the
link
to
that.
It's
just
operated
framework,
slash
cool
catalog
in
github
and
then,
if
you
have,
if
you're
interested
in
olm
and
you
want
to
join
the
olm
or
the
operator
framework
community
we'd,
be
super
interested,
we're
a
cncf
incubation
project.
B
We
have
weekly
meetings
between
just
kind
of
open
public
working
group
meetings
and
then
issue
triage
meetings.
If
you're
interested
in
discussing
anything
with
us
at
those
we'd
be
happy
to
have
you
and
then
we
also
have
a
a
mailing
list,
so
we're
interested
in
you
know
hearing
what
you
think
about
the
project.
If
you
have
questions
or
comments
and
want
to
talk
to
us
there
awesome.
B
If
you
want
more
live
chat,
discussion,
we've
got
an
olm
dev
channel
in
the
kubernetes
slack
and
then
there's
always
you
know
the
full
listing
of
ways
to
get
in
touch
with
us
in
the
olin
community
page.
So
we're
we're
super
interested
in
getting
more
eyes
on
this
project
and
more
contributors
to
the
project.