►
From YouTube: Harbor- Enterprise Cloud Native Artifact Registry - Steven Zou, Daniel Jiang, Alex Xu, & Steven Ren
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Harbor- Enterprise Cloud Native Artifact Registry - Steven Zou, Daniel Jiang, Alex Xu, & Steven Ren, VMware
Project Harbor is an open-source trusted cloud native registry project that stores, manages, signs, and scans content, thus resolving the management and distribution challenges of container image, Helm Chart, CNAB or other OCI compatible artifacts. In this presentation, the speakers will review and provide solutions to the challenges faced by organizations, including RBAC, content replication, vulnerability scanning, large scale content distribution, content trust (notary), webhook, tag retention, non-blocking GC, proxy cache and user defined artifact support etc.. Furthermore, the speakers are also going to share the future roadmap and challenges of Harbor including but not limited to K8s operator for Harbor, improvements of windows containers support, observability, networking(IPV6), backup & restore, and IAM&RBAC enhancements etc.. More importantly, the speakers would love to get feedback from Harbor users, contributors and partners to the current features and future roadmap.
https://sched.co/ekI5
A
Hi
everyone-
I'm
alex,
I'm
very
happy
to
be
here
at
kubicon,
north
america,
to
talk
to
everyone
about
the
harper
project.
It's
a
shame
that
this
is
still
a
virtual
conference
and
we
couldn't
come
in
and
share
this
in
person,
but
I
hope
everyone
is
doing
really
well
and
thank
you
for
tuning
in
so
today
my
co-presenter
and
I
will
talk
a
little
bit
about
the
harvard
project.
A
A
A
quick
introduction,
I'm
alex
jr,
I'm
a
product
manager
in
the
cloud
native
team
at
vmware,
leading
the
harbor
effort
so
responsible
for
trying
to
understand
the
requirements
around
the
container
registry
for
responsible
driving
the
road
map
collecting
feedback
from
from
you
guys
and
making
sure
that
we're
building
something
that
people
actually
use
and
will
actually
want
to
recommend
to
to
their
friends.
So-
and
today,
with
me,
is
my
co-presenter
daniel.
A
B
C
A
So
we,
you
know,
we
start
with
the
intro
slide.
What
is
harper
well
harvard
is
a
trusted
cloud
native
registry
that
can
store,
sign
and
scan
content.
So
it's
basically
a
place
to
host
and
manage
your
artifacts,
and
you
know
when
we
started
this
project
back
in
2014
building
an
on-prem
registry
leveraging
talk
distribution.
A
We
came
across
some
issues
right
when
we
were
using
docker
hub
and
other
public
registries,
so
we're
over
time.
We've
you
know
addressed
those
issues
and
also
added
more
services
and
features
related
to
life
cycle
management,
security
features
like
scanning
and
signing
images,
and
recently
we've
been
focusing
on
image
distribution
through
p2p
and
proxy
cache,
and
now
we're
focusing
on
high
availability
delivering
a
high
highly
available
hardware
cluster
through
our
operator.
A
A
So,
let's
start
by
looking
at
the
community,
which
is
thriving
doing
really
well,
so
we
have
more
than
13k
github
stars.
We
have
200,
plus
committers
50,
plus
contributing
companies
more
than
4
000
forks.
We
have
14
maintainers
across
five
different
companies
across
three
different
continents
in
north
america
and
europe
in
asia.
A
Tuning
and
so
I
want
to
just
quickly
jump
into
this
and
start
with
some
of
the
things
that
we've
been
working
on
for
the
2.1
release.
The
first
one
I
want
to
talk
about
is
called
proxy
cache,
which
is
the
ability
for
harvard
to
act
as
a
pull
through
cache
for
another
remote
registry,
and
you
know
that's.
A
They're
trying
to
hit
right
docker
hub
is
a
registry
where
docker
clients
from
all
over
the
world
are
trying
to
pull
images
from,
but
if
you're
pulling
too
fast
or
you're
pulling
too
frequently,
then
you
can
trigger
the
dot
the
docker
hub
rate
limiter,
which
can
throttle
your
connection
or
even
get
uip
band.
So
hardware,
as
a
proxy
cache,
is
meant
to
address
this
very
problem.
A
A
A
So
the
second
project-
or
the
second
feature
I
want
to
highlight-
is
called
non-blocking
gc.
It's
called
non-blocking
because
prior
to
2.1
collection
will
put
the
registry
in
read-only
mode,
so
you
can
still
interact
with
your
existing
artifacts
in
harbor.
You
can
still,
you
know,
use
the
apis.
You
can
still
use
the
web
console
run.
You
know
certain
policies,
you
can
still
pull
the
images,
but
you
can't
push
images
and
you
can't
delete
them
essentially
for
the
entire
duration
of
that
gc
execution,
and
so
with
the
2.1
non-blocking.
A
C
A
Know,
data
corruption,
anything
like
that
data
corruption
is,
you
know
it
was
a
potential
concern
for
us
because
of
the
the
way
the
images
are
laid
out
and
stored
in
distribution
they're.
Essentially,
you
know
this
is
owing
to
the
the
cross-referenceable
nature
of
stock
distribution,
where
the
image
layers
are
stored
in
the
most
efficient
way
possible,
with
no
redundancy
they're,
essentially
deduped
across
all
the
projects
in
a
single
harbor
instance.
A
To
we've
seen
tens
of
terabytes
of
images
stored
on
each
harbor
instance
and
and
because
you
know
gc
can
be
quite
time
consuming
depending
on
how
often
you
run
it
if
it's
placed
in
read-only
mode
for
an
extended
amount
of
time,
and
on
top
of
that,
there's
no
way
to
tell
how
long
that
job
will
run
for
or
how
much
disk
space
your
you
will.
It
will
be
deallocated
from
each
execution.
You
know
that
can
be
problematic
and
it
can
be
quite
stressful,
all
right.
A
So
imagine
if
you're
in
the
software
delivery
business
sort
of
software
supply
chain
business,
where
all
the
images
are
landing
in
your
harper
instance
and
you're
the
one
responsible
for
distributing
these
to
your
customers,
and
so
that's
why
we've
added
the
ability
to
perform
a
dry
run,
which
will
give
you
an
estimation
of
the
amount
of
disk
space
that
will
be
freed
up
and
so
by
making
it
non-blocking.
Not
only
is
it
you
know
not
incurring
a
penalty
on
your
user
operations.
A
Not
only
is
it
running
in
a
faster
and
more
efficient
manner,
but
you
know
the
time
it
takes
to
run.
Gc
is
no
longer
as
much
of
a
factor
it's
no
longer
as
debilitating
to
your
business.
If
it
goes
over,
you
know
if
it
goes
runs
for
for
quite
some
time
right,
you're
you're,
taking
the
pressure
off
the
garbage
collection
aspect
of
maintaining
a
registry.
C
A
C
Oh
okay,
yeah,
it
is
still
recording
yeah.
So
let
me
get
started,
hey
guys,
so
the
first
demo
I'm
gonna,
show
you
is
proxy
cache.
I
need
to
use
a
recorded
demo
due
to
the
instability
of
my
network.
So,
let's
move
on
so
in
the
proxy
cache
to
implement
it.
We
reuse
the
code
of
replication,
so
the
user
experience
also
has
some
overlap
for
you
to
create
a
proxycad
project.
First,
you
need
to
create
a
registry
endpoint.
C
C
C
C
So
it
has
all
the
information
and
you
can
do
all
the
stuff
to
this
image,
such
as
scanning
or
adding
tabs
removing
text.
Here.
I
simulate
a
outage
of
docker
hub
by
messing
up
my
dns,
so
that
my
hardware
instance
cannot
access
this
docker
hub
and
you
can
see
the
test
connection.
The
ping
doesn't
work
now,
but
if
I
remove
this
image
locally
and
pull
it
again
from
my
docker
client.
C
C
C
Is
the
non-blocking
gc
non-blocking
garbage
collection
demo,
also,
as
alex
mentioned
in
when
we
implement
this
feature
instead
of
relying
on
the
docker
distributions
code,
we
use
this
data
in
hubble's
database
as
a
single
source
of
truth
to
help
us
decide
what
layers
to
be
removed
and
what
you
know.
How
much
space
will
be
freed
is
also
available.
C
So
when
you
hit
the
dry
run,
we
we
just
mark
the
layers
that
are
candidates
to
be
removed
and
calculate
the
size,
and
we
can
provide
a
hint
for
the
admin
to
you
know.
Let
them
know
how
much
space
will
be
freed
if
we
run
the
dc
right
now.
No,
this
is
a
very
rough
estimation,
because
this
is
a
non-blocking
dc,
so
images
may
be
pushed
when
the
gc
is
running.
You're
gonna
see
now.
C
B
C
C
C
A
A
A
The
first
one
is
metrics
and
observability,
and
this
is
really
just
a
fancy
way
of
saying
we're
going
to
be
supporting
prometheus.
So
the
goal
is
to
monitor
services
deployed
actions
performed
as
part
of
your
harvard
deployment
and
expose
those
metrics
through
an
http
endpoint
for
prometheus,
and
so,
if
you
have
prometheus
already
set
up
in
your
environment,
if
it's
already
part
of
your
stack,
you
can
then
scrape
whatever
data
you
want
off
that
endpoint,
I'm
going
to
make
it.
A
You
know
highly
customizable
in
terms
of
the
kind
of
data
you
can
retrieve,
and
you
know
when
you
think
about
key
telemetry
telemetry
can
be
divided
into
several
layers
right
at
the
very
bottom.
You
have
the
platform,
which
is
your
node
level
information,
and
you
can
monitor
your
resource
consumption,
so
cpu
ram,
disk
basic
event,
logging
health
status
of
the
of
the
the
deployment
as
a
whole,
and
then
you
have
the
container
orchestration
layer
which,
in
the
case
of
kubernetes,
will
provide
monitoring
for
all
the
pots
deployed
in
your
cluster.
A
A
So
the
web
portal
has
its
own
nginx,
has
its
own
chart
museum
if
you're
using
trivia
or
clear
image
scanners
they
have
their
own
and
then
finally,
we
have
telemetry
at
the
application
layer,
which
is
you
know,
basically
the
relevant
registry
operations.
Anything
that
the
hardware
api
provides
is
a
potential
candidate
here
and
we
can
potentially
expose
aggregated
statistics
over
some
period.
A
So,
looking
at
the
number
of
image,
poles
or
number
of
images
pushes
image
deletions,
also
looking
at
top
images
repositories,
projects
being
requested,
possibly
looking
at
you
know,
number
of
push
errors
or
delete
errors
or
authentication
errors,
and
you
know
any
other
statistics
that
could
be
useful
for
detecting
abnormalities.
Right,
like
users
number
of
users
logged
into
a
particular
instance,
number
of
failed
attempts,
unique
ips
number
of
image
requests
served
in
some
period
requests
in
flight
number
of
replications,
so
this
is
still
being
actively
worked
on.
A
A
A
So
we've
also
got
a
lot
of
requests
to
deliver
system
level.
Robot
accounts,
which
is
a
robot
account
scoped
to
the
harper
instance
level.
So
it
has
access
to
multiple
projects,
as
opposed
to
the
current
robot
implementation,
which
is
scoped
to
a
particular
project
right
if
you're
familiar
with
that,
how
that
works,
you're
creating
the
robot
account
from
within
the
project
configuration
page
right
now-
and
you
know
that's-
that's
also
understandable,
project
skill
permissions.
A
A
A
So
if
that
token
is
hard-coded
somewhere
into
your
ci
or
it's
being
used
to
construct
a
pull
secret,
it
needs
to
be
manually
recycled
right
now.
So
we're
trying
to
kind
of
we're
trying
to
remove
this
kind
of
friction
so
that
changing
the
project,
scope
or
changing
the
permissions
of
that
robot
will
not
impact
your
ci
in
any
way.
A
So
again,
this
is
being
actively
discussed
and
being
worked
on.
I've
linked
the
pr
as
well
to
this
proposal
in
our
community
repo.
If
you
have
any
questions
comments,
please,
you
know
share
your
feelings
on
the
pr.
If
you
have
already
implemented
your
own
solution,
we'd
be
really
interested
in
hearing
that
as
well.
A
Ovh
cloud
team
built
the
first
version
of
the
harper
operator,
and
you
can
find
this
as
the
core
operator
rebuild
under
go
hardware
project
on
github,
but
it
doesn't
include
certain
dependent
services,
like
databases,
redis,
cache
or
storage,
because
you
know
some
vendors
already
have
those
services
running
as
managed
services
and
they're,
perhaps
already
deployed
in
a
much
more
highly
distributed
or
highly
available
manner,
and
that
makes
sense
right.
That's
that's.
A
Definitely
one
use
case
that
we
have
to
we
have
to
cover,
but
you
know
for
the
other
users
that
are
perhaps
not
as
advanced
that's
looking
for
an
all-in-one
deployment
that
comes
with
all
the
components
that
harvard
needs
to
run.
We
have
built,
what's
called
the
hardware
cluster
operator
and
that
will
basically
deploy
harper
with
all
the
dependent
services
like
database,
redis,
cache
storage
and
deploy
these
in
aja
mode.
A
So
we
don't
have
to
deep
dive
every
term.
Every
aspect
of
this
diagram,
but
the
key
takeaway
here
is
that
you
can
see
that
in
this
diagram,
there's
a
harbor
hardware,
cluster
cr
at
the
very
top
right.
So
there's
a
there's,
a
hardware
cluster
controller,
that's
essentially
managing
all
the
services
needed
for
the
registry
and
the
cluster
operator.
A
Customer
resource
encapsulates
the
core
operator
customer
resource
that
I
just
talked
about,
as
well
as
the
customer
resources
for
all
the
other
individual
components,
and
so
the
harvard
cluster
is
basically
watching
over
everything
and
continuously
reconciling
to
make
sure
that
what
is
running
is
is
what
is
actually
specified
in
the
cr
spec.
And
so,
if
there
any
failures
or
hiccups,
then
it
can
automatically
redeploy
those
services
so
now
for
anyone
who's
wondering
about
high
availability
in
the
sense
of
multiple
harbors
harbor
clusters
across
different
data
centers,
perhaps
in
different
availability
zones.
A
Even
we
we
do
have
a
story
in
the
works
to
address
that
as
well,
but
that's
sort
of
outside
the
scope
of
the
harvard
cluster
operator.
I
know
that
there's
people
have
been
asking
about
it,
so
I
just
want
to
address
it
and
say
you
know.
This
is
something
that
we
are
aware
of,
and
it's
something
that
we're
working
towards
and
so
for
each
of
these
components
you
can
customize
the
spec.
For
that
crd.
A
A
So
we
have
a
really
detailed
proposal,
detailed
meeting
notes
on
the
progress
for
the
cluster
operator
under
the
go
harvard
project,
so
please
feel
free
to
check
that
out
and
share
your
thoughts.
You
know
we're
still
wanting
more
feedback
on
design,
because
we
will
only
have
released
the
very
first
cluster
operator
version
1.0
by
the
time
this
recording
is
shared.
A
Please
follow
our
twitter
if
you
want
to
be
alerted
to
any
of
the
latest
announcements
on
releases,
community
events
collaborations
or
anything,
harbor
related
we're
still
holding
the
the
bi-weekly
community
meetings,
one
for
u.s
time
zone
and
one
for
apec.
This
is
where
you
know
we
have
some
of
the
future
demos
like
the
ones
that
daniel
just
did,
and
we
highlight
some
of
the
latest
developments
in
the
project
and
anyone
can
come
to
resume
meeting
and
ask
questions
or
share
your
experiences
around
harper.
A
C
A
The
detailed
meeting
notes
in
or
the
meeting
details
in
the
readme
for
the
harper
project
on
github
and
finally,
we
do
have
a
demo
deployed
that
anyone
can
just
go
on
and
and
give
it
a
try,
get
familiar
with
the
the
ui
create
a
project.
Add
users
push
and
pull
images
to
and
from
that
project,
so
demo
dot
go
harder,
dot,
io.