►
From YouTube: Am I Using It Right? Checking Best Practices on Live Kuber... Varsha Varadarajan & Adam Wolfe Gordon
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Am I Using It Right? Checking Best Practices on Live Kubernetes Clusters - Varsha Varadarajan & Adam Wolfe Gordon, DigitalOcean
While Kubernetes is stable, best practices for using it are a moving target. Some are generally applicable, others unique to a particular configuration or platform. Following best practices helps ensure workloads stay running as expected through cluster maintenance and upgrades, but checking them can feel like playing whack-a-mole in the dark. This talk introduces a new open source tool, clusterlint, that checks compliance with best practices. Unlike other linters that work on deployment manifests, clusterlint identifies risks and problems in running Kubernetes clusters, making it useful for finding potential problems before performing cluster maintenance. We'll discuss what clusterlint checks, why, how it works, how we use it in DigitalOcean's managed Kubernetes product to warn users of danger, and future plans for the tool.
https://sched.co/UaW2
A
B
A
We're
gonna
talk
a
bit
today
about
how
cluster
Linnet
works,
how
we
built
it
and
how
we've
integrated
it
in
our
product,
but
I
want
to
start
with
why
we
wanted
to
build
this
tool,
and
that
story
starts
with
something
that
a
lot
of
you
probably
know
which
is
like
kubernetes
is
super
flexible,
I,
Brian
kind
of
touched
on
this
in
his
keynote
this
morning,
actually
that
it's
a
very
flexible
tool.
This
is
one
of
the
reasons
why
he
mentioned
kubernetes
complex.
It's
also
one
of
the
reasons
why
kubernetes
has
become
very
popular.
A
It
can
suit
a
lot
of
different
use
cases
and
it's
flexible
in
a
number
of
different
ways.
It's
flexible
in
that
it
can
run
pretty
much
anywhere.
You
can
run
it
on
premise
on
hardware
or
on
VMs.
You
can
run
it
in
the
cloud,
but
run
your
own
cluster,
so
create
a
bunch
of
instances,
turn
them
into
a
cluster.
You
can
also
use
one
of
the
many
managed
kubernetes
offerings
like
our
offering
to
UK
s
or
UK
s
or
GT.
A
There's
a
lot
of
them
out
there
and
you
can
configure
kubernetes
in
a
lot
of
different
ways.
You
can
use
different
Network
overlays,
which
you're
going
to
have
deep
effects
on
how
your
networking
works.
You
can
use
different
storage
drivers
which
are
going
to
affect
how
your
persistent
storage
works.
You
can
use
different
container
runtimes
like
docker,
container
D
or
G
visor
and
beyond
that,
there's
all
kinds
of
different
ways
to
deploy
your
applications
to
Cooper
Nettie's.
A
What
this
all
adds
up
to
is
that
there's
a
lot
of
ways
to
get
things
wrong
in
kubernetes
and
if
you
get
things
wrong,
you're
gonna
have
been
expected
downtime
for
your
application
and
it's
also
hard
to
define
best
practices
and
sort
of
say.
If
you
do,
this,
you'll
be
fine,
because,
depending
on
your
environment,
depending
on
your
configuration
that
might
or
may
not
be
true,
there's
kind
of
different
things.
You
need
to
set
up,
depending
on
your
environment,
to
keep
your
application
safe,
so
just
to
give
a
quick
sort
of
motivating
example.
A
This
is
what
I
do
chaos
cluster
looks
like
in
the
abstract
I.
We
build
our
clusters
out
of
virtual
machines,
which
we
call
droplets
and
you
have
a
control
plane
which
we
run,
and
then
you
have
a
bunch
of
worker
nodes
that
run
your
workload
and
we've
decided
that
nodes
in
do
KS
are
essentially
immutable.
This
kind
of
fits
with
the
kubernetes
philosophy
of
livestock,
not
pets,
so
we
provision
these
nodes
automatically
for
you
and
once
they're
provision
we
don't
really
touch
them
manually.
We're
not
gonna,
go
in
and
modify
them.
A
So
if
something
goes
wrong
with
one
of
your
worker
nodes
and
it
becomes
unhealthy,
I
we're
not
gonna
SSH
in
and
fix
it,
because
we
don't
have
as
a
sage
access
to
it,
we're
not
gonna,
try
and
fix
it.
We're
gonna
recommend
that
you
just
hit
the
replace
button.
It's
gonna
go
away
and
you're
gonna
get
a
brand
new
node
and
we
do
upgrades
in
the
same
way.
A
So
if
you
want
to
upgrade
to
this
new
teal
version
of
kubernetes,
we're
gonna
replace
your
control
plane
with
a
new
teal
control
plane
and
then
we're
going
to
replace
all
of
your
mouths
all
of
your
worker
nodes,
one
by
one,
with
a
new
teal
worker
node.
Now
what
this
means
is
that,
at
the
end
of
an
upgrade,
you
have
an
entirely
new
set
of
nodes
that
are
unrelated
to
the
nodes
that
came
before
them
other
than
that
they're
running
the
same
workloads,
they're
part
of
the
same
cluster.
A
So
obviously
this
has
like
a
pretty
deep
impact
on
your
workloads.
So
when
customers
started
using
do
KS
and
started
doing
upgrades
on
TAS,
they
noticed
some
of
these
implications.
One
of
the
things
they
noticed
is
that
the
node
names
aren't
stable
because
we
actually
replace
the
nodes.
The
know
the
new
node
is
a
new
kubernetes
object.
It
has
a
different
name
than
the
old
one.
A
So
if
you
were
using
a
node
name,
specifier
to
schedule
your
workloads
onto
a
specific
node
that
node
that
workload
is
not
going
to
be
schedulable
after
the
upgrade
same
with
any
custom
labels
that
you
said,
we
don't
persist
those.
So
if
you're
using
labels
to
schedule
workloads
onto
a
specific
set
of
nodes,
you're
gonna
have
problems.
A
After
an
upgrade,
we
do
provide
a
node
pool
concept
with
the
label,
so
you
can
schedule,
though,
a
schedule
we're
close
to
a
specific
set
of
nodes
or
a
specific
node,
but
you
just
can't
use
your
own
custom
labels
for
that.
At
this
point,
likewise,
IP
address
has
changed,
because
these
are
new
nodes,
again
I.
So
if
you're
pointing
something
it
directly
at
a
at
a
node,
IP
I
you're
gonna
have
a
problem.
We
recommend
people
use
our
matters,
build
balancers
for
that
and
your
file
systems
aren't
persistent
I.
A
Anything
that's
on
the
local
disk
of
the
node
is
gonna
be
after
every,
so
we
recommend
that
people
use
our
block
storage
volumes
if
they
want
persistent
storage.
We
documented
all
of
these
things,
but
not
everyone's
gonna
read
the
docs
all
the
time
when
they're
deploying
things
and
in
particular,
people
are
going
to
deploy
things.
The
way
they've
deployed
them
before
in
different
environments
on
different
providers.
They're
gonna
use
tools
like
helm
that
are
generic
and
don't
know
about
the
sort
of
subtleties
of
the
environment
that
you're
running
in.
A
So
what
we
wanted
was
to
automatically
warn
our
customers
if
they
had
workloads
that
were
configured
in
a
way
that
we
thought
was
going
to
cause
problems,
because
we
didn't
want
someone
to
do
an
upgrade
and
then
discover
that
all
of
their
workloads
were
down
and
have
to
file
a
support
ticket
and
wait
for
us
to
get
to
it.
We
wanted
to
proactively
warn
them,
make
sure
that
they
understood
what
was
gonna
happen,
given
the
opportunity
to
fix
things
before
men
replaced
all
their
notes.
A
B
The
best
practices
that
we
check
for
can
be
specific
to
the
platform
the
cluster
is
hosted
on
are
generally
applicable,
irrespective
of
where
your
cluster
is
posted.
It's
a
non-invasive
tool,
which
means
that
the
class,
the
checks
that
we
run
our
run
externally
by
accessing
the
objects
via
the
kubernetes
api,
based
on
our
experience
with
do
KS.
We
had
three
specific
goals
in
mind
before
we
started
building
it,
which
we
didn't
find
was
satisfied
in
any
of
the
existing
tools.
First,
we
wanted
to
run
the
checks
against
a
live
cluster
and
I.
B
Don't
object
manifests
because
actual
workloads
can
sometimes
differ
from
your
object
manifests.
Hopefully
you
have
a
CI
pipeline
and
place
to
deploy.
The
object
manifests
to
your
cluster,
so
that
that's
not
the
case,
but
we
wanted
the
cluster
operators
to
identify
issues
in
one
place
and
then
go
fix
their
configurations
later
on
and
because
we
wanted
to
run
this
on
Bo
KS
we
cluster
and
not
the
object,
manifests.
B
We
also
wanted
to
run
some
platform
specific
checks.
As
Adam
mentioned,
there
are
some
implications
for
workloads
in
the
cluster,
which
means
that
there
will
be
issues
occurring
in
your
cluster,
based
on
which
platform
you
use.
We
wanted
to
run
platform,
specific
checks
for
those
issues
and
because
we
wanted
to
run
this
tool
on
behalf
of
customers
on
do
Kias.
We
wanted
a
static
library
of
checks
and
an
API
to
interact
with,
and
not
just
the
command
line.
B
So
with
these
goals
in
mind,
we
started
building
cluster
lint.
How
bustling
operates.
It
is
quite
simple.
First,
we
fetch
all
the
relevant
kubernetes
api
objects
from
the
cluster
and
store
them
in
memory.
We
didn't
want
the
checks
themselves
to
worry
about
which
objects
to
fetch
fetch,
but
rather
focus
on
the
operations
themselves.
We
have
several
part
related
checks
in
flatulent
and
we
didn't
want
every
part
related
check
to
query
for
pods
from
the
cluster.
So
once
all
these
objects
are
in
memory,
classify
runs
the
applicable
checks
in
parallel
and
each
check
reports.
B
The
objects
that
have
owners
the
owning
object,
information
is
also
provided
so
that
cluster
operators
can
identify
the
manifests
they
can
fix.
Their
configurations
for,
for
example,
pots,
can
have
a
deployment
or
a
job
as
owner
and
provides
that
information
as
well
the
CLI
reports.
These
results
in
a
pretty
text
or
JSON
format,
but
you
can
also
use
the
API
to
get
a
nice
possible
structure
when
we
developed
class
land.
It
was
intended
for
all
clusters
that
aspect
of
the
cluster
is
hosted,
but
there
are
some
issues
that
will
occur
in
your
cluster.
B
Based
on
the
platform
you
use,
workloads
that
are
fine
on
a
self-hosted
cluster
or
bare
metal
can
be
dangerous
if
you're,
using
a
managed
service
like
kubernetes,
based
on
how
the
service
is
managed
for
you,
we
came
up
with
check
groups
to
be
able
to
run
only
the
applicable
checks
in
such
cases
to
identify
platform-specific
issues.
For
example,
on
do
case
we
have
the
no-name
mode
selector
check
bot
selectors
can
be
used
to
schedule
pods
on
specific
nodes.
While
this
is
generally
harmless.
B
Because
of
how
do
KS
is
design,
you
can
have
a
breaking
change
or
introduced
when
you
upgrade
or
recycle
the
node,
which
is
why
we
wanted
to
identify
and
want
customers
of
this
issue
before
they
upgraded.
Currently,
we
have
three
groups:
the
BOK
s
group,
which
consists
of
all
the
checks
applicable
to
do
KS
clusters.
We
run
them
by
default
before
performing
an
upgrade
on
do
KS.
This
consists
of
the
new
names
pot,
selector
check,
node
labels
check
web
hooks.
B
That's
the
security
related
group,
which
consists
of
checks
like
identifying
privileged
spots
in
your
cluster
containers,
running
as
root
user,
and
then
there's
the
basic
group
which
contains
checks
that
are
applicable
to
your
cluster,
irrespective
of
where
they
are
like.
If
you
have
resources
in
the
default
namespace
or
you
have
spot
volumes
for
your
pots,
a
checking
belong
to
any
number
of
groups.
This
provides
the
cluster
operators
with
the
flexibility
of
choosing
what
checks
are
applicable
for
their
cluster.
B
B
So
we
wanted
to
warn
users
of
that
before
the
OKs
customers
of
creator,
which
made
sense
to
have
the
bear
pots
check
in
the
do
case
groups,
but
it
is
a
fairly
dangerous
thing
to
have
in
any
cluster,
so
it
means
nice
to
have
it
in
base
as
well.
The
CLI
and
the
API
provide
options
to
whitelist
or
blacklist
groups
and
checks
within
those
groups
so
that
you
can
choose
what
checks
are
applicable
for
your
cluster.
B
Here's
the
check
API
for
those
of
you
looking
to
write
a
check.
We
have
some
metadata
methods,
like
name
and
groups
and
the
run
method
where
the
operation
of
the
check
is
defined.
It
has
access
to
all
the
kubernetes
objects
which
fetched
earlier
and
after
the
check
is
performed
reports.
Diagnostics
forecasts
learn
to
be
aware
of
the
check
and
which
group
it
belongs
to
checks
must
registered
themselves
in
which
a
crash
is
free.
This
is
the
example
who
sparked
volumes
check
that
belongs
to
the
basic
group,
so
this
check
is
applicable
for
clusters
hosted
anywhere.
B
This
just
identifies
pods
that
use
host
pot
volumes.
Most
parts
are
great
when
you
have
when
you're
trying
something
out
in
your
cluster
but
are
fairly
dangerous
when
you're
using
it
in
production.
When
you
know
you
need
to
persist
data,
this
just
identifies
the
pods
and
you
can
go
and
fix
your
configurations
to
use
a
cloud
provider
volume
again
like
des
has
digitalocean
block
storage
than
EPS
for
gas.
B
B
B
A
work
around
here
would
be
to
use
the
do
provided
note
to
label
to
share
your
pods
on
specific
nodes
or
set
of
nodes
so
that
when
your
pods
will
be
scheduled
after
an
upgrade,
there
are
always
some
special
objects
in
your
clusters
that
are
exceptions
to
all
these
best
practices
like.
If
you
have
privileged
spots
in
your
cluster.
You
know
you
want
to
check
for
parts
that
are
progression.
You
possibly
also
know.
There
are
some
parts
that
you
that
need
to
be
privileged.
B
One
way
to
disable
checks
on
such
cases
would
be
to
use
the
command
line
and
white-faced
practice
feature,
but
that
would
disable
the
checks
for
all
objects
cluster.
While
so,
you
can
use
annotations
for
cluster
lint
to
disable
checks
on
the
specific
object
and
trustlink
will
just
ignore
the
object
and
continue
to
perform
the
checks
on
the
clusters.
B
This
is
a
demo
we
have
for
cluster
lint
ion
do
KS.
Let's
start
by
creating
it's
a
recorded
demo.
I
start
by
creating
a
cluster
on
the
yes,
and
once
the
cluster
is
created,
you
will
have
access
to
the
cube.
Config
ya
know
which
you
can
use
with
class
length
like
you
would
with
cube
CDL
or
anything
else.
B
B
B
B
We
can
go
to
the
class
land
repo
to
find
of
the
recommendations
that
we
have
for
such
issues
or
the
checks
that
we
perform
so
far
and
then
see
how
to
fix
the
context.
For
instance,
we've
deployed
a
bit
pod
here,
first,
having
a
deployment
or
a
job
as
the
owner
is
what's
recommended
so
that
the
pods
can
string.
Should
you
laughter
a
disruptive
operation,
so
the
bad
config
is
now
translated
into
something
that
would
work
without
any
issues.
B
A
So
I,
as
we
mentioned
earlier,
one
of
our
goals
of
cluster
lint
was
to
actually
integrated
into
our
product
and
I
use
it
tomorrow.
Our
customers,
if
they're
gonna,
have
problems
before
we
do
something
disruptive
I.
So
we've
done
that.
At
this
point
we
have
an
endpoint
in
our
public
API,
which
lets
you
run
cluster
lint
on
your
do
KS
cluster
and
get
the
results.
It's
also
supported
in
the
do
control
panel.
In
a
limited
way,
we
always
run
cluster
lint
asynchronously
through
our
API
just
in
case
it
takes
a
few
seconds.
A
A
What
this
returns
is
a
run
ID
and
you
can
use
that
run
ID
to
fetch
your
results
later.
We
keep
the
results
around
for
about
a
week.
So
if
you
want
to
run
cluster
lint
at
a
regular,
cadence
and
kind
of
track
your
results
or
something
on
your
cluster,
you
can
do
that
I.
The
second
request
here,
the
one
in
orange,
is
the
request
to
get
the
results
in
this
case.
We're
requesting
the
results
of
just
the
last
run
that
completed.
So
that's
what
we
get.
A
You
can
also
use
the
run
ID
to
request
results
of
a
particular
right.
In
this
case,
we
have
a
badly
configured
admission
control
webhook
in
our
cluster
I.
This
is
one
of
the
number
one
problems
we've
seen.
Customers
have
with
upgrades.
Is
they
have
admission
control
web
hooks
that
prevent
some
of
our
managed
components
from
starting,
so
our
CNI,
for
example,
and
that
makes
an
upgrade
to
go
really
badly
because
you
don't
have
networking
with
you
start
up
here
new
nodes,
so
this
that's
the
diagnostic
to
get
here.
A
We
do
also
run
cluster
lid
before
you
request
an
upgrade
if
you
request
an
upgrade
through
our
control
panel
and
you
can
see
on
the
slide.
This
is
the
dialog
box
that
pops
up
if
it
returned
any
warnings.
So
this
is
the
same
cluster
from
the
previous
slide.
It's
got.
The
Mission
Control
web
foot
problem
and
I
gives
us
a
couple
of
options
for
one.
It
gives
us
a
link
to
the
docs,
so
you
can
actually
see
how
you
might
fix
this
problem.
A
Now
we're
not
just
telling
you
there's
a
problem,
then
leaving
it
up
to
you
to
fix
it.
There's
also
a
rerun
button.
So
if
you
go
and
fix
your
config
really
quickly
come
back,
you
can
hit
the
rerun
button
and
see
the
new
results
and
we
do
provide
an
option
to
just
bypass
the
check
and
say
yeah
I'm,
fine
I.
If
you're
really
confident
that
it's
not
going
to
cause
problems
or
if
you
just
want
to
see
what
happens,
you
can
click
that
box
and
it's
still
proceed
with
your
upgrade.
A
We
are
still
working
on
cluster
lint
and
I.
We
have
a
few
things
in
mind
that
we
would
like
to
do
for
one
we'd
like
to
add
more
checks.
We
definitely
don't
cover
everything,
there's
a
wide
array
of
kubernetes
best
practices
and
we
only
cover
some
of
them.
Another
is
better
integration
into
our
product.
So,
as
I
mentioned
right
now,
we
rented
only
before
manually
triggered
upgrades
automatically
we'd
like
to
run
it
also
before
our
auto
upgrades.
So
if
you
said
a
maintenance
window,
we
will
do
auto
upgrades
on
your
cluster
right
now.
A
We
don't
run
it
in
that
case,
but
we'd
like
to
run
it
and
maybe
send
you
an
email
if
you're
gonna
have
problems
with
the
auto
upgrade
so
that
you
know
ahead
of
time.
We'd
also
like
to
run
it
before
other
destructive
operations
like
replacing
a
node
or
I've,
even
scaling
their
cluster,
or
something
like
that
I.
A
Ideally,
we
would
run
it
periodically
and
just
report
the
results
somewhere.
So
you
have
a
persistent
place.
You
can
go
and
see
maybe
even
track
results
over
time,
but
I've
got.
Finally,
we
thought
a
lot
about
how
to
do
version,
kubernetes
version,
specific
checks
in
cluster
lint,
especially
for
the
upgrade
case
we'd
like
to
do
checks
for
the
next
version.
A
Before
you
do
the
upgrade,
so
you
can
see,
am
I
using
resources
that
are
deprecated
in
the
next
version
am
I
using
fields
that
have
changed
their
meanings
or
their
names
in
the
next
version
that
I'm,
my
own
pirkled,
aren't
gonna
behave
as
I'd
expect.
This
is
a
little
bit
tricky,
because
the
kubernetes
api
and
client
will
do
some
translation
for
you.
B
Before
we
started
building
plus
hland,
we
looked
at
some
of
the
existing
tools.
There
are
some
great
man
if
I
space
link
the
tools
like
you
cube,
lint
and
copper,
which
are
ready
to
use
in
your
CI
Singh.
These
systems
to
check
if
the
yanil
is
sound
before
deploying
onto
the
cluster
Buckland
identifies
issues
with
your
cluster
after
the
workloads
being
deployed.
So
they
could
be
platform
specific
or
that's
something
that
you
would.
The
animal
would
be
sound
for,
but
would
create
an
issue
later.
B
On
you're
interested
in
security
spell
checks
specifically
than
there's
cube
bench.
Cube
bench
can
be
deployed
as
an
in
cluster
tool
as
a
workload
itself,
and
it's
gonna
start
reporting
issues
after
you.
Deploy
it
on
to
your
cluster
plus
length
is
different
from
that,
because
it
works
externally
runs
the
checks
on
objects
by
accessing
the
kubernetes
api.
B
If
we
want
to
focus
on
workload,
health
and
get
a
dashboard
for
it,
there's
Polaris
we
intentionally
did
not
want
to
focus
on
our
workload.
Help
because
there's
the
kubernetes,
dashboard
and
other
observables
are
there
to
give
you
this
information
in
terms
of
the
checks
pop
icons
closest
to
the
class
length.
I.
Think
the
difference
between
the
two
tools
is
that
class
length
also
has
this
ability
to
provide
specifics.
A
So,
finally,
before
we
wrap
up,
we
would
love
help
with
custer
lint
and
assuming
that
most
of
the
people
in
this
room
use
kubernetes
in
some
way.
There's
a
couple
of
ways
that
most
of
you
can
help
us
if
you're
interested
one
is
to
just
give
it
a
try
on
your
clusters
and
let
us
know
if
it
doesn't
work
or
if
it
has
surprising
results.
A
If
you
have
false
positives
or
false
negatives,
things
that
you
didn't
expect,
that's
something
we'd
like
to
know
the
other
is
we
would,
as
I
mentioned
before,
love
to
have
more
checks,
and
especially,
if
you're
running
in
an
environment,
that's
not
do
KS.
We
would
like
to
know
what
are
the
specific
gotchas
in
those
environments,
so,
if
you're,
using
e,
KS
or
gk
e
or
you're
running
on
pram
on
hardware,
or
something
like
that,
we
would
love
checks
for
those
environments
to
add
check
groups
to
the
tool
that
covers
those
environments.
A
I
think
some
of
the
checks
that
we
have
for
do
KS
are
probably
applicable
to
other
clouds.
So
if
would
start
with
those
and
sort
of,
let
us
know
what
they
think
is
applicable.
We
would
would
love
pull
requests
or
issues,
it's
all
up
and
github,
so
yeah.
Let
us
know
what
you
find
and
that's
all
we
wanted
to
say
today,
I
see
on
the
side.
Our
email
addresses,
if
you
want
to
get
in
touch,
there's
also
another
link
to
the
github,
so
go
check
out.
A
C
D
A
So
right
now
that
the
check
library
is
totally
static,
checks
registered
themselves
when
the
tool
starts
and
it's
it's
all
static.
It
would
be
interesting
to
maybe
use
like
go
dynamic
libraries
for
this
to
provide
that
ability
to
do
like
really
org
specific
ones,
but
it's
not
something.
We've
been
planning
right
now.
So
the
interesting
idea.
F
Hey
so
I
really
quick
question
so
just
about
optimization
I
saw
the
linting.
It
seemed
like
a
relatively
small.
You
know,
example
cluster.
What
would
you
say
the
standard
run
time
would
be
for
something
that
maybe
is
because
the
linting
is
gonna,
be
done
after
you've
applied
the
configurations
right,
it's
gonna
be
done.
What
is
that?
What
am
I
doing
right
now?
So
it's
gonna
be
generally
from
what
I
see
used
for
enterprises
and
organizations
that
are
pretty
deep
in
already
and
need
to
see
like.
A
So
the
I
would
say
the
slow
part
of
cluster
lint
is
fetching
all
the
resources
from
the
API
server.
Once
we've
done
that
the
checks
run,
I
didn't
memory
object,
so
they're
pretty
fast.
You
know,
I
mean
if
you
had
a
huge
number
of
pods,
it
might
take
some
milliseconds
to
iterate
over
them,
but
in
general
it's
pretty
quick
I.
Think
that
in
terms
of
optimizations
being
able
to
limit
it
more
to
like
a
namespace
or
a
particular
set
of
objects,
that
might
be
helpful
thing
anything
that
we
can
do
to
kind
of
reduce.
G
G
A
A
Yeah
and
you
could.
You
could
build
it
into
your
CD
pipeline
with
close
friend,
I.
Think
clustering
is
really
more
focused
on
like
proactive
identification
of
things
that
aren't
problems
right
now,
but
will
be
problems
in
the
future
I,
so
really
identifying
like
things
that
are
gonna
break
when
you
do
something
disruptive
or
if
you
lose
it
or
things
like
that,.
H
A
Lint
sure
I
I
think
it's
not
something
we
thought
of,
but
it's
an
interesting
idea
it
it's
each.
Each
of
the
checks,
like
we
showed
in
the
API
actually
just
returns
to
structure,
so
the
outputting
of
that
information
is
decoupled
from
the
actual
check.
So
it
would
be
fairly
straightforward
to
add
another
output
format.
If
people
wanted
it.
A
Right
yeah
I,
just
mention
the
the
grouping
of
the
checks
is
a
little
bit
like
grouping
of
test
Suites,
it's
a
little
bit
different
in
the
a
check
can
live
in
multiple
groups
and
it
can
also
live
in
no
groups.
If
they're,
if
you
have
a
check,
that's
like
kind
of
an
outlier,
it
doesn't
have
to
be
in
a
group.
So
it's
a
yeah
a
little
bit
different
than
that
scenario,
but
yeah.