►
From YouTube: K8s Labels Everywhere! Decluttering With Node Profile Discovery. - Conor Nolan & Dave Cremins, Intel
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
K8s Labels Everywhere! Decluttering With Node Profile Discovery. - Conor Nolan & Dave Cremins, Intel
A recent CNCF community survey showed that 57% of respondents have 100+ machines in their fleet and 17% have more than 5000 machines (including VM, bare metal etc.). When managing such broad and diverse clusters, variations in node capabilities and features are inevitable. So how exactly are individual features tracked on a node-by-node basis? Node Feature Discovery (NFD) is commonly used for basic feature discovery and labelling across a Kubernetes cluster. This talk, however, introduces a new component: Node Profile Discovery (NPD). NPD provides an extra layer of abstraction from NFD, alleviating the burden of managing individual features. NPD is designed to work in conjunction with NFD, aggregating individual features into higher level profiles and applying these profiles to suitable nodes. This talk will show how NPD can make life easier for application developers and sys-admins alike.
A
A
A
We
also
intend
to
provide
a
conceptual
overview
of
node
profiles
and
we
have
orchestrated
a
very
simple
demo
that
highlights
the
the
problem
and
presents
a
possible
resolution
to
the
to
the
issue.
And
finally,
we
will
have
some
key
takeaway
summaries
for
you
guys
too
so
before
discussing
the
problem
statement
at
hand,
let's
focus
on
what
nfd
is
today.
A
Given
that
numerous
different
types
of
workloads
need
to
or
have
need
special
attention,
let's
say
to
specific
features
of
a
platform.
So,
for
instance,
you
know
if
some
platform
had
a
particular
feature
that
was
required
as
part
of
your
workload.
Then
we
want
to
ensure
that
the
workload
lands
on
a
compute
node
that
has
the
the
intended
feature
available,
but
again,
even
though
this
helped
in
the
the
placement
of
workloads
or
at
least
complemented
the
placement
of
the
workloads,
it
still
kind
of
promoted.
A
This
tight
coupling
to
individual
platform
capabilities-
and
you
know,
if
I
drill
into
that.
What
I'm
really
saying
is-
is
that
your
workload
was
very
individual
feature
aware.
So
each
feature
that
was
required
or
intended
to
be
leveraged
by
your
workload
needs
to
be
specified
up
front.
And
that's
what
I
mean
by
thai
coupling
and
when
we
look
at
this
across
numerous
different
types
of
workloads
and
especially
from
my
background,
where
it's
really
based
in
telco
workloads.
A
There
are
a
lot
of
features
required
from
the
platform
in
order
for
a
workload
to
run
deterministically
and
with
the
right
level
of
performance
and
throughput.
And
when
we
look
at
what's
expected
of
the
platform,
it
can
become
a
configuration
nightmare
very
quickly.
It
really
adds
to
the
complexity
and
not
to
mention
that
it
becomes
that
bit
more
difficult
to
schedule
when
you
have
so
many
different
requirements
specified
as
part
of
your
either
workload
or
your
pod
specification.
A
Also,
you
know
when
we
look
at
the
discovery
mechanisms.
There
are
actually
multiple
ways
to
do.
Detection
and
labeling
nfd
is
just
one
there's
also
net
node
labeler
and
there
are
other
various
components
out
there
as
part
of
the
ecosystem
that
are
also
capable
of
labeling
their
their
nodes
or
labeling
their
their
features
as
well
and
making
them
available.
A
But
with
each
offering
and
each
component.
That
is
capable
of
doing
that.
We
essentially
extend
the
amount
of
features
that
are,
let's
say,
claimable
or
schedulable,
let's
say
in
kubernetes,
and
we
end
up
creating
this
laundry
list
of
all
these
different
features
and,
as
I
say,
this
can
become
unmanageable
and
adds
extra
level
of
complexity,
and
you
know,
presents
new
levels
of
scheduling,
headaches
and,
and
the
number
of
features
continue
to
grow,
and
this
is
the
existing
pattern.
A
You
know
this
is
what's
out
there
today
and
you
know,
as
every
couple
of
months
there'll
be
new
feature.
Detections
added
to
nfd,
for
instance,
and
there'll
be
new
node
labels
available
and
if
a
workload
does
need
a
specific
feature
capability
out
of
a
platform,
it's
going
to
be
added
somewhere
so
that
it
can
be
claimed.
A
But
again,
this
particular
pattern
focuses
too
much
on
the
individual
features
and
when
we
do
that,
when
we
focus
on
individual
features
for
a
workload
we
tend
to
misalign
on.
The
abstraction
layers
provided
by
kubernetes
like
kubernetes,
is
a
stable
api
with
the
right
level
of
abstraction,
and
you
know
when
we
look
at
it
and
compare
it
to
that.
A
We
kind
of
break
the
abstraction
by
tying
our
workloads
to
specific
individual
platform
capabilities.
You
know,
so
the
point
of
my
slide
here
today
really
is
to
try
and
pivot
towards
a
platform-centric
perspective
right.
So
no
longer
do
I
need
to
be
concerned
with
all
of
the
individual
feature,
capabilities
of
a
particular
node.
Instead,
I
want
to
be
able
to
avail
of
a
platform
offering
you
know
holistically
speaking.
What
can
my
my
nose
offer
me?
A
So
I
can
land
my
my
workload
over
there
when
we
look
at
it
under
that
lens,
we
start
to
perceive
it
as
kind
of
like
an
almost
an
optimized
accelerator
for
your
specific
workload.
An
example
in
this
case
would
be
something
like
you
know:
a
power,
efficient
power,
efficient,
packed
processing
node.
So
when
I
have
a
packet
processing
workload,
I
don't
need
to
be
concerned
with
all
of
the
individual
feature
items
that
I
need
for
that
workload.
A
A
In
my
next
slide,
I
wanted
to
showcase
an
example
of
a
very
complicated
specification.
This
is
taken
from
a
real
real
world
application.
It
is
telco
oriented,
but
it
gives
you
guys
an
idea
of
how
fast
things
can
you
know
become
unmanageable
and
out
of
control
when
the
the
list
of
feature
requirements,
be
it
for
your
node
or
your
workload
continues
to
grow.
A
We
put
it
in
video
format
because
we
couldn't
fit
it
on
the
slide
and
I
think
we're
almost
done
so
as
well
as
evident,
or
at
least
as
showcased.
By
that,
then
I
think
it
should
be
clear
that
you
know
there
are
a
lot
of
opportunities
for
a
really
large
level
of
complexity
within
you
know
feature
capability
detection,
not
just
with
nfd,
but
the
the
the
feature
detection
mechanism
itself,
so
that
was
an
example
of
how
complicated
a
specification
can
become
and
how
hard
it
can
be
to
to
manage.
A
B
B
So
if
I'm
the
scheduler
and
I'm
going
through
my
filtering
process,
I'm
going
to
look
at
this
part
and
the
label
specified
and
say:
okay,
so
this
pod
wants
a
sni
crypto
instruction,
so
node
5
doesn't
have
that
so
that
gets
filtered
out.
This
part
wants
avx
512
instruction
set,
node
1
has
avx
but
doesn't
have
avx
512.
So
that's
no
good.
This
part
wants
sstbf,
so
speed,
select
technology
base,
frequency,
node
7
doesn't
have
that,
and
this
part
also
wants
layer.
B
B
So
let's
say
the
clustered
main
has
configured
some
nodes
in
the
cluster
to
run
the
single
luminal,
topology
manager
policy
and
then
they've
applied
a
label
to
those
nodes
to
signify
that
likewise
they've
configured
some
nodes
with
isil
cpus
and
applied
a
label
to
those
and
also
represent
that
as
well.
So
this
pod
wants
a
node
where
it
will
be
guaranteed
pneuma
alignment,
so
that
rules
out
nodes
two
and
four
and
finally,
the
pod-
wants
to
run
on
a
node
configured
with
iso
cpus.
So
node
three
is
no
good
there.
B
So
now,
instead
of
polluting
our
pod
spec
with
a
bunch
of
individual
features,
let's
create
a
node
profile
cr
containing
all
the
features
we
want
to
make
up
a
profile
that
best
suits
our
workload.
So
once
the
cr
is
created,
no
profile
discovery
reacts
and
applies
a
new
profile
label
to
any
nodes
that
fit
that
profile.
B
B
So
I
have
a
four
node
cluster
three
worker
nodes
in
the
control
plane
node.
If
I
look
at
the
pods
that
are
run
so
you
can
see
here,
the
nft
master
is
running.
I've
got
an
nfd
worker
on
each
of
my
worker
nodes
and
down
here
we
can
see
that
the
no
profile
discovery
operator
is
running.
So
this
is
a
single
pad
deployment,
so
there's
no
need
for
individual,
no
demons
or
for
host
level
discovery
or
anything
like
that.
It's
just
a
single
point
of
contact
with
the
kubernetes
api.
B
Let's
take
a
look
at
the
labels
on
our
worker
nodes,
so
we
can
see
up
here
on
cube
worker
one
as
a
bunch
of
feature
labels
applied
by
nfd
cube
worker
two,
also
a
host
of
nft
labels.
We
can
also
see
a
couple
of
custom
labels
kind
of
labels.
We
touched
on
earlier,
like
a
specific
topology
manager
policy
or
a
node
configured
with
isil
cpus
and
cube
worker
3
watch
the
same
mix
of
nfd
labels
and
a
couple
of
custom
labels.
B
This
is
a
simple,
no
profile.
Cr,
it's
been
given
the
name,
high
performance
pack
processing
and
in
the
spec.
Our
number
of
desired
feature
enables
to
make
up
this
profile
so
again
like
in
the
example
we
have
before,
we
have
asni
avx,
512,
sdbf
layer,
3,
cache,
icpus
and
topology
manager
policy
create
that
profile.
B
B
B
B
So,
for
example,
if
I
was
to
remove
the
topology
manager
feature
label
from
the
spec
and
then
reapply
that
spec
again
I'll
just
leave
it
open
for
reference,
and
now
we
go
back
and
check
our
labels
once
more,
we
can
see
that
now,
nodes,
2,
cube
worker,
2
and
cube
worker
3
both
fit
the
profile
and
are
given
the
label.
The
profile,
high
performance
pack
processing
label,
because
both
of
these
nodes
now
match
the
criteria
listed
over
here.
Now
that
the
topology
manager
policy
has
been
removed.
B
So
what
we're
aiming
to
achieve
here
with
these
additional
attained
behavior
parameters
is
that
50
of
nodes
which
fit
the
high
performance
profile
should
be
tainted
with
a
no
schedule
taint.
So
in
our
small
example,
we
have
two
nodes
now
labeled
with
the
high
performance
profile
and
so
we're
looking
for
one
of
those
nodes
to
be
tainted.
B
B
B
And
this
time
I'm
going
to
check
the
taints
of
my
worker
nodes,
so
we
can
see
here,
there's
no
values
for
cube
worker
1
and
cube
worker
2,
but
for
cube
worker
3,
we
can
see
that
it
has
been
tainted
with
the
high
performance
profile
and
the
no
schedule
change.
So
this
is
what
we
were
expecting
to
see.
We
had
two
nodes
labeled
with
the
correct
high
performance
profile
and
we
want
to
retain
50
of
those
which
is
one
and
that's
what
happened
so
also.
B
B
So
this
allows
workloads
with
tolerations
to
target
this
node,
specifically
as
opposed
to
non-designated
nodes,
which
also
match
the
profile,
and
this
information
is
also
reflected
in
our
cr.
So
if
we
describe
our
high
performance
profile,
we
can
see
down
here.
So
this
is
the
spec
with
the
feature
labels
that
we
specified
and
the
chain
behavior
that
we
set,
and
here
in
the
status
you
can
see
so
two
nodes
were
labeled,
I.e,
two
nodes
match
the
profile
and
we
specified
we
wanted
50
of
those
tainted,
and
you
can
see
here.
B
B
So
what
that
means
is
that
we
want
to
change
basically
all
of
the
nodes
which
match
the
profile
so
I'll
reapply
that
cr
and
again
I'll
check
the
teens.
And
now
we
can
see
that
workers
node
worker
nodes,
two
and
three
are
both
changes,
and
likewise,
with
our
labels,
we
should
see
that
again
value
for
our
profile
label
has
changed
to
tainted,
and
if
I
check
my
cr
again,
we
should
see
that
reflected
in
the
status.
B
So
again,
this
is
updated
here,
like
I
said
we
we
specified
100
and
that's
what
we've
been
given
all
nodes
which
match
the
profile
are
now
tainted,
I.e
designated
for
a
specific
workload.
Okay,
we've
seen
what
it
is
and
how
it
works.
So
now,
let's
cover
why
you
would
use
no
profile
discovery
and
the
advantages
it
can
provide.
B
So
what
do
we
mean
by
this?
We
want
to
get
away
from
the
bottom-up
mindset
of
my
app
needs.
This
feature
this
feature
and
this
feature
in
order
to
fulfill
a
certain
quality
of
service.
Instead,
let's
move
towards
a
my
app
needs
to
fulfill
a
certain
quality
of
service.
So
what
make
up
our
profile
of
features
is
going
to
achieve
that
for
my
application,
and
then
we
can
align
these
profiles
of
these
makeups
with
the
abstractions
that
are
most
prevalent
in
today's
deployments.
B
So
cluster
slicing,
no
profile
discovery
naturally
lends
itself
to
a
model
of
cluster
slicing
or
partitioning
of
a
cluster
into
groups
of
nodes
with
common
use
cases,
and
the
demo
showed
us
some
functionality
for
designating
nodes
for
specific
performance,
critical
workloads
and
then
providing
this
mechanism
in
a
lightweight
and
scalable
way.
That's
kubernetes
native
and
could
potentially
remove
a
lot
of
overhead
for
a
cluster
admin.
A
So
if
we
do
have
proliferation
of
this
type
of
mechanism-
and
you
know,
new
patterns
emerge
based
on
utilizing
something
like
this,
you
know,
there's
always
the
question
of
what
could
we
do
next?
Could
we
build
on
that?
Is
there
another
avenue
of
work
we
could
look
at
and
there
is
so
we're.
Also
looking
to
you
know,
based
on
you
know
your
profiles,
you
know
so
like
it,
it's
easy
to
have
this
get
out
of
control
and
to
protect.
You
know
the
integrity
of
of
profiles.
A
You
know
we're
looking
at
a
json
schema
to
validate
and
promote
consistency
of
them.
You
know,
and
if
we
do
you
know,
start
work
in
that
particular
domain.
Then
it's
easy,
then,
to
tie
this
into
you
know
existing
automation,
pipelines
and
validate
the
the
creation
of
new
profiles
right.
So
I'd
see
this
as
something
that
would
be
beneficial
going
forward.
A
A
A
We've
also
looked
at
a
potential
integration
point
with
nfd,
whereby
we
build
two
separations
and
one
to
focus
on.
You
know
profile
concerns
that
can
then
act
as
a
complementary
component
to
profile
management.
So
nfd
are
profile
concerns
and
management
of
them
could
be
done
in
one
component
and
nfd
could
manage
the
actual.
A
The
labels,
the
profile
labels
themselves
and
we've
also
seen
some
use
cases
or
potential
possibilities
with
policy-based
control
systems,
whereby
you
know
like
policy
system,
could
create
the
profile
based
on
its
own
policies,
and
this
is
something
that
you
know
could
be
leveraged
in
kubernetes.
Given
that
it's
it's
policy
or
it
can
be
policy
heavy
depending
on
your
infrastructure
and
platform.
A
So
like
there
are
numerous
things
that
we
can
do
next
with
this
particular
approach,
so
I
I
hope,
you've
you've
enjoyed
today's
talk.
It's
I
wouldn't
say
that
it's
too
much
of
a
push
beyond
what
nfd
does
or
what
you
know,
features
of
feature
labels
have
brought
to
the
the
community
or
to
the
cabrillo's
ecosystem.
But
it's
definitely
you
know
bringing
an
awareness
to
your
your
kind
of
your
platform,
your
holistic
platform
capabilities
versus
the
individual
items
that
are
prevalent
today.