►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Scavenging for Reusable Code in the Kubernetes Codebase - Kevin Lingerfelt, Buoyant
The magic of open source software and Go's support for remote import paths makes it trivial to import and run code from many of the most popular CNCF projects directly in your own project. Sometimes, however, the process of finding that one piece of code in a large codebase that’s just right for your use case can feel like a scavenger hunt. But the spoils of the hunt are vast. This talk explores how the Linkerd project has leveraged the Kubernetes codebase to replace their bespoke code with existing code that's more robust and better tested. Specific examples include switching the project to use Kubernetes' shared informers for caching API responses and adding rate limiting via work queues for TLS certificate distribution. Linkerd's Kevin Lingerfelt will show you how to undertake an effective Go scavenger hunt, and he'll discuss when you should skip the hunt and write it from scratch.
https://sched.co/MPXB
A
Prospective
on
how
the
kubernetes
project
is
laid
out,
like
I,
said
I'm,
not
a
kubernetes
developer,
so
it's
gonna
be
fairly
limited
in
its
scope,
but
you
really
need
a
good
map.
If
you're
gonna
figure
out
where
you're
going
in
the
codebase,
and
then
we
have
two
missions
to
go
on
to
find
some
really
cool
technology
and
then
I'll
wrap
it
up
with
a
few
lessons.
A
Any
good
scavenger
needs
really
good
tools.
So
these
are
the
tools
that
I
use,
I
call
them
the
five
g's
get
grep
google
go
doc
and
then
go
run
tests
I
go
basically
what
I
mean
by
the
last.
One
is
sometimes
it's
easier
for
me
to
explore
the
libraries
in
kubernetes
or
in
any
large
go
project
by
writing
some
simple
throwaway
scripts,
then,
rather
than
trying
to
do
it
fully
within
the
context
of
my
go
project,
so
that's
an
approach
that
they
take
a
lot
and
they
I'll
demonstrate
it.
A
Let's
see
in
the
bash
on
the
left,
it's
a
little
bit
of
pseudo
script.
I!
Don't
actually
run
that
that
often,
but
when
I
want
to
explore
the
kubernetes
codebase
Pinchot,
specific
revision
will
check
out
the
main
repo
and
then
I'll
go
through
and
check
out
all
the
staging
repos
and
advance
them
all
to
the
revision
that
I
want
and
have
like
a
cohesive
view
of
the
code
that
way
cool
everyone.
I
just
I,
don't
run
over
time
cool
okay.
So
let's
talk
about
linker,
D,
real,
quick,
linker
D
is
a
service
mesh.
A
It's
a
CN
CF
project,
it
has
lots
of
users
and
it
provides
observability,
reliability
and
security
for
your
applications
running
on
kubernetes
there's
a
lot
of
other
talks.
You
can
go
to
to
learn
more
about
G,
but
I
will
sort
of
just
talk
about
the
part.
That's
relevant
to
this
talk
right
now,
so
lincolni
is
divided
into
a
control
plane
and
a
data
plane.
The
data
plane
consists
of
proxies
that
run
alongside
your
application.
A
They
use
the
kubernetes
pod
model
and
the
proxies
handle
all
inbound
and
outbound
traffic
for
the
application,
and
then
the
proxies
receive
their
configuration
from
a
control
plane
and
the
control
plane
also
pulls
all
the
telemetry
data
out
of
the
proc
so
that
it
can
be
displayed
in
the
web
UI
of
the
CLI
that
kind
of
thing.
So
this
is
sort
of
the
overall
architecture
of
linker
D
the
control
plane,
though
the
proxies
are
written
in
rust.
A
In
the
data
plane,
the
control
plane
is
written
in
go
and
there
are
a
lot
of
parts
of
the
control
plane
that
really
are
built
on
top
of
kubernetes
and
I'm.
Gonna
focus
on
two
of
them.
In
this
talk,
the
first
one
is
the
CLI,
so
we
have
a
linker.
Dcl
is
very
similar
to
coop
petal
and
second
one
is
the
tap
server
so
tap.
A
It's
a
feature
that
allows
us
to
tap
into
live
traffic
and
it
receives
a
bunch
of
raw
routing
data
from
the
routers
and
needs
to
make
sense
of
it
and
it's
written
and
go
so
a
lot
of
things
in
the
controller
pod
operate
like
kubernetes
controllers.
So
if
you're
familiar
those
that
will
help
okay
here,
we
go
I,
have
sort
of
bit
off
an
enormous
task.
A
Okay,
so
we'll
start
at
the
kubernetes
project,
it's
the
original
repo
I
guess.
It
looks
like
that
when
you
LS
it
there's
a
lot
of
stuff
here
that
I
don't
understand.
Most
of
my
entry
points,
though,
for
the
project
are
through
the
either
the
command
directory
CMD,
which
contains
all
the
mains
for
all
the
processes
that
run
in
kubernetes.
A
So
this
has
the
controller
manager,
the
API
server,
the
couplet
coop
cuddle,
it's
kind
of
like
the
domain,
the
mains
for
all
of
those
processes
and
then
a
lot
of
the
meat
of
the
implementation
of
those
processes
lives
in
the
package
directory.
There's
a
lot
of
other
stuff
in
the
package
directory,
but
that's
sort
of
where
I
start
and
then
I
am
not
too
familiar
with
the
history.
The
kubernetes
project,
but
I
take
it
at
some
point.
A
They
decided
to
split
it
up
into
multiple
separate
projects
and
those
projects
ended
up
in
the
staging
directory
and
in
order
to
sort
of
continue
to
build
the
project.
Sanely
the
projects
and
the
staging
directory
on
publish
or
publish
out
to
other
repos,
but
they're
sort
of
developed
in
the
main
repo
I
think,
but
there's
a
lot
of
stuff
in
there
if
you
dig
into
the
staging
repos.
So
these
are
all
separate
projects
that
have
live
in
separate
repositories
but
work
in
coordination
with
kubernetes,
and
that's
not
all.
A
If
you
look
even
farther
there's
more
projects
that
are
in
the
case
do
or
that
started
outside
of
the
repo
itself,
so
they
aren't
in
the
staging
directory.
So
we're
not
just
talking
about
the
community's
repo
we're
talking
about
40
of
these
projects,
basically
that
we
could
dig
through
and
rummage
around
in,
but
I
don't
actually
know
a
lot
about
all
of
those
projects.
So
I
will
stick
with
the
ones
that
I'm
familiar
with,
which
is
CLO.
A
Cli
runtime
for
building
coop
cuddle
and
cuddle,
like
commands,
seems
relevant
to
the
linker
T
project
client
go
which
is
the
main
repo
for
interfacing
with
kubernetes
api,
and
then
that's
also
very
tightly
coupled
to
the
API
and
API
machinery.
Repos
API
has
the
schema
for
the
eye
itself.
It's
a
lot
of
protobuf
and
API
machinery
is
the
libraries
and
interfaces
for
interacting
with
the
API,
and
we
use
a
couple
other
libraries
too,
but
we're
not
gonna
talk
about
that
in
this
talk.
A
Okay,
so
we've
made
it
to
our
first
mission.
First
scavenging
mission
into
the
kubernetes
codebase.
There
is
a
very
common
pattern
when
you
use
linker
d
of
injecting
the
practice
so
I
mentioned
that
the
data
plane
has
these
proxies
in
your
pod
model.
So
we
call
that
injection
when
the
pod,
the
container
for
the
proxy
gets
put
into
your
hot
spec,
and
so
typically
the
way
that
works
is.
A
Where
is
linker
deployed
prints?
Appointment,
hello
in
quotes
clearly
there's
code
somewhere
in
the
the
domain
of
coop
cuddle
that
matches
these
strings
together
in
some
way
and
we're
not
doing
it
the
same
way,
but
we
probably
could
if
we
could
just
find
the
code.
That
does
that
so
that's
kind
of
what
what
I
set
out
to
do
one
day,
and
so
we
can
see
if
we
can
fix
it.
A
Okay,
so
we're
about
to
enter
the
zone
with
all
of
the
amazing
alien
technology.
Like
I,
said:
I'm
gonna
put
a
lot
of
stuff
up
on
the
screen
and
I
hope
it's
sort
of
coherent
to
walk
through
it
and
at
the
end
we
can
follow
up
with
any
questions.
If
you
have
them
so
we
start
our
journey
in
the
criminals,
repo
namespaces.
So
for
our
very
very
rough
implementation
of
the
find
pod
by
IP,
we
can
take
the
lists,
all
pods
approach,
where
we
create
a
new
config.
A
We
call
this
pods
and
then
we
just
iterate
through
every
pod,
and
we
return
the
one
that
has
the
matching
IP
I
am
gonna
rate.
This
approach
three
Frannie
faces
because
it
is
very
inefficient.
Every
time
you
need
to
look
up
a
pod
by
IP.
It
doesn't
make
sense
to
pull
every
single
pod
in
every
single
namespace
and
try
to
try
to
find
it,
and
we
can
do
a
lot
better
and
we
do
a
lot
better
without
a
lot
more
sophistication
from
the
kubernetes
api,
because
kubernetes
also
includes
watching
you
know.
A
Every
api
endpoint
has
a
watch
as
well,
which
will
send
you
incremental
updates
as
things
change.
So
this
approach
is
a
little
more
complicated
because
we
need
to
have
a
map
with
concurrent
access
that
we
maintain
a
memory
of
pods
by
their
IP
address.
So
that
is
this
struct.
It
has
a
mutex
and
really
it's
just
a
map
of
string
to
pod
to
very
basic
functions,
one
to
add
things
to
the
map
and
one
to
retrieve
things
from
the
map,
and
with
that
we
can
sort
of
rewrite
our
list.
A
Pods
approach
to
set
up
a
new,
a
new
map
and
then
call
watch
and
then
in
a
separate,
go
routine.
We
can
process
the
watch
events
and
populate
the
map
and
then
once
we
start
the
Guru
team,
we
just
sleep
for
a
second,
because
let's
hope
that
that's
how
long
it
takes
for
all
the
watch
events
to
propagate
through
and
for
the
map
to
be
completely
populated
before
our
script
precedes.
A
And
after
that
we
can
just
retrieve
from
the
map
and
we
are
able
to
return
the
pot.
This
roach
gets
three
neutral
phases
because
we'll
sleep
for
one
thing,
but
it's
also
just
sort
of
a
lot
of
machinery,
and
it's
also
hard
to
know
when
the
map
is
fully
populated
and
if
we
make
a
mistake
processing
our
watch
events
there's
no
type
of
reconciliation
right.
We
could
end
up
like
really
out
of
sync
with
what's
actually
in
the
API.
A
So
at
this
point
in
the
development
process,
like
our
group
starts
to
think
like
well,
there's
probably
a
better
way
to
do
this
and
we
start
using
a
different
scavenging
tool
that
I
like
to
call
Google
where
we
start
start
just
fumbling
around,
for
how
to
cache
responses
with
client
go,
and
that
takes
us
to
the
cache
package
from
client.
Go-
and
this
is
a
big
package
package-
cache
is
a
client-side
caching
mechanism
and
then
it
is
useful.
A
Reducing
the
number
of
server
calls
you
would
otherwise
need
to
make
sounds
great
reflectors
the
reflector
watches
a
server
and
updates
a
store,
less
clear
what
that
means,
and
then
two
stores
are
provided,
one
that
simply
caches
and
one
that
acts
as
a
FIFO
queue
so
anyway,
I
think
we're
in
the
right
place.
So
at
this
point,
I
kind
of
scroll
up
and
down
the
page
and
I
get
some
information.
So
package
cache
store,
isn't
object.
Nervous
queue
is
like
a
store,
but
there's
also
a
heap
in
here.
A
There's
a
reflector
which
watches
a
resource
and
updates
a
store.
There's
a
thing
called
config,
which
produces
a
controller
controller,
is
not
documented.
All
actually
still
don't
know
what
a
controller
is
from
the
cache
package,
but
there's
something
called
an
indexer
and
hey.
We
were
building
an
indexer
in
our
previous
implementation
and
it's
a
storage
interface
that
lets
you
list
objects
using
multiple
indexing
functions.
A
A
So
at
this
point,
I'll
go
back
to
the
grep
scavenging
tool
and
search
the
code
base
and
actually
not
a
lot
of
places
are
using
the
new
indexer
function,
but
the
couplet
is,
and
if
it's
working
for
the
Kubla,
it's
probably
good
enough
for
us.
So
we
can
look
at
that
implementation,
and
here
it
calls
new
indexer
and
it
creates
a
index
based
on
the
namespace
of
the
service
I
think
and
rather
than
go
through
this
whole
example.
A
A
A
Basically,
you
create
a
new,
oh
another
thing
that
he
uses
is
new,
config
or
Die,
which
we
were
not
using
in
our
own
code
base,
but
it's
pretty
fun.
So
we
can
improve
our
script
with
that
and
then
yeah
you
get
a
new
shared
and
former
factory
and
from
that
produces
informers
for
various
types
of
API
objects,
like
secrets
and
namespaces,
and
in
order
to
not
run
out
of
time,
we'll
just
rushing
this.
So
this
is
now
the
way
we've
updated
our
code.
So
we
of
our
new
config
or
die.
We
create
our
shared
informers.
A
We
grab
the
pot
Informer
and
we
add
an
indexer
to
it.
Using
the
same
pod
by
IP
index
function
that
we
had
already
created,
we
call
shared
informers
start
and
then
there's
actually
a
cache
utility
that
we
found
in
the
cache
package
that
lets
you
wait
until
shared
informer,
succinct,
the
implementation
that
is
exactly
like
the
imitation
we
had.
A
It
just
checks
every
100
milliseconds
to
see
if
it
has
synced,
but
it's
a
lot
less
code
in
our
own
codebase
and
then
at
that
point
it's
able
to
read
the
IP
from
the
index,
so
I
would
say
we
have
found
it
we've.
Sort
of
this
is
the
trove
of
alien
technology
that
we
can
really
use
to
our
advantage
and
there's
a
couple
things
that
we
get
for
free
by
adopting
shared
informers.
A
One
of
them
is
that
you
can
configure
shared
informers
to
run
in
a
restricted
mode,
so
we
have
a
need
to
run
our
control
plane
with
limited
are
back
permissions.
It
would
typically
read
all
resources
in
all
namespaces
in
the
cluster,
but
if
we
wanted
to
run
it
in
a
single
namespace,
we
would
have
to
sort
of
make
sure
we're
not
trying
to
read
any
objects
from
any
other
namespaces,
and
you
might
think
that
that's
like
a
lot
of
code
but
shared
informers
already
has
an
implementation.
A
That's
restricted
to
a
single
namespace,
and
this
is
the
entirety
of
the
change
to
have
all
the
existing
code
work
in
a
single
namespace.
So
it
really
helps
us
configure
it
in
that
way
and
there's
other
ways
that
you
can
configure
shared
informers
too
and
then
the
other
thing
you
get
is
straightforward
test
fixtures.
So
if
you're
already
using
the
kubernetes
client
interface,
there's
a
really
good
new,
like
a
fake
client
that
you
can
use
that
you
can
pre
populate
with
API
objects
and
then
those
will
be
returned
immediately.
A
So,
in
this
example,
I
have
just
a
stubbed
IP
address
and
I
create
a
fake
pod
associated
with
addressing
when
I
initialize
the
client
I
pass
in
the
pod
and
all
the
rest
of
the
code
runs
as
expected,
and
it's
able
to
find
the
pod
by
IP
with
just
that
amount
of
change
in
the
script.
Okay,
so
I
just
have
a
couple
more
minutes
to
stick
with
me
and
we
will
go
through
the
finale,
but
you
know
I
think
we
did
a
good
job
here.
We
got
some
intelligence
about
the
lincolni
codebase.
A
We
sort
of
set
up
a
rough
mental
map
of
the
kubernetes
zone,
planned
out
a
few
routes
executed
on
those
routes
returned
with
some
interesting
technology
and
use
some
tools
of
the
trade
get
grab.
Google
go
duck
that
kind
of
thing.
So,
at
the
end
of
all
this
I
want
to
leave.
You
is
just
a
few
more
thoughts.
A
One
of
them
is
research
in
advance.
Like
a
lot
of
this
information
that
I
have
told
you
is
out
there
in
bits
and
pieces
on
the
internet,
it's
in
talks,
it's
in
blogs,
you
know,
I,
don't
actually
have
my
finger
on
the
pulse
of
what
the
kubernetes
project
is
doing.
So
it's
a
little
hard,
sometimes
to
just
jump
in
cold
turkey.
So
it's
it's
helpful,
I
think
to
sort
of
do
some
do
some
scavenging
outside
the
code
base
before
you
really
dive
in
full
force.
A
Yeah.
Another
thing
is
reusable
examples,
so
in
the
staging
directory
of
the
project.
There's
all
these
libraries
so
client
go
is
one
of
them.
Clank
o.
Has
this
examples
directory?
That's
really
rich
for
people
just
getting
started
with
that
library.
It
would
be
great
to
see
an
examples
directory
for
every
other
staging
project
in
the
repo
that
shows
how
to
use
that
project
in
isolation
from
the
rest
of
the
projects.
For
instance,
the
LA
runtime
could
have
an
example
project
of
building
like
a
really
small
CLI
that
interacts
with
like
kubernetes.
That
kind
of
thing.
A
Another
point
to
make
is
sort
of
based
on
my
combing
of
the
cache
package
go
doc.
Is
maybe
not
sufficient
for
fully
documenting
how
all
the
pieces
of
a
package
fit
together?
I
think
it's
really
good
for
documenting
the
individual
pieces,
but
if
we
could
figure
out
a
way
to
get
kind
of
a
higher
level
overview
and
steer
people
in
the
right
direction,
I
think
that
would
be
really
helpful
and
in
some
cases
I
think
it
would
be
good
to
break
up
the
packages
into
smaller
pieces.
A
It's
not
like
an
ultimatum,
but
I
do
feel
like
it's
a
little
easier
to
wade
through
some
of
these
packages
if
they
come
in
a
little
smaller
chunks.
At
least
that's
been
my
experience
so
far
so
anyway.
So
that's
all
I
really
have
to
say
on
the
topic.
I
really
appreciate
you
coming
to
the
talk.
If
you
want
to
learn
more
about
linker
D,
there
is
a
booth
in
the
booth
area
and
yeah
I'll
be
around
off
the
talk.