►
Description
KubeCon: A Story of Two Service Discoveries: Bringing Legacy to Kubernetes - Mark Petrovic, PayPal/Xoom & Brandon Philips, CoreOS, Inc
"Many companies have a service discovery solution whether it is home grown, purchased, or something in-between. Kubernetes will have to inter-operate with those systems as it begins to move into existing environments.
This talk outlines the DNS and proxy based service discovery system that Xoom (a PayPal company) uses in all of their production infrastructure. And how a combination of containerization of the existing system, Kubernetes configuration changes, and architectural creativity gave the new Kubernetes-based platform the ability to bridge into existing systems."
About Mark Petrovic
Mark Petrovic is software engineer and architect at Xoom, A Paypal Service. Prior to joining Xoom, Mark was VP R&D for EarthLink, and Visiting Scientist at IBM Research Division. He now enjoys as much Go and Kubernetes as time allows.
About Brandon Philips
Brandon Philips is helping to build modern Linux server infrastructure at CoreOS as CTO. Prior to CoreOS, he worked at Rackspace hacking on cloud monitoring and was a Linux kernel developer at SUSE. As a graduate of Oregon State's Open Source Lab he is passionate about open source technologies.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
So
I
think
it
was
labeled
as
a
panel,
but
it's
not
a
panel.
It's
two
talks.
I'm
laying
foundation
marks
building
on
the
foundation,
all
right,
so
I'm,
Brandon,
Phillips,
I'm,
the
CTO
and
co-founder
of
core
OS
Kouros
builds
a
bunch
of
open
source
projects
and
also
enterprise
products
in
this
entire.
In
this
ecosystem,
tectonic
is
a
pure
upstream
kubernetes
distribution
of
kubernetes,
with
a
bunch
of
additional
utilities
packaged
together
that
make
it
possible
for
enterprises
to
adopt
kubernetes
and
quasar
container,
build
and
hosting
service.
A
So
what
I'm
gonna
be
doing
is
walking
through
a
few
examples
going
through
very
practical
service
discovery.
It's
gonna,
take
about
15
minutes
or
so
very
practical
services
discover
examples
on
top
of
kubernetes.
This
is
kind
of
a
one
on
one
101
level
foundation
setting
in
all
of
these
I'm
going
to
be
using
this
tool
called
mini
cube,
which
allows
you
to
build
or
bring
up
simple
single
node
at
CD
or
simple
single
node
kubernetes
clusters
on
your
laptop
there's,
a
QuickStart
guide
available
here.
A
If
you
search
for
github.com,
slash,
kubernetes,
slash,
mini
tube,
it
talks
through
just
how
to
bring
it
up
under
Windows,
Linux
and
OS
X,
and
then,
if
you're
interested
in
trying
out
the
rocket
container
engine,
there's
a
couple
of
flags
that
change
and
you
can
bring
up
under
rocket
once
you
get
everything
working.
Essentially,
it
should
look
something
like
this
coop
cuddle
get
nodes
and
you
should
have
a
single
mini,
tube
and
since
running
all
right.
So
that's
what
I'll
be
using
for
all
the
demonstrations.
A
A
So
kubernetes
has
this
really
powerful
concept
of
labels
that
you
can
apply
to
essentially
every
single
resource
and
object
inside
of
kubernetes
and
these
resources,
or
these
labels
are
concept
inside
of
the
kubernetes
api
server,
the
api
server.
You
can
think
of
it's
just
any
other
web
application.
That's
not
really
any
magic
or
anything
fancy.
It's
a
container
that
stores
all
this
data
and
persistent
data
store
called
Etsy
D.
A
So
the
idea
of
this
label
is
that
when
I
do
something
like
coop,
cuddle
and
I
create
a
new
replica
set
or
a
new
application
that
I
want
deployed
inside
of
kubernetes,
various
labels
will
be
applied
to
it
and
these
are
used
throughout
the
system,
not
just
for
service
discovery.
So
let's
say
that
I
deploy
an
application.
This
application
is
going
to
have
a
couple
of
labels
like
a
people's
web
and
environment
environment
equals
prod
I'll,
deploy
a
replica
set,
say
I
want
to
have
three
copies
of
that
application
running.
A
Perhaps
only
one
is
running,
so
the
the
cassette,
the
software
that
manages
is
that
replica
set
is
inside
of
a
component
in
kubernetes
called
the
controller
manager
that
controller
manager
essentially
is
doing
service
discovery
at
via
the
labels
inside
of
the
API
finding.
What
is
the
current
state
based
on
that
label?
Query
saying
all
right
well,
based
on
the
label
query:
the
current
state
of
the
system
is
one
I've
been
told
that
I
need
to
move
the
system
to
three
and
then
it
goes
through
and
launches
new
pods
on
site
on
top
of
the
cluster.
A
So
what
we'll
do
is
just
give
a
practical
example
using
the
command
line
tool
of
how
this
works.
So
the
first
thing
we're
gonna
do
is
use
coop,
cuttle,
run,
launch
an
engine,
X
server
and
then
we're
going
to
apply
a
label
to
that
deployment
which
will
apply
the
label
to
the
replica
set
which
will
apply
a
label
to
the
pod.
So
let's
go
ahead
and
look
through
there,
so
all
I'll
do
cout
cuddle,
run
engine,
X,
etc.
A
I've
already
done
this
because
it
takes
a
couple
of
seconds
to
download
the
container
and
one
conference
Wi-Fi
and
I
don't
want
to
bore
you
all.
So
the
the
application
is
already
running
now.
What
we
can
do
is
we
can
find
all
right.
Well,
we're
all
the
replica
sets
were
all
the
instances
of
this
application.
A
So
if
we
do
a
cube,
cuddle
get
replica
set
which
shouldn't
be
shortened,
is
our
s
and
then
do
a
query
where
we
say
give
me
all
the
replica
sets
the
match:
label
PAP
equals
index,
it
returns
one,
and
similarly,
we
can
do
the
same
thing
for
the
pod,
which
finds
the
actual
concrete
running
process.
Inside
of
the
cluster,
so
again
that
whole
chain
of
commands
is
like
we're
able
to
find
the
deployment.
The
deployment
is
the
thing
that
could
kind
of
run
created
for
us
to
tell
us
to
run
this
engine
X
service.
A
The
deployment
then
creates
replica
sets.
This
is
how
rolling
updates
happen
in
deployments,
so
the
replica
set
says
alright.
I
want
to
have
one
copy
of
this
running
and
then
finally,
we
go
down,
and
we
see
that
the
pod
that
same
label
at
people's
engine
X-
and
we
find
the
concrete
running
process
that
concrete
running
container
any
questions
on
that.
So
far,
good.
A
So
you
can
start
to
imagine
this
doing
other
things
you
can.
The
concept
of
a
service
is
that
concrete
load
balancer
inside
of
the
system?
So
if
I
want
to
put
this
engine
X
service
behind
a
load,
balancer
I
say
I
want
a
IP
address.
Allocated
I
want
this
IP
address
to
be
the
central
incoming
load,
balancer
that'll
may
be
put
into
DNS
or
whatever,
for
my
service
that
that
load
balancer
will
then
use
label
queries
to
find
all
the
pods
that
are
running
that
can
service
that
service.
A
Okay,
and
the
way
that
we
do
this
with
mini
cube
is
because
mini
coop
doesn't
have
a
load
balancer,
we
don't
have
an
f5.
We
don't
have
anything
fancy
is
that
we
just
directly
expose
the
service
via
thing
called
a
node
port,
so
that
happened
really
fast.
But
what
we're
telling
it
is
all
right.
I
have
internet
running
I
want
you
to
expose
that
nginx
deployment
I
want
you
to
expose
port
80
I
want
to
give
it
a
node
port.
A
Oh
yeah
mul.
What
we'll
see
here
is
that
there's
a
thing
called
a
node
port
that's
been
assigned
along
with
the
port.
That
was
that
we
want
to
target
for
this,
the
port
that
we
want
to
target
for
this
load
balancer,
and
what
this
node
port
does.
Is
it
says
on
every
single
node
in
our
node
in
our
cluster
and
if
we
remember
every
single
node
in
our
cluster,
is
this
single
node,
the
single
mini
cube,
node
every
single
node
in
the
cluster?
A
We
want
to
expose
a
port
that
is
going
to
route
to
this
application
and
that
port
in
particular
is
going
to
be
thirty
thousand
nine
hundred
and
eighty
five
and
it
just
randomly
assigns
it.
So,
if
you
remember
back
before
the
world
of
kubernetes
is
how
we
all
did
sort
of
internal
service
discovery
right.
We
had
like
single
IP
addresses
for
servers,
the
single
servers
when
we
need
to
expose
a
port.
A
We
go
into
our
next
Excel
spreadsheet,
we'd
record,
hey
I'm,
reserving
port
4098
for
the
reddest
cash
or
whatever
it
is,
and
then
we
would.
We
tell
network
operations
all
right,
I
need
to
use
this
port
for
that,
and
all
the
application
developers
would
hard
code
that
into
their
configuration.
This
is
allowing
us
to
go
back
to
that
port
mapping
world
where
we
may
not
have
sophisticated
load,
balancers
or
networking.
So
after
we
get
that
placed
mini
to
the
tool
that
we're
using
to
run.
This
has
a
convenience
application
called
mini
cube
service.
A
A
lot
mini
cube
service
will
do.
Is
it
will
figure
out
what
operating
system
I'm
on
on
OS,
X
or
Windows
or
Linux
it'll,
run
the
correct
command
to
launch
my
browser
and
actually
open
up
that
service
on
that
port?
So
what
happens?
Is
that
you'll
see
this?
Is
the
IP
address
of
my
mini
cube,
one
seven,
two,
sixteen
something
something,
and
then
it
opened
up
port,
three.
Thirty
thousand
nine
hundred
eighty
five
right
so
immediately
we're
able
to
kind
of
simulate
the
idea
of
a
load
balancer
here
on
our
laptop.
A
Yeah
great
question,
so
the
question
was:
if
node
port
always
load
balances
like
the
local
machine
or
if
it
load
balances
across
the
cluster
and
so
what's
actually
happening
behind
the
scenes.
Is
that
there's
a
thing
called
the
coop
proxy
and
every
single
machine?
The
cluster
is
going
to
expose
that
node
port
once
you
define
a
node
port.
So
if
you
have
five
workers,
all
five
workers
will
begin
listening
on
thirty
thousand
nine
hundred
eighty
five
and
then
you'll.
A
When
they
hit
that
machine,
it
will
essentially
do
round-robin
eight
through
the
iptables
system
inside
of
the
kernel
to
hit
random
hosts
and
so
generally,
what
you'll
want
to
do
is
you'll
want
to
like
if
you
have
an
e
lb
or
something
or
you
have
an
f5
you'll
want
to
have
that
f5
load
balanced
across
your
entire
cluster
and
then
essentially
you
have
a
second
like
application
tier
that
load
balances
through
kubernetes.
This
is
all
like
just
one
way
of
doing
load
balancing.
Obviously
there
are
more
kubernetes
native
ways,
but
this
is
super
convenient.
Particularly.
A
We
see
a
lot
of
people
as
they're
getting
early
adopting
early,
adopting
kubernetes
inside
of
their
environment
or
inside
of
their
companies.
They'll
use
node
port
as
a
way
of
not
having
to
engage
the
network
operations
team
right
from
the
beginning.
Great
question.
Any
other
questions:
ok,
ok!
So
this
is
all
about
V
API,
the
kubernetes
api,
we're
using
labels,
we're
using
services
we're
using
deployments,
and
these
are
fundamental
parts
of
the
kubernetes
api.
Now
above
the
api,
there
are
a
lot
of
conveniences
that
kubernetes
provides.
A
The
dns
addon
is
one
of
those
conveniences,
so
dns
is
not
built
into
kubernetes
is
an
extension
point.
It
is
something
that
is
a
convenience
offered
by
kubernetes,
but
something
that's
pluggable
and
that's
what
Mark
is
going
to
be
talking
about
is
plugging
into
dns,
so
the
dns
addon.
What
it
does
is
on
every
single
machine.
In
your
system.
A
You
provide
this
IP
address
via
the
cluster
dns
flag
of
the
couplet,
and
what
happens
is
that
every
single
pod
launch
on
that
machine
or
on
that
machine
with
that
flag,
we'll
get
injected
into
its
@c
resolve
cough
the
IP
address.
That's
provided,
and
this
IP
address
is
a
IP.
That's
exists
that
exists
on
top
of
the
kubernetes
network,
so
it's
going
to
be
a
ten
dot
network
or
whatever
you
defined
as
your
kubernetes
service
network.
So
it
gets
a
little
bit
of
meta
and
recursive
here,
so
we're
telling
every
single
pod
on
the
network.
A
So,
if
we,
if
we
look
into
kubernetes
on
this
mini
tube
cluster,
we'll
run
a
query
that
says
describe
the
describe
the
service
cube,
DNS,
that's
running
inside
a
cube
system
namespace,
and
what
we'll
see
is
that
there
is
a
service,
that's
running
at
10.0
at
0.10
and
it's
exposing
port
53,
both
UDP
and
TCP,
and
this
again
this
IP
address
is
injected
into
the
at
sea,
resolve
comp
of
every
pod
running
in
the
cluster.
So
this
is
where
they
go
to
to
get
their
dns
resolution
service.
A
So
the
basics
of
this
service.
What
happens
is
that
there's
this
piece
of
software
called
the
cube
DNS
pod?
It
runs
as
a
deployment
in
your
kubernetes
cluster,
it
synchronizes
with
the
API
server.
So
it's
asking
the
API
server
give
me
all
the
services
that
have
been
defined.
Give
me
all
the
pods
that
have
been
defined
and
it's
creating
essentially
a
cache
of
all
those
things
that
it
can
resolve
as
DNS
requests.
A
If
you
want
to
understand
the
system,
so
I'll
post
these
slides
later,
but
if
he
is
Google
like
kubernetes
network
troubleshooting,
core
OS,
we
have
this
one-liner
here
that
will
port
forward
the
kubernetes
dns
server
to
your
local
machine.
So
I'll
go
ahead
and
run
that,
and
so
what
this
will
do
is
kubernetes
is
going
to
run
its
DNS
server
port
for
its
DNS
server
to
port
5300.
On
my
laptop.
A
Alright,
by
doing
that,
what
I
can
do
is
I
can
start
to
play
around
with
the
kubernetes
DNS
server,
so
I
can
run,
dig
and
I
can
see.
Oops
I
can
run
dig
and
I
can
see.
Ask
a
question
like
the
cluster
about
local
sub
domain,
which
is
the
top-level
domain
of
the
cluster,
and
it
gives
me
back
that
SOA
record
with
the
name
server.
The
next
thing
I
could
do
is
I
can
start
to
resolve
services,
so
I
have
a
service
inside
of
my
cluster.
A
So
if
I
do
at
CD
cluster
Asadero
zero
zero
one
dot
default
service
that
cluster
local
I'll
see
here
that
I
get
the
exact
same
service,
IP
back
that
is
listed
here
inside
of
the
service
list
and
similarly
you
can
do
serve
records
as
well.
It
will,
if
you
have
a
service,
that,
like
the
the
DNS
service,
that
expose
a
TCP
and
UDP
port,
you
can
dig
for
service
records
and
find
out
what
port
number
it
is
based
on.
The
protocol.
A
There's
a
bunch
of
other
features
that
the
DNS
Adhan
has
one
of
the
most
convenient
is
that
it
will
automatically
put
a
search
prefix
for
the
DMS,
so
for
an
individual
pod.
But
let's
assume
that
you
want
a
hard
code
inside
of
your
application.
I
want
to
be
able
to
reach
Postgres,
you
can
just
say
Postgres.
A
A
Sure
so
the
question
was:
what
do
you
get
back
for
SRB
queries,
so
on
a
service?
What
you'll
get
back
on
a
SRV
queries?
You
can
do
things
like
underscore
dns
underscore
tcp
dot
service
will
give
you
back
the
port
and
then
the
rest
of
the
SRV
record
there
are
I'll
have
to
look
again,
I
believe
that
it
will
often
look
I
think
on
some
of
them.
If
you
have
multiple
IPS
it'll
return,
all
the
IPS
as
well
for
the
service
they'll
have
to
check
on
that.
It's
true.
A
A
A
A
A
B
Okay,
Thank
You
Brandon.
My
name
is
Mark
Petrovich
I'm,
an
engineer
at
PayPal,
more
specifically
I'm
an
engineer
at
a
company
called
zoom,
which
is
a
PayPal
company.
We
were
acquired
late
last
year
and
the
zoom
is
a
money
service
whereby
we
help
folks
living
in
the
United
States,
send
money
back
home
every
payday
to
their
families
back
home
in
any
number
of
any
one
country
among
dozens
of
countries
that
we
service.
B
B
Briefly,
on
the
matter
of
what
challenges
we've
faced
along
the
way,
some
specific
to
integrating
our
legacy
service
discovery,
some
not,
but
if
you've
started
down
the
path
of
container
izing
your
applications,
you
probably
have
noticed
that
it
touches
everything
your
builds
are
affected.
Your
CI
pipeline
is
affected.
How
you
deploy
is
affected.
How
you
talk
about
deploying
is
affected.
B
If
you
haven't
started
container
izing
your
applications,
then
you
can
expect
to
rediscover
how
container
izing
will
touch
everything
so,
under
that
umbrella
of
a
lot
of
my
tooling
is
going
to
change,
and
my
language
is
going
to
change
language
about
how
it
is
how
I
talk
about
deployments
will
change.
There
are
a
couple
of
interesting
things
that
we
discovered
along
the
way
unrelated
to
service
discovery,
but
I'll
mention
them
anyway,
because
I
think
it's
worth
it.
B
For
example,
if
you
have
logging,
dashboards
or
metrics
collection,
dashboards
and
the
folks
who
read
those
dashboards,
if
they're
accustomed,
based
upon
how
you
deploy
to
seeing
they're
accustomed
to
seeing
a
particular
application,
always
logging
from
a
particular
host
or
a
particular
IP
address,
then
you
should
visit
with
them
now,
because
container
IPs
and
container
host
names
are
not
persistent
things.
Every
time
you
restart
a
container,
it
gets
a
new
IP
address.
It
gets
a
new
host
name.
B
So
if
your
container,
if
your
logging
framework
or
your
metrics
collection
of
framework
has
customers
internal
customers
who
are
used
to
seeing
artifacts
of
your
deployment
whereby
applications
always
log
from
the
same
host,
go
visit
with
those
folks
and
tell
them
that
with
containers,
an
application
will
retain
its
identity,
but
it
will
be
logging
from
a
different
host
every
single
time.
The
different
IP
address
every
time
that
application
is
deployed,
we're
also
facing
the
challenge
of
crafting
developer
environments
for
our
developers.
B
A
hundred
some-odd
developers
currently
developers
at
zoom
will
deploy
their
dependencies
onto
their
laptops
and
then
begin
to
develop
against
those
locally
running
dependencies.
What
we
want
to
do
is
to
give
developers
their
own
kubernetes
cluster,
a
small
cluster,
so
that
they
can
begin
to
deploy
in
the
same
way
that
their
counterparts
in
QA
and
production
will
deploy
applications
so
that
we
can
all
talk
about
deployment
in
the
same
way,
not
in
terms
of
puppet
configurations,
puppet
modules,
ansible
scripts,
but
instead
speak
in
terms
of
kubernetes.
B
First
principles
like
replication
controllers,
deployments,
namespaces
secrets
things
like
that,
so
we
want
to
give
developers
their
own
clusters
and
we're
working
on
a
couple
of
ways
to
do
that.
If
you
are
currently
doing
that
or
you
have
solved
this
problem,
I
would
like
to
talk
to
you
in
the
hallway.
Maybe
we
can
share
notes,
so
developers
are
going
to
have
to
become
familiar
with
how
coop
control
works,
which
is
the
command
line
tool
with
which
you
control
a
cluster
as
well.
B
B
Finally,
the
challenge
we
face
is
integrating
our
legacy
service
discovery
design
into
kubernetes,
so
that
some
applications
can
run
inside
the
cluster
on
day.
One
and
some
will
be
continued
to
run
outside
the
cluster,
we're
going
to
production
with
kubernetes
sometime
next
year,
we're
preparing
with
our
plan
to
do
so.
So
we
want
to
start
slowly,
deploy
one
or
two
applications
into
kubernetes
and
have
everything
else
continue
to
run
outside.
B
But
these
applications
can't
know
that
they're
running
inside
the
cluster,
nor
can
applications
that
need
them,
know
that
they're
running
in
the
cluster
that
would
be
counterproductive.
So
I'll
share
with
you
how
a
couple
of
designs
that
we
came
up
with
that
lets
us
move
applications
into
the
cluster
seamlessly.
B
So
we've
been
in
business
for
10
some
odd
years
and
we
have
legacy
that
we're
dealing
with.
We
have
traditionally
run
a
monolith
that
manifested
all
of
our
business
logic:
new
functionality,
we're
porting
we're
implementing
in
micro
services
immediately
and
we're
setting
aside
time
to
vacate
our
monolith
of
its
functionality.
This
so
that
someday
we'll
have
just
a
set
of
cooperating
micro
services.
We
are
container
izing
everything.
B
We
have
started
down
this
path
a
good
year
ago,
and
if
you
are
just
starting
this
or
you're
contemplate
starting
it,
you
will
discover
that
it
takes
you,
you
and
your
organization,
many
many
months
to
surface
how
you
work,
how
what
is
your
docker
file?
What
should
your
docker
file
contain,
or
your
AC
build
file?
B
What
is
my
container
entry
point?
Look
like
for
all
of
these
micro
services,
and
hopefully,
you'll
come
up
with
a
model
where
all
of
my
micro
services
have
pretty
much
the
same
dockerfile,
hopefully
the
same
dockerfile,
not
the
same
instance,
but
they
read
the
same
and
hopefully
you'll
find
that
your
applications
all
can
use
the
exact
same
entry
point
which
in
our
case
is
a
shell
script.
We
found
that
shell
scripts
are
universally
sort
of
the
the
entry
point
that
just
falls
out
of
most
designs.
So
your
entry
point
will
read
a
certain
way.
B
Your
docker
file
will
read
a
certain
way
and,
if
you're
an
organization
of
any
size,
it
will
take
you
many
months
to
surface
what
those
patterns
are
so
that
you
can
move
on
to
the
next
problem.
So
we
also
need
an
orchestration
framework.
We've
decided
to
use
kubernetes
some
many
many
many
many
months
ago
and
I'll
tell
you
how
we've
integrated
our
service
discovery
into
this
thing.
B
So
we
currently
run
our
micro
services
on
bare
metal
or
VMs,
and
we
don't
know
what
IP
address
a
micro
service
will
take
on
when
it
begins
to
run
by
design.
We
don't
know
where
clients
future
clients
of
that
micro
service.
Don't
know
the
IP
address
of
that
micro
service
running
right
now
in
production
intentionally.
We
want
to
develop
a
way
where
clients
that
need
that
micro
service
can
discover
its
address
just
in
time,
so
we
need
a
service
discovery
framework.
B
B
They
post
that
to
service
proxy
and
service
proxy
remembers
that
binding,
so
that
when
clients
come
calling
for
that
name,
service
proxy
knows
where
to
find
it.
So
the
big
blue
box
is
a
physical
host.
This
is
how
we
currently
run
big
blue
blocks,
a
big
blue
box
as
a
physical
host
on
every
host
as
a
service
proxy
instance,
which
is
an
HTTP
proxy,
so
on
this
particular
host,
there's
an
application,
foo
and
bar
running.
B
So
when
bar
comes
up,
it
posts
a
request
to
service
dot,
one
dot,
zoom
API,
we've
hijacked
the
top-level
domains,
M
API,
to
return
one
27001
in
response
to
any
question
asked
on
it,
so
service
dot.
One
one
is
the
version
number
service,
not
one
gotta
zoom
API
resolves
to
127
zero
zero
one
which
finds
bar
posting
to
the
local
host
service
proxy,
with
an
assertion
that
my
name
is
bar
and
my
IP
address
is
such-and-such.
B
Please
record
me
in
the
system
of
record
as
being
there
when
someone
comes
calling
for
me,
so
someone
is
foo,
foo
comes
up
and
it
needs
to
consume
bar.
It
forms
the
URL
with
bar
dot
one
dot.
Zoom
API
is
the
host
name
part.
That
name
also
resolves
to
one
27001
service
proxy
running
on
the
node
gets
a
hold
of
the
request,
looks
at
the
host
header
and
says
this
is
for
bar
I
know
where
bar
is
bar
registered
a
moment
ago.
I
will
proxy
that
request
and
to
end
on
behalf
of
the
caller.
B
Foo
food
doesn't
know
the
service
proxy
is
there.
All
it
knows
is
that
it
got
an
answer
from
something
going
by
the
name
bar
dot.
One
dot,
zoom,
API
barré
doesn't
know
service
proxy.
Is
there
either
the
tricks
of
DNS
service
proxy
is
insinuated
into
the
call
path
for
both
of
these
clients
for
bar
during
registration
for
foo
to
consume
bar.
B
So
without
changing
how
my
DNS
returns,
one
27001
to
every
question,
I
can
move
a
service
proxy
into
the
pod
and
when
foo
or
bar
wants
to
interact
with
any
name
on
zoom'
api,
its
returned
again,
the
answer:
127
0,
0,
1
and
lo
and
behold-
there's
a
service
proxy
in
that
pod.
With
that
application
that
can
be
addressed
at
localhost
and
the
well-known
port
of
service
proxy.
So
this
does
work.
This
works
very
well.
It
works
exactly
as
it
was
intended
to
work
with
respect
to
the
localhost
trick
inside
of
a
kubernetes
pod.
B
We
could
stop
here.
If
we
had
gobs
of
memory,
we
could
stop
here,
but
we
always
want
more.
We
don't
have
gobs
of
memory.
We
want
to
see
if
we
can
advance
this
model
to
improve
it.
The
reason
that
you
don't
want
this
model
long-term
is
because,
if
you
have
many
pods
or
many
applications,
every
application
will
have
many
replicas.
B
You
end
up
having
that
number
of
service
proxy
instances
on
your
network
and
for
us
service
proxy
is
a
java
application.
So
a
java
application
that
has
been
warmed
up
in
doing
production
work
will
typically
want
hundreds
of
megabytes
of
memory
to
run.
So
you
don't
want
to
have
to
consume
hundreds
of
megabytes
per
pod
on
your
network.
B
So
there
are
a
number
of
ways
forward
from
here
to
improve
this
model.
This
is
the
model
that
we've
come
up
with,
and
it
reads
like
this:
it
depends
on
three
things.
The
first
trick
is
to
pull
service
proxy
out
of
the
pod
and
run
it
as
a
daemon
set
on
each
node
daemon
set
with
a
capital
D
as
a
kubernetes
thing,
whereby
you
can
get
a
pod
to
run
on
on
a
per
node
basis.
B
B
The
second
piece
of
the
solution
is
to
run
DNS
mask
as
a
daemon
set.
Also
in
the
host
network.
Dennis
mask,
if
you
don't
know,
is
a
lightweight
name
server.
We
program
DNS
mask
in
its
entry
point,
which
I'll
show
you
in
a
second
to
return:
the
node
IP
as
the
answer
to
any
question
on
the
domain:
zoom
API,
so
I'm,
hi,
I've
hijacked
the
top-level
domain
again
in
a
slightly
different
way,
using
DNS
mask,
as
a
mildly,
programmable
web
server
or
DNS
server.
B
The
third
leg
is
to
configure
your
couplet
on
every
node
to
deposit
the
nodes
own
IP
address
as
a
name
server
inside
of
each
containers
are
resolved
on
file.
So
now,
when
the
container
goes
to
look
up
any
name,
it's
directed
to
the
node
IP,
which
is
directed,
which
turns
out
to
be
DNS,
mask
running
on
that
node.
B
In
the
hosts
network,
namespace
returning
the
node
IP
itself
in
response
to
any
question
on
zoom'
API,
which
you
want
to
interact
with
a
host
in
that
domain,
sends
the
request
to
service
proxy
running
on
the
node
in
the
hosts
network
namespace.
So
this
is
the
functional
equivalent
of
locating
a
service
proxy
inside
of
every
pot,
and
this
works,
and
the
advantage
of
this
model
over
the
previous
model
is
that
now
I
have
one
service
proxy
instance
running
on
my
node,
rather
than
one
in
each
pod.
B
B
B
If
I
make
a
request
for
any
name
inside
of
one
of
those
pods,
the
name
server
for
that
pod
is
DNS
mask
running
on
the
node
I'll
show
you
the
entry
point
for
DNS
mask
shortly,
to
show
you
how
it
discriminates
between
questions
on
zoom',
API
and
all
other
questions,
but
it
gets
ahold
of
the
name.
Server
query
request,
so
that
it
can
do
the
right
thing
with
the
request,
based
upon
the
domain
of
the
request.
B
B
B
In
order
to
get
the
note,
IP
address
here
for
DNS
mask
if
you're
familiar
with
DNS
mask
and
how
its
configured
this
address
flag
says
for
any
requests
on
the
domain.
Zoom
API
return,
the
note
IP
I
acquire
the
note
IP
just
above
this
line
by
parsing
the
output
of
if
config
on
each
0.
That's
how
I
come
up
with
my
note,
IP
and
because
DNS
mask
is
running
in
the
host
namespace.
This
is
the
same
IP.
The
service
proxy
is
running
on
because
it
is
running
in
the
hosts
namespace.
B
B
There
are
a
lot
of
different
ways
that
you
can
arrange
for
pods
to
be
networked
amongst
each
other,
but
we're
using
the
Calico
networks
network
tooling
to
make
our
pod
ip's
fully
routable
on
the
network,
so
that,
when
I
address
any
one
of
my
micro
services
by
IP
address
that
routing
doesn't
have
to
traverse
encapsulation
or
NAT
or
special
ingress
controllers
to
make
its
way
to
the
destination.
It's
fully
rounded
pod
IPS
are
fully
routed
on
the
network.
B
So
that's
how
we
have
ported
our
legacy
service
discovery
into
kubernetes
within
none
of
the
technology
that
brandon
was
originally
describing.
That's
for
a
different
use
case.
We'll
probably
use
some
of
that
stuff,
but
not
for
this
purpose.
So
thank
you
very
kindly.
I
think
we
have
time
for
a
couple
questions.
B
B
B
The
reason
we
don't
use
any
of
the
kubernetes
native
service
discovery
is
because
we
didn't
want
to
introduce
that
much
change
to
our
existing
design.
The
way
our
services
currently
find
each
other
we're
suffering
enough
change
as
it
is
porting
our
applications
into
kubernetes
and
changing
our
organization's
tooling.
So
we
have
settled
on
let's
port,
the
existing
service
discovery
mechanism
into
kubernetes
and
don't
touch
the
other
kubernetes
magic
service
discovery
at
all.
B
B
B
Three:
the
third
piece
was
the
couplet,
which
has
been
told
to
deposit
the
nodes
IP
address
into
the
resolved
comp
of
every
container.
That's
the
last
piece
of
glue
to
direct
containers
to
use
DNS
mask
on
the
node
for
their
name
server.
So
thank
you
very
kindly.
I
love
kubernetes.
The
community
is
super
welcoming
imminently,
competent
and
I
learned
something
every
day,
I
deal
with
these
folks.
Thank
you.
So
much.