►
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).
gRPC Deep Dive – Sree Kuchibotla, Google (Intermediate Skill Level)
About Jayant Jayant is Director of Engineering at Google working in Google Cloud Organization. He has been at Google for last 10 years. He manages teams that work in areas of Networking, distributed systems and APIs. He has led and managed multiple Open Source Projects such as gRPC and Protocol Buffers. Earlier, Jayant led and managed Google Earth Engineering teams. Prior to Google, Jayant worked at NVIDIA. He led and managed compilers and shading languages teams at NVIDIA such as CUDA Compiler, Cg Compiler. He also has extensive background in GPUs, 3D graphics and General Purpose Programming on GPUs. Jayant has led several developer platform projects in his career such as gRPC, Protocol Buffers, CUDA, Cg. As a part of these projects, he has presented at several meetups, conferences and universities.
About Sree
Sree Kuchibhotla is a Software Engineer on the gRPC team at Google. He works on gRPC C-Core and C++ stack and is currently focussing on gRPC performance and lower level components in the stack.
Before joining Google, Sree worked at Twitter and Microsoft on Distributed storage systems, Databases and Operating Systems
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
All
right
good
afternoon
folks,
my
name
is
Sri
coochie,
Butler
and
I'm,
a
developer
in
the
gr
PC
team.
So
today,
I'm
gonna
talk
about
the
C
core
architecture
and
also
the
life
of
gnar
PC.
So
I
must
warn
you
that
I
have
a
lot
of
slides
I'll,
try
to
go
through
them
as
quickly
as
possible
and
try
to
leave
some
time
in
the
end
for
Q&A
all
right.
So
who
is
the
target
audience
for
this
talk?
So
this
talk
is
primarily
aimed
at
like
power
users
of
their
PC.
A
You
know
someone
who
uses
their
PC
in
their
application,
who
may
have
had
to
debug
any
issues
who
may
have
had
to
look
at
some
of
the
GRP
see
traces
in
the
process.
Hopefully
this
talk
will
help.
You
like
understand
at
least
the
big
picture
right.
Anyone
is
a
contributing
or
thinking
of
contributing
they
might
find
the
talk
useful
or
anyone
just
curious
like
about
er
PC
architecture.
Just
just
understand.
What's
going
on
so
all
right,
so
gr
PC
is
implemented,
I
mean
gr.
A
Pc
has
idiomatic
API
s--
in
over
nine
languages,
actually
excluded
a
few,
but
over
nine
languages,
and
it
has
three:
it
is
natively
implemented
in
three
languages.
Java
go
and
see
all
the
other
languages
are
wrapped
around
the
C
implementation,
so
my
talk
is
primarily
going
to
focus
on
the
C
core
architecture,
I'm
just
going
to
refer
it
to
a
seek,
or
from
this
point
forward
so
yeah
before
diving
into
the
details.
A
It
helps
to
sort
of
set
some
context
on.
Most
of
you
may
already
know
this,
but
what
are
some
of
the
key
features-
and
this
is
primarily
these
are
some.
What
some
of
these
are
core
specific,
so
its
payload
diagnostic
G,
RPC
code
does
not
know
anything
about
the
payload
doesn't
know
about
the
serialization
format,
whether
it's
wrote
above
flat
buff.
It
doesn't
know
that
that
features
are
implemented
in
the
in
the
rap
language
layer,
and
then
it
has
extensible
architecture
that
the
actual
functionality,
of
course,
stack
is
very
lean.
A
A
A
A
A
A
They
typically
use
the
wrapped
language
api's
and
the
consumers
of
this
of
the
wrapped
language
ApS,
so
I
just
wanted
to
make
it
clear
and
the
second
is
the
transport
transport
implements
the
wire
protocol
and
by
default
it's
HTTP,
but
you
can
use
other
transports
as
well
and
at
the
bottom
layer
you
have
the
IO
manager
that
provides
low
level,
networking
I/o
functions,
network
utilities,
timers
and
a
bunch
of
primitives
that
are
sort
of
platform.
Specific
most
of
the
company
see
I
tend
to
be
platform-specific.
We
support
multiple
platforms,
there's
windows-based
implementation.
A
A
A
So
so,
let's
start
with
the
channel.
A
channel
is
a
communication
pipe
between
a
client
and
a
target
and
I'm
deliberately
not
using
the
term
connection
to
describe
a
channel
and
we'll
get
into
it
later
so
channel
is
a
communication
pipe
between
a
client
and
a
target.
A
call
is
bound
to
a
channel
the
relation
between
a
call
and
the
channel
is
many-to-one.
So
that
means
we
can
have
multiple
calls
on
the
same
channel,
and
then
you
have
a
bunch
of
operations
that
could
be
performed
on
a
call.
A
A
We'll
look
at
that
a
little
and
these
operations
are
done
in
batches.
So
batch
is
an
important
concept.
A
batch
can
contain
operations.
Yeah
batch
can
contain
one
or
more
operations
and
then
finally,
the
most
important
concept
here
is
this
notion
of
a
completion
queue?
So
when
you
start
a
batch
of
operations,
you
wait.
The
completion
of
the
batch
is
notified
on
what
we
call
a
completion
queue
so
typically
clients
since
I
was
interacting
with
interacting
with
core.
You
know
the
semantics
are
something
like
they
start
a
batch
of
operations
and
then
they
wait.
A
A
So
those
are
the
ops
that
we
support
I'm,
not
going
to
read
them
out
loud,
but
the
interesting
thing
there
is
just
wanted
to
highlight
that
you
know
the
message:
the
messages
that
are
exchanged
between
client
and
server
pretty
similar.
The
client
always
starts
by
sending
the
initial
metadata,
followed
by
zero
or
more
messages
followed
by
failing
metadata
and
server.
Does
the
same
in
case
of
unary
or
pcs?
There's
exactly
one
message:
that's
sent
from
the
client,
which
is
essentially
the
parameters.
A
Initial
metadata
from
client
to
server
in
a
unit
in
all
our
pcs
essentially
contains
stuff
like
the
method,
name,
encoding
type
or
any
other
custom
metadata.
We
expose
api's
for
applications
to
send
their
own
custom.
Metadata
tailing
metadata
from
client
to
server
is
just
end
of
stream
indicator
server
to
client
initial
metadata
against
has
the
same
encoding
type.
A
If
it's
a
if
it's
a
unary
RPC,
the
message
contains
the
return
value
from
the
server
trailing
metadata
contains
the
status,
whether
the
RPC
the
error
code.
If
that
PC
failed
all
right,
so
I
think
we
are
ready
to
dive
deep
into
life
up
an
RPC.
So
it
starts
with
a
channel
creation
and
I'm
dividing
this
into
two
phases.
So,
first
part
I
focus
on
the
channel
creation,
workflow
and
the
second
part
I'll
focus
on
call
creation
and
the
life
of
a
caller
on
over
this
channel.
A
So
again,
so
that's
the
client,
as
I
mentioned
before,
completion
queue
is
a
very
important
construct.
So
the
client
starts
by
creating
a
completion
to
you
and
then
a
thread
waits
on
the
completion
view
for
events
as
much
as
it
can
right.
We
always
that's.
That's
a
contract.
We
expect
from
the
users
of
the
consumers
of
gr
PC
core,
so
the
client
rate,
the
completion
queue
client
then
calls
an
API
to
create
a
channel
and
passes
the
server's
URI
and
that
creates
sort
of
a
filter
stack
specific
to
that
specific
to
that
channel.
A
That
is
in
creation.
So
there
are
a
bunch
of
filters,
but
the
interesting
filter
there
is
the
client
channel
filter.
That's
the
one
that
sort
of
orchestrates
the
service,
discovery
and
load
balancing
part.
So
the
client
channel
then
calls
the
resolver
resolver
is
resolver
is
pluggable.
There
are
multiple
resolves
that
we
support
and
then
the
resolve
returns
your
list
of
addresses
that
the
URI
resolves
to.
So
you
are
a
solute
server.
You
are
I,
might
actually
resolve
to
multiple
backends,
so
I
might
I
might
be
referring
to
the
term
backends
and
addresses
interchangeably.
A
But
essentially,
when
I
said
back-end,
it's
one
of
the
bases
that
there
is
all
were
resolved
so,
okay.
So
the
resolver
is
returned.
A
bunch
of
addresses
and
I
listed,
the
the
resolvers
that
we,
that
are,
that
the
GPC
open
source
version
has
all
right
and
then
the
client
channel
filter
then
creates
an
LP
policy.
That's
associated
with
that
channel
and
keep
in
mind
that
this
is
this
is
a
client,
Channel
load,
balancing
I
think
we
had
a
talk
yesterday
by
yarn
that
talked
about
load
balancing,
and
so
this
is
the
client
channel
load
balancing.
A
At
this
point,
I
should
mention
that
this
process
is
typically
done
in
a
lazy
fashion
like
when
you
create
a
channel.
It
doesn't
always
immediately
do
that.
We
wait
until
the
very
first
call
is
created
on
the
channel,
but
for
illustration
purposes:
I'm,
just
showing
it
here
so
anyway.
So
we
declined
issues
connect
to
all
the
back
ends
and
just
let's
look
at
one
of
the
one
of
the
back
ends.
What
happens
when
one
of
the
backups,
so
the
server
our
manager
receives
receives?
The
connection
accepts
the
connection
so
that
acts
back
to
the
client.
A
It
also
notifies
the
surface
that
a
there
is
an
incoming
incoming
channel.
So
server
creates
a
channel
object,
thereby
initializes
the
filter
stack
on
the
server
it's
pretty
similar
to
the
client
side,
except
if
you
filters
that
our
client
specific,
like
line
channel
and
at
this
point
the
server
is
done,
server
has
done
it's
part
of
general
creation.
Now
the
client
gets
like
server
has
sent
the
accept
my
face.
So
line
gets
the
act.
A
Clients,
IO
manager
tells
the
channel
filter
that
yeah
one
of
your
backends
accepted
the
client
channel
filter
then
creates
what
we
call
a
sub
channel.
So
this
is
something
that
I
didn't
introduce
before
so
so
creates
what
we
call
a
sub
channel
to
the
particular
back-end,
and
each
sub
channel
comes
with
its
own
set
of
stack
SEC
of
filters,
and
that's
it
once
it
creates
a
sub
channel.
A
Now
you
have
a
sub
channel
that
is
established
in
the
meantime,
the
other
beckons
also
might
have
responded,
so
that
results
in
sub
channels
being
created
for
the
other
backends
and
in
the
end
you
end
up
with
what
I
just
showed
here.
So
you
have
a
bunch
of
sub
channels
each
connecting
to
one
of
the
backends,
and
so
the
one
thing
I
wanted
to
highlight
is
so
really
all
these
sub
channels
are
part
of
a
channel.
A
All
right
now,
let's
move
on
to
a
life
of
a
call
just
to
keep
like
just
to
keep
things
simple.
I
will
talk
about
unit
unity
calls
here,
but
streaming
calls
should
be
very
similar
like
it's
really.
The
difference
is
just
based
on
the
number
of
messages
being
transmitted
back
from
client
back
and
forth
between
client
and
server.
A
A
Now
client
applications
call
it
API
to
create
a
call.
I
saw
that
we
had
some
in
memory
for
a
call
object.
Keep
in
mind
that
at
this
point
this
call
is
not
bound
to
any
of
the
sub
channels.
We
know
that
the
call
is
bound
to
a
channel.
We
are
creating
a
call
over
the
channel,
but
it's
not
bound
to
a
sub
channel,
so
that
happens
only
when
some
operations
are
started
on
the
call
right.
A
A
So
since
it's
a
first
batch
depending
on
the
LP
lb
policy
attached
to
the
channel,
one
of
the
channels
is
picked.
So
when
I
previously
mentioned
about
lb
policy,
I
forgot
to
mention
what
kind
of
plain
side
lb
policies
we
support.
So
we
support
3G
RPC
supports
3
lb
policies.
Well,
so
one
is
pick:
first,
that's
the
default,
so
we
just
picks
the
first
sub
Channel
other
one
is
round-robin,
so
it
just
keeps
round-robin
in
over
the
sub
channels
and
the
third
one
is
what's
called
gr.
A
Pcl
B,
which
is
the
Lucas
ID
load
balancing,
so
I
have
linked
in
the
slide
that
talks
about
a
load
balancing
so
anyway.
So
let's
assume
this
is
the
sub
channel.
That
was
pick
whatever
might
be.
There'll
be
policy,
but
this
is
the
subchannel
that
respect
now
from
this
point
forward.
Let's
sort
of
simplify
diagrams
so
I'm
only
including
the
the
components
that
are
important.
So
we
have
a
call,
there's
a
subchannel,
client
and
server
and
completion
queues
on
either
side.
A
So
yeah,
where
we
left
off
as
client
started
a
batch
of
operations,
and
so
since
it's
a
unary
call
klein
just
starts
a
batch
with
all
the
possible
ops
right.
Why
did
it
start
a
batch
with
all
the
ops?
Why
not
split
send
and
receive
into
two
different
batches?
It
could,
but
there's
really
no
need,
as
we
will
see,
because
it's
a
unit
unary
call
can
just
send
all
the
operations
in
one
batch,
so
the
client
starts
like
once
the
client
starts.
A
The
batch
initial
metadata
is
sent
followed
by
the
message
followed
by
the
trailing
metadata
and
now
so
three
three
of
the
steps
in
the
operations
in
the
batch
are
done,
but
the
batch
is
not
complete
yet
because
there
are
three
more.
The
client
is
just
waiting
for
those
two.
When
I
said
waiting,
it's
pulling
the
completion
queue.
It
knows
that
once
the
batch
is
completed,
the
result
will
be
post
on
the
completion
queue.
So
it's
just
wait.
The
client
is,
the
expectation
is
that
the
client
will
be
polling.
A
The
completion
key
for
the
result
so
now
at
the
server
server,
sees
the
initial
metadata.
So
much
more
specific.
To
be
more
specific,
it's
the
I/o
manager,
side
of
the
server
that
sees
the
initial
metadata
reads
off
a
few
bytes
passes
it
to
transport
transport
passes
it
realizes
it.
It's
in
initial
metadata
and
then
notifies
the
surface
that
you
have
an
incoming
call
so
then
surface
on
the
server
realizes
that
okay,
there
is
an
incoming
call.
So
what
do
I
need-
and
it
note
it
notes
based
on
the
method
name.
A
It
knows
that
it's
a
unary
call
and
then
it
says:
okay,
let
me
start
a
batch.
I
know
that
the
client
is
sending
message
and
trailing
metadata.
So
let
me
start
a
batch
to
read
to
receive
initial
metadata,
receive
message
and
receive
tailing
metadata
right,
so
it
starts
a
batch
and
that
since,
since
all
the
messages
are
already
on
the
way,
this
bath
completes
typically
very
quickly.
So
once
the
batch
completes,
a
notification
is
sent
on
the
completion
queue.
A
The
thread
picks
it
up
now
calls
your
higher
level
higher
layer
application
handler.
So
this
is
this.
Is
your
RPC
implementation
like
this?
Is
your
unity
RPC
implementation
that
is
called
so
that
one
thing
I
didn't
show
here
is
client
calls
the
wrap
language
which
deserialized
it
and
then
call
your
handler
and
then
your
handler,
the
application
handler
performs
the
operation
writes
the
result
back
that
essentially
results
in
another
batch
of
operations
right,
so
it
is
sending
its
initial
metadata.
A
The
server
is
pretty
much
done
with,
with
with
the
call
the
call
is
complete
on
the
server
on
the
client
so
recall
that
client
is
waiting
for
the
for
a
few
ups
right.
So
client
now
reads
them
reads.
The
initial
printer
data
reads
the
message
realizes
that
the
batch
is
complete,
also
a
notification
on
the
completion
queue
and
matches
done
on
the
client-side
application
thread.
Picks
up
call
is
complete
on
the
client,
so
this
is
unary.
A
Streaming
is
pretty
similar
like
it's
just
that
the
number
of
operations
in
the
batch
are
different
because
a
client
initially
the
only
difference,
is
that
client
cannot
in
a
unary
call
all
the
batches.
All
the
operations
were
sent
in
one
batch,
whereas
in
a
swimming
call
typically
client
sends
initial
metadata
and
then
sends
each
message
as
a
separate
batch,
because
it
doesn't
know
like
how
many
messages
might
be
sent
that
that
is
a
variable
number,
so
that's
other
than
that.
The
flow
is
pretty
similar
between
client
and
server.
A
So
that's
good
I
did
better
on
time
than
I
thought,
so
we
have
15
minutes
for
questions,
and
so
hopefully,
hopefully
all
of
you
found
there-
is
something
useful
from
it.
Some
takeaway
I
certainly
did
when
preparing
this
I,
mostly
work
on
the
I/o
manager
and
transport
components
and
I
know
enough
to
debug
issues
at
the
rest
of
the
stack,
but
not
in
not
in
this
much
detail.
So
I
certainly
learned,
and
so
it
definitely
I
encourage
you
to
please
contribute,
and
these
are
some
of
the
additional
resources
so
cool.
B
A
By
default,
that
is
the
behavior.
You
wouldn't
know
whether
the
server
is
reachable,
so
you
wouldn't
know
until
the
first
call
is
sent,
but
there
is
a
way
I
mean
there.
Is
there
are
api's?
If
you
really
want
to
know,
if
you
want
to
be
certain
that
you
can
let
the
server's
are
reachable,
there
is
an
API
to
force
force.
The
connection
for.
B
A
C
E
E
A
A
A
G
A
A
A
A
A
A
So
let's
say
that's
our
URI
resolves
to
only
to
two
backends,
but
you
created
like
ten
channels,
because
you
wanted
to
paralyze
in
your
application,
so
you
created
ten
channels
to
the
same.
You
array
they'll,
still
use
the
same
underlying
sub
channels
right,
so
we
do
that.
So
we
there's
no
explosion
of
connections.
If
we
try
to
reuse
the
connections,
the
underlined
connection
as
much
as
we
can,
which
is
something
I
didn't
talk
about,
but
so,
if
you
have
two
channels,
they
still
essentially
use
the
same
sub
Channel.
So
there
are
some
optimizations
there.
J
A
C
So
there's
also
this
geo
PC
feature
that
you
can
annotate
your
HTTP
a1
API,
with
options
in
the
service.
Ok
and
there's
also
Google
Cloud
endpoints,
where
you
can
do
this
this
way
over
the
llamo
extra
firearm.
Is
there
like
the
open?
Is
this
tool
open-source
as
well,
where
you
can
do
the
channel
llamo
specification,
because
I
only
found
the
inline
option
thing.
C
C
E
A
I
mean
from
the
core
perspective,
the
things
that
are
that
we
are
currently
working
on
very
actively
is
to
make
the
performance
even
better
it.
The
performance
is
actually
quite
good,
but
we're
still
still
a
lot
of
room
to
improve.
So
that's
that's
been
one
of
the
focus.
The
other
focus
is
also
on
sort
of
some
of
the
code
cleanup
like
making
the
filters.
It's
not
exactly
super
easy
to
add
a
new
filter
if
you
want
today,
so
we're
working
on
that
to
make
it
easier.
D
Like
so,
there
are
two
big
features
and
we
would
love
help
on
those
also
one
is
retries
and
retries
is
fairly
complex
because
it's
doing
retransfer
entry
tries
as
well
as
policy
based
sweet
rice
and
hedging
of
requests.
So
you
can
hedge
requests
at
multiple.
So
that's
one
big
feature
and
second
is
called
service
config.
So
what
service
config
does
is
basically
allows
us
to
specify
a
config
and
client
ownership.
D
So
when
you
deploy-
and
you
want
to
say
change
time-
you
you
want
to
instruct
the
tines
to
change
deadlines
because
you
updated
server,
then
you
can
handle
do
that
on
the
running
clients
and
service
to
service
convicts.
So
those
are
two
big
features
that
we
are
kind
of
have
proposed
renner.
If
working
on,
but
we'd
love
to
get
those
out.
A
Sure
a
good
question,
sir:
how
do
we
ensure
feature
parity
between
seeker
Java,
core,
and
so
we
I
mean
first,
the
wild
protocol
is
pretty
clear
and
we
have
we
done
a
bunch
of
interrupt
tests,
but
very
regularly
between
with
various
it's
a
matrix
of
clients
and
servers
and
and
I
mean
that
catches
most
of
the
issues.
So
we
don't
add.
E
A
M
A
Answer
written
may
be,
the
end,
can
add
more
so
I
mean
the
completion
queue
model
and
the
threading
model
is
completely
different
from
stubby
other
I
mean
so
that
being
very
different
and
and
the
code
really
they
do
not
share
any
code
at
all.
So
it's
completely
written
from
scratch.
However,
many
of
the
components
inspired
by
stubby
design
like
the
polling
engine
like
stubby
stubs
polling
engine,
has
been
battle,
tested
highly
optimized.
So
many
of
the
ideas
at
that
level
at
the
granularity
heavily
inspired
from
from
stubby,
he
wondered
anything
yep.
D
I
think
it's
mostly
the
ideas
came
from
sabe
we've
shared
whenever
we
can,
but
we
updated
the
protocol,
so
it's
HTTP
to
best,
so
anything
HTTP
to
specific
is
new
and
our
async
models
switch
to
being
a
completion
queue
based.
This
is
only
for
seeker
instead
of
being
a
callback
based,
which
is
what
we
used
in
study.
A
A
A
Mean
there's
a
lot
rewrite
everything
I'm
just
kidding,
but
like
probably
the
the
layering,
we
would
have
probably
implemented
it
a
bit
cleaner
right
now.
It
looks
a
lot
cleaner
on
the
slides,
but
it's
not
really
the
that
clean.
So
that's
definite
and
we
are
working
on
it.
That's
not
one
of
the
code
clean
up
things
so
yeah.
N
A
A
Yes,
I
mean,
however,
I
mean
I
must
add
that
I
mean
if
the
URI
completely
changed.
You
have
to
do
it
yourself,
but
if
the
resolution
like,
if
now
it's
resolving
to
new
back
ends,
no
that's
a
different
story
that
depends
on
the
elbe
policy
and
the
resolver
that
you
are
using
so
and
actually
again.
Stop
from
yesterday
covers
some
of
that.
O
J
On
this,
on
the
server
side,
the
service
itself
can
be
achieved,
correct,
correct
and
in
that
it
is
in
REO.
The
reads
and
writes
are
essentially
associated
with
master.
That's
right,
so
the
resolution
aspect
of
it
is
essentially
always
associated
with
the
master.
Is
that
something
that
we
are
PC
handles?
Or
is
it
the
server
side.
A
It
should
be,
it
has
to
be,
if
I
understand
your
question
correctly.
So
if
clearly,
the
resolver
cannot
like,
if
you
are
doing
it
right
clearly,
the
resolver
has
to
resolve
to
the
master
right,
not
the
so
that
functionality
is
cannot
be
at
the
GFP
sea
level
that
that's
something
you
have
to
handle
probably
have
a
different
URI
for
the
master
or
like
I,
have
your
own
custom
resolver
that
can
that
can
understand
it,
and.
D
I
A
And
we
we
experiment
a
lot
with
threading
models
and
it
keeps
it
still
evolving
a
mess.
I
mean
we
keep
experimenting
with
it,
but
one
thing
I
can
tell
you
is
applications
like
when
you
design
your
servers
when
you
arrange
your
servers
in
such
our
clients
in
such
a
way
that
there
are
you're
minimizing
threat,
jumps
that
yields
the
best
performance
so
when
I
say
thread,
jump
yeah
and
like
G
RPC
makes
it
very
easy
for
you
to
choose
like
create
your
own
completion
queues
in
the
in
the
pictures.
A
I
showed
only
one
completion
queue,
but
you
can
create
as
many
as
you
want
right
like
in
you,
can
create
multiple
completion
queues
and
you
can
associate
a
call
with
a
particular
completion
queue.
So
as
long
as
even
image
contention
on
the
completion
cubes
and
you
see,
RPC
tries
to
like
minimize
thread,
jumps
and,
and
that
usually
heals
the
best
performance.
A
C
Again
suggest
the
question
about
the
deployment
of
the
back
end
so
would
make
sense.
We
have
many
endpoints
for
the
different
backends,
because,
like
now,
everybody's
talking
about
sto
and
having
like
share
PC
load
balancing
so
where
we
have
apps
from
the
outside
coming
in,
would
will
be
better
to
have
like
a
single
NLB
and
then
hung
into
ISTE.
Oh
then,
it's
dancing
or
if
multiple
anal
bees
and
then
public
your
eyes
and
then
to
the
client
side,
load,
balancing
I.
D
So
I
think
he
covered
load
balancing
yesterday
and
there
are
multiple
options
and
it
really
makes
sense
in
everybody's
individual
scenario.
But
yes,
one
of
the
models
is
either
run
a
sidecar
on
Y
or
SPO
type
of
this
thing
and
use
that
to
do
load,
balancing
and
it's
very
convenient.
So
people
do
use
it.
So
that's
the
that's
the
benefit
of
it
and
it
can
do
very
advanced
load.
Balancing
the
cost
is
that
you
are
going
through
a
proxy.
So
if
you're
super
latency
or
throughput
sensitive
that
may
affect
you,
but
other
than
that.