►
From YouTube: Kong SF In-Person Meetup - 3/15/2023
Description
We are thrilled to welcome you to our first in-person meetup for 2023! Get ready for an evening filled with interesting talks, networking, food & drinks, and more! When we say more... well, you will have to come in-person to find out the surprises we have prepared for you
Agenda (and timecodes)
0:00 Everyone arrives, networking, drinks & snacks
4:26 Talk 1: Debug This: Enter Open Telemetry and Kong Gateway by @ViktorGamov
32:22 Talk 2: Kong Internals: Combating Latency with Yield() by Saju Pillai
1:01:39 Talk 3: Changing Engines in Flight Kong 3.x Series by Harry Bagdi
1:28:37 Q&A, networking, food & drinks
▬▬▬▬▬▬ KONG COMMUNITY ▬▬▬▬▬▬
⭐ https://github.com/Kong/kong
🐦 https://twitter.com/thekonginc
🌎 https://konghq.com/community/
🍻 https://www.meetup.com/pro/kong
❓ ️https://discuss.konghq.com/
💯 https://konghq.com/kong-champions/
A
A
A
Do
you
know
who's
Harry?
The
Harry
is
the
guy
he
will
be
presenting
in
some
gonna
be
talking.
Some
cool
stuff
I
came
to
hangout
with
all
of
you
guys
here
and
it's
great
to
get
back
on
some
of
the
in-person
in-person
event.
I,
don't
know
about
you,
but
I'm
super
excited
about
this
type
of
things.
I
work
at
kwank
as
a
developer
advocate
so
talking
to
developers.
You
know
if
you're
building
something
cool
with
konk
or
with
apis
come
to
talk
to
to
me
we're
very
interested
to
learn.
A
What
are
you
building
and
maybe
you
will
be
presented
for
you
know
some
of
the
meetups
here
all
right,
so
we're
gonna
have
a
three
speakers
today.
It's
like
super
exciting,
so
I'm
gonna
be
like
lightning
like
warming
up
people
for
the
for
main
stage,
presenters,
saju
and
Hari,
and
today
I'm
going
to
talk
about
like
a
small
thing,
called
open
Telemetry.
This
is
the
new
feature
that
we
added
in
the
it's
a
natively
supported
in
in
the
Quan
to
3.0.
A
How
many
of
you
actually
heard
about
Quan
before
how
many
of
you
I,
don't
know
like
building
some
apis
and
you're?
Looking
for
the
best
cloud
native,
open
source,
API
Gateway
for
your
API,
perfect,
perfect
I
want
to
see
more
more
users
of
Quant.
After
this
talk
and
my
my
my
goal
is
to
show
you
what
con
can
do
for
you,
so
you
can,
you
know,
be
more
effective
with
with
building
your
apis.
A
So
a
lot
of
times
getting
this
call
from
the
customers
and
saying
hey
like.
Why
are
you
slow?
And
the
first
thing
is
that,
like
it
can
be
true,
no
we're
trying
to
we
put
the
gateway.
We
configure
everything
and
like
we're,
hitting
this
API
and
it's
slow.
We
we
we
think
it's
called
because
before
that
API
was
working
fine
and
before
it
will
require
some
interesting
conversation
between
between
us
and
customers
and
how
they
how
they
would
investigate
this.
A
They
need
to
put
if
there's
some
metrics,
because
if
you
cannot
measure
something
you
probably
can
control-
and
this
conversation
looks
something
like
this
they're
explaining
what
they
do
in
this
or
what
they
configure.
What
kind
of
plugins
they
have
there?
What
kind
of
the
service
is
the
heating?
Maybe
there's
some
like
home,
brew,
aggregation
that
they're
building
here,
so
we
we
thought
that
we
can
do
better
and
help
people
to
solve
this
problem,
or
at
least
investigate
this
help
to
investigate
those
problems
very
very
easily.
A
A
You
didn't
expect
that
right.
Do
you
think
you
thought
you
will
come
here
to
draw
enjoying
the
beer
and
no
questions
asked
actually
there's
three
things
there.
Please,
sir?
Yes,
it's
a
good
good
point.
It's
a
it's
an
open
specification
where
different
companies
are
collaborating
on
top
to
create
a
specification
to
define
format
for
metrics
format
for
logs
and
format
for
traces.
So
that's
the
three
pillars
of
open
Telemetry.
A
So
when
you,
when
every
time
you're
talking
about
some
tool
that
either
support
ingestion
in
open
Telemetry
format
or
some
tool
that
can
export
this
data
in
open,
Telemetry
format,
you
need
to
ask
yourself
what
exactly
we
we're
tracking
here
since
Quant
involved
in
the
a
very
critical
path
from
traffic
that
comes
from
your
users
to
apis.
We
want
to
know
what
happens
along
the
way
and
the
way
how
to
know
this
is
to
by
implementing
traces.
A
A
So
well,
there's
a
gentleman
point
out
correctly:
it
is
open
process
of
creating
this
specification.
Many
companies
are
bringing
some
of
the
people
who
work
together,
so
open
Telemetry
went
from
the
very
long
way
from
open,
metrics,
open
senses
and
after
that,
finally,
everything
is
joined
and
everything
right
now
as
a
part
of
cncf
process.
So
if
you
go
to
opentelmetry.org
you'll
find
everything
about
this,
but
the
most
important
thing
is
that
you
want
to.
A
If
open
Telemetry
is
supported
by
the
product,
it
actually
allows
to
different
tools
to
swap
you
can
swap
those
tools
between
one
tool
that
can
support
up
into
another
tool.
Can
support
open
terms
you
can
mix
and
match.
So
as
a
user
you
are
winning
here
and
for
the
vendor,
obviously,
because
it's
open
source
vendors
want
to
support
Open
Standards,
rather
than
lock
people
into
some
proprietary
standards
so
and
for
for
business
benefits.
Why
you
want
to
do
this,
is
that
you
know
hello,
conquer.
Why
are
you
so
slow
I
want
to
answer?
A
Show
me
the
numbers
show
me
show
me
your
traces
show
me
your
your
data
and
we
will
figure
out.
So
it
is
not,
it
is
not
kind
of
like
a
after
thought.
It's
a
it's.
A
the
open,
Telemetry
instrumentation
is
the
built-in
inside
the
core
of
conch
and
saju,
and
Harry
will
be
talking
about
like
details
a
little
bit
later
and
many
things
that
Kong
does.
With
your
request.
You,
you
immediately
start
seeing
this
once
you
start
collecting
those
metrics
using
the
opentel
meter
format.
A
So
with
this
this,
so
what
what
it
can
do.
So,
even
if
you've
writing
the
plugin
for
con.
If
you're
writing
the
plugin
using
Lua,
you
will
be
able
access
to
this
in
the
maybe
instrument,
instrumentation
information
or
like
augmented,
instrumentation
information,
User
data
that
required
for
your
plugins.
A
So
far,
we
we
have
only
Lua
pgk
available.
However,
we
have
a
PD
case
for
grow,
Python
and
JavaScript,
but
also
we're
working
on
something
very
exciting
that
we're
going
to
share
very
soon
that
allows
you
re,
write
the
plugins
pretty
much
on
any
language
that,
like
I,
don't
know,
go
over
us
and
and
and
so
far
so
on,
but
right
now
the
the.
If
you
want
to
use
this
like
open,
Telemetry
stuff
inside
your
plugins,
you
will
be
tied
to
Lua.
A
That's
the
overall
platform
obligatory
slide.
That
explains
like
we
do
more
than
just
the
just
the
API
Gateway
there's
a
lot
of
things
that
we
we're
going
to
be
showing
in
the
next.
You
know
years
and
stuff
like
that,
and
today
we're
going
to
focus
only
one
small
small
bit
and
this
small
bit
is
the
API
Gateway
like
one
one
thing
that
will
be
standing
in
front
of
traffic,
that
between
your
users
and
your
your
application.
A
I,
don't
know
if
you've
seen
this
so
I
have
my
demo
running
in
inside
the
docker
of
my
container
I
run
I
run
my
I
run
my
Quant
that
front
my
two
local
Services
one
service
exposes
a
Lord
of
the
Rings
quote.
Another
service
exposes
Back
to
Future
quotes
and
the
way
how
it's
configured
right
now
it
is
using
I
have
a
one
service
that
is
in
everything
is
local
and
another
service
that
I
also
like
that
service.
That
creates
a
splash
in
the
recent
recent
month.
A
I
also
have
in
its
remote
service
that
I
don't
have
any
control
here.
So
two
Services
have
a
Lord
of
the
Rings
Back
to
Future
and
open
AI
service.
So
if
I,
my
Quant
is
already
now
running
this
locally
two
Services
again
running
locally
and
one
service
running
the
Somewhere
In
The
Clouds
of
open
API,
open
AI
folks.
So
if
I
will
go
and
say
just
do
something
like.
A
A
So
you
can
see
that
Kong
adds
additional
information
about
the
latency
that
invocation
of
external
service
might
cause,
even
though
we're
running
this
here,
it
still
takes
some
time
if
I
run
this
local
or
if
I
was
switching
something
more
more
serious.
For
example,
I
want
to
invoke
Da
Vinci
model
from
open
AI.
So
when
I
click
Send
see
the
times
started,
running
out
and
I
don't
know,
was
it
conk
oop?
Why
it's
not
showing?
Because
Zoom
is
not
allowed
me
to
do
cool
things?
A
Okay,
so
when
I
run
this
times,
like
you
know,
is
running
running
fast
and
I,
don't
know
if
it's
if
it's
clunk
or
my
API,
obviously
with
this
with
the
Quant
headers
I
can
see
that
this
is
actually
up
stimulates,
but
I
want
to
know
more
and
want
to
know
more
what
is
going
on
here.
So
with
this
this
UI,
it's
a
part
of
the
Quant
connect,
which
is
a
managed
service
for
managing
con.
A
Even
though
I'm
running
my
Quant
locally
here
on
my
computer
I,
still
like
to
use
this
nice
UI,
because
it's
you
know
available
for
for
free,
and
you
can
sign
up.
You
know
and
play
around
with
this.
So
let's,
let's
investigate
couple
things
here
so,
first
of
all,
in
order
to
enable
open,
Telemetry
I
need
to
start
my
con
with
a
special
special
parameters.
Where's,
my
docker
h,
Docker
compose
so
I
just
need
to
enable
two
things:
I
need
to
say
that
open,
Telemetry
tracing
also
everything
that
will
be
heating
quality.
A
Another
thing
is
that
I
want
to
use
some
providers
that
we
not
like
explicitly
communicate
it
or
partnered
with
data
dog
is
a
good
example,
because
data
dog
provides
open,
Telemetry
format
and
they
support
ingesting
data
in
open
Telemetry
format.
So
I
was
thinking.
Okay,
I'm
gonna
use
this.
So
the
way
how
it
will
work
Kronk
will
will
expose
those
those
data
there
would
be
the
intermediate
plugin.
That's
called
open,
simulated
collector
that
will
be
reading
data
from
quonk
in
pushing
data
to
datadoc,
and
we
will
hopefully
with
the
magic
of
the
demo.
A
A
So
let's
try
to
do
something
generate
some
some
traffic
for
Fort
Lord
of
the
Rings
for
local.
First,
like
let's
start
with
the
local
service,
and
after
that,
we
can,
we
can
see
what
we
can
do
with
other
plugins,
so
it
generated
some
some
traffic
and
generated
some
something,
and
what
I
want
to
see
here
inside
my
service
catalog
on
datadog.
A
As
always,
you
know
when
try
to
show
you
show
something
in
in
in
the
real
time.
Something
is
not
it's
not
working,
let's
see
if
it
is,
will
appear
for
the
last
15
minutes,
maybe
that
it
requires.
It
will
take
some
time
because
I'm
not
saying
this
directly
I'm
batching
those
those
traces
and
it
usually
takes
some
time
to
to
appear
here
inside
inside
a
service.
A
Yes,
that's
the
proof
of
the
this
is
live
demo.
It's
still,
it's
still
still
loading,
but
I
can
go
ahead
and
see.
Actually,
let
me
see
if
my
Zoom
go
up
yeah,
so
my
exporter
is
working
so
they're
sending
some
data
and
the
data
doc
needs
some
time
to
to
process
this
I
I
saw
this
in
in
my
previous
version.
I
actually
did
like
a
whole
live
stream
when
I
was
like
breaking
this
down,
so
it
will
take
some
time.
A
It's
a
data
dog,
you
know,
I,
don't
I
I,
don't
I,
don't
know
who
pays
for
for
data
Dot
massage
your
face
for
for
my
data
dog,
yes
yeah.
So
one
of
the
things
that
I
have
this
in
in
my
slide
is
that
you
know
the
open,
Telemetry
included
in
a
in
a
you
know
both
yeah,
it's
included
in
open
source
version
of
conk
and
Enterprise
version
of
con.
So
to
use
this,
you
know
you
just
need
to
start
using
this.
A
The
way
how
it
needs
to
be
done,
whereas
my
where's,
my
here
so
I'm,
go
ahead
here
and
say
where's
my
route,
my
Lord
of
the
Ring
route,
and
it
doesn't
see
this
is
what
I'm
hearing.
So
this
is
the
information
that
I'm
getting
from
my
connect,
but
I
want
to
see
like
a
deeper,
deeper
analysis,
so
for
this
I
need
to
go
and
say
open,
telemetry
I
need
to
select
the
open,
Telemetry
and
also
and
point
something
like
I
need
to
specify
a
location
of
my
open,
Telemetry
traces.
A
I
have
everything:
okay,
because
typing
is
always
not
fun,
so
quonk
in
this
case
will
be
using
and
pushes
these
traces
to
this
collector.
This
collector
will
take
this
this
data
and
push
it
into
into
Data
dot.
So
so
now,
with
a
little
bit
of
luck
and
again,
my
zoom
is
not
here
so
now
this
should
work
now.
This
should
work,
because
the
plugin
now
enabled
on
this
route
and
if
I
switch
back
to.
A
A
Take
some
time
take
some
time,
so,
while
it
is,
loading,
Quang
has
like
a
bunch
of
different
ways
how
we
can
extend
and
the
the
using
the
plugins
it's
the
way,
how
we
can
extend
this
one
and
in
order
to
I,
really
want
to
show
what's
the
problems
people
having
is
to
when
they
enable
too
many
plugins.
You
know,
Quang
has
a
lot
of
plugins
and
many
people
will
enable
those
many
plugins
and
after
that,
they
start
calling
saying
hey
like
I.
A
A
A
You
know
take
some
time,
which
is
pretty
easy,
because
there
is
only
thing
it
has
a
get
request
and
after
that
it
needs
to
use
the
internal
balancer
to
find
the
Upstream
server
and
get
request
back,
and
you
know
there's
nothing
really
exciting
here.
Exciting
things
will
start
happening
when
we're
starting
enable
like
more
plugins.
A
So
in
this
case,
I
will
go
ahead
and
enable
another
plugin
here
say
pre-function,
so
I
can
also
include
some
of
the
additional
information
and
the
Quant
Will
Call,
some
of
the
arbitrary
alua
function
and
I'll
just
say
so.
In
this
particular
case
it
will
add
some
header
to
my
response
that
will
take
Upstream
upstream
hostname
and
include
this
in
the
header.
Sometimes
it's
useful
information,
I
I
use
this
a
lot
when
I
need
to
know
from
what
pod
I'm
getting
this
response.
A
In
my
compress
cluster,
which
you
know
conk
also
runs
in
kubernetes
successfully
and
let's
generate
some
traffic
a
little
bit
more,
so
we
can
have
a
little
more
data
while
it
is
doing.
This
I
also
enable
few
things
on
my
on
my
DaVinci
route,
so
my
DaVinci
route
will
be
front
open.
Ai,
API
and
I
want
to
simplify
this
consumption
of
this
API
for
my
developer.
A
A
C
A
So,
let's
get
back
into
into
Data,
though,
hopefully
the
data
was
already
ingested
and
processed.
So
we
can
investigate
something
something
here.
So
one
of
the
recent.
Let's
take
a
look
here
if
we
will
be
able
to
see
traces
from
it's,
it's
already
a
look
on
these
ones.
Let's
see,
let's
see
live,
and
we
will
see
something
that
came
in
just
right
now,
16,
yes,
so
let's
take
a
look
on
this
get
request
now
so
since
I
since
I
added
that
this
this
UI
is
not
good.
A
I,
don't
see
it
it's,
it
can
not.
Allow
me
to
zoom
in
let's
see
if
I
yeah.
No,
it's
also
not
very
interesting
thing.
Let's
see
something
interesting,
okay,
so,
for
example,
this
is
actually
a
very
good
example,
because
it
shows
how
fast
Lua
jit
that
runs
inside
the
Quan.
This
is
only
what
just
only
29.
A
What
is
it
like
a
microseconds
to
to
execute
functions?
So
it's
pretty
cool
and
you
can
ask
such
why
it
is.
It
is
the
way
it
is
and
you
will
be
able
to
see
like
all
these
like
different
phases
in
this.
In
this
particular
case,
my
plugin
for
pre-function
will
be
executed
inside
the
access
phase.
Every
time
when
you're
hitting
this
get
endpoint,
it
will
go
into
this
axis
phase
and
let
me
see
if
I
will
be
able
to
show
you
if
I
will
be
able
to
show
you
yeah.
A
This
is
this
is
something
that
I
will
be
able
to
see
from
from
my
open,
Ai
call,
because
I
see
what's
the
URL
I
was
calling
and
I
see
that
you
know
my
request
takes
me
for
seconds
and
all
this
time.
It's
actually,
you
know,
spending
time
sitting
there
in
sitting
here
in
the
waiting
response
from
here,
but
also
I
see
how
much
time
I
spent
on
open
Telemetry
phase
to
to
to
capture
those
things.
And
you
see
this
request
this.
A
This
thing
is
like
a
negligible
comparing
to
how
long
it
takes
how
long
it
takes
to
get
this
response
from
from
openness
back
for
open,
AI
API.
Let's
see
if
I
have
any
like
a
cool,
very
long,
it's
milliseconds,
it's
not
very
interesting,
so
I
can
put
the
things
around.
Let's
do
one
last
thing
to
to
do
and
I
will
enable
Global
plugin
for
at
plugins,
say
rate
limiting
so
rate
limiting
and
I
will
will
goes.
It
will
tell
me
I
need
to
configure
second
I'll
just
do
here.
A
Where's.
The
second
fix
seconds
say
like
a
10
requests.
I
will
like
limited
this
very
hard
and
I
also
try
to
generate
some
load.
Let's
see
if
I
will
be
able
to
exceed
this
rate,
limiting
before
that,
I
didn't
have
any
errors,
so
everything
went
smoothly
and
now
it
says
None
at
200
and
300
responses,
so
meaning
that
429,
it's
the
rate,
limited
response.
Let's
see
if
we
will
be
able
to
see
this
in
in
the
traces
for
inside
the
datadog.
B
A
So
some
of
the
responses
like
get
like
nanoseconds
and
meaning
that
the
response
will
return
immediately
because
Quan
checked
the
your
client
doesn't
have
enough
the
free
yeah.
Where
is
it
internal?
It's
a
pre-function
in
rate,
limiting
text,
also
160
States,
what
the
microseconds
so
and
there's
no.
There
was
no
call
to
external
API,
so
this
is
how
we
can
prevent
like
very
expensive
API
to
get
invoked.
So
that's
the
quickly
quick
thing
about
the
open,
open,
Telemetry
support
inside
the
clock.
A
A
A
Yes,
that's
very
good
question
and
as
well
as
the
free
tier
on
the
Quant
connect
where
we
can
have
a
free,
UI,
free
analytics
and
all
this
kind
of
stuff
I
don't
know
actually
how
much
data
dot
cost,
though
so
probably
expensive,
but
you
can
use
I.
Think
the
Jagger
has
the
open,
Telemetry
ingestion
point.
So
you
you
don't
have
to
use
this
and
it
works
with
Jager.
It
works
with
with
honeycomb
with
honeycomb
setup,
it's
even
easier
with
honeycomb.
A
B
B
A
So
short
answer:
there's
no
right
now,
but
we
have
a
pgk
like
a
plug-in
development
kit
that
you
can
use
JavaScript
right
now,
Python
and
go
if
you
want
to
have
like
a
the
the
best
performance.
You
need
to
stick
to
Lua
because
stay
forward
for
the
next.
For
the
next
presentation.
It's
it's
that
it's
very
fast,
other
other
plug-in
models,
stay
working
out
of
process.
So
in
order
to
communicate
with
some
concrete
requests,
they
need
to
go.
A
You
know
outside
of
the
Chrome
VM
and
go
back
into
this
and
there's
Marshall
link
and
serialization.
We're
working
on
very
cool
thing
called
was
Max.
We
did
the
announcement
on
Kong
Summit
last
year
that
will
allow
you
to
run
proxyvas
and
filters
that
written
on
any
language,
as
long
as
this
is
supported
by
the
the
proxy
was
proxy.
Wasn't
specification,
okay,
yeah,
eventually,
that's
kind
of
like
a
future
where
we
can
see.
Conch
will
be
so
you
can
do
Rust
go
or
whatever
for
plugins.
E
E
You
could
call
it
a
customer
situation,
but
there
were
performance
issues
in
very
extreme
situations
with
Kong
that
we
kind
of
handle
and
I'm
going
to
use
that
example
as
an
excuse
to
actually
dive
a
few
layers
down
to
so
that
I.
Can
it's
interesting
for
me
and
I
I
I
hope
to
have
all
of
you
understand
a
little
more
how
com
actually
works
by
means
of
introduction.
I'm
saju
I
actually
run
engineering
and
support
here
and
with
the
permission
of
the
engineering
team.
Sometimes
they.
E
E
Ok,
let
me
just
figure
out
how
to
do
this
navigation
here.
Okay.
First.
Apologies
for
splashing,
a
big
graph
on
the
screen
so
step
one,
but
it's
interesting.
This
is
both
the
problem
statement
and
the
answer
here,
really
what
you
do.
What
this
graph
is
trying
to
tell
you
is
each
of
those
points
whatever
the
color
be.
They
are
the
99th
percentile
latency
from
Kong,
so
we
are
running
say
like
1000
for
10
minutes.
We
we
run
at
a
constant
RPS.
We
measure
the
latency
of
every
request.
E
For
that
run
we
take
the
99th
percentile
and
put
that
on
the
graph
and
again
and
again
and
again
the
the
the
two
lines,
the
the
red
and
green
line
going
up.
That's
the
problem.
Ask
the
number
of
routes
grow
in
Kong
like
if
you
just
saw
Wick,
add
a
route
like
two
routes
and
drop
plugins
on
them.
We
have
customers.
We
have
users
open
source
users
who
run
with
very
large
number
of
routes.
We
have
some
cases
where
routes
are
crossing.
E
Fifty
sixty
thousand
routes
now
in
steady
state
Kong
is
fine,
no
problem,
but
if
you
have
that
situation
and
you
start
making
a
lot
of
configuration,
changes
to
Kong
at
the
same
time-
so
imagine
maybe
you
are
you.
You
are
a
e-commerce
site,
that's
doing
a
Black
Friday
traffic
and
you
have
enabled
your
engineers
to
tune
the
Gateway,
the
API
Gateway.
E
As
they
see
traffic
coming
in
orders
coming
in
and
they're
responding
every
time
you
make
a
change
to
Kong
back,
then
you
would
see
the
latency
climb
up
if
you
had
a
large
number
of
routes
and
we're
just
talking
about
the
99th
percentile
latency.
Okay,
that's
a
problem
statement.
We
solved.
It
did
a
few
things.
E
I'm
just
going
to
talk
about
one
thing:
we
did
I'm
going
to
use
that
as
an
excuse
to
go
deep,
no
more
graphs
after
this,
so
a
quick
preamble
Kong
runs
in
many
mode.
The
mode
that
I've
chosen
to
talk
about
today
is
called
the
hybrid
mode.
That's
the
on
the
top.
You
see
the
control
plane
at
the
bottom.
You
see
the
data
planes
in
vix
demo
when
Rick
was
going
through
the
UI
enabling
plugins
Etc,
that
was
the
control,
plane,
UI
and
then
on
his
laptop
on
his
kubernetes
Etc.
E
He
was,
he
was
running
the
actual
data
plane,
so
this
is
a
very
standard
mode
for
large
installations
that
run
Kong.
Each
of
these
are
actually
very
many
Kong
instances
that
are
running
underneath.
So
when
you
make
changes
to
Kong
like
like
Rick
was
doing
through
the
UI,
you
can
do
it
through
the
API
and
we
have
various
other
mechanisms.
E
The
configuration
gets
pushed
over
the
wire
to
the
data
planes
and
the
data
plane
like
I
was
saying,
is
actually
made
up
of
many
workers,
so
we're
going
to
dive
into
one
of
those
workers
to
understand
what
could
be
going
on,
which
would
have
cost
that
bad
P99
latency
client.
So,
let's
understand
what
happened
so
that's
configuration
coming
down.
It's
literally
Json
blob
coming
over
a
websocket
connection.
E
Okay,
you
don't
have
to
pass
the
code,
don't
worry
about
it.
I've
very
helpfully.
Put
these
big
red
markers.
Well,
I
said
you
don't
have
to
pass
the
code,
but
just
permit
me
you
can
see
some
for
loops.
You
can
see
sorts,
you
can
see
essentially
a
lot
of
string,
processing,
kind
of
thing,
happening,
comparisons,
etc,
etc,
etc,
but
that's
gone
by
the
way.
The
problem
with
all
that
stuff
that's
happening
there
is
all
of
that
is
processing
on
the
CPU
right.
E
So
when
you're
processing
on
CPU,
it
prevents
Kong
from
paying
attention
to
to
to
its
throughput
and
we'll
explain
more.
Why?
But
this
this
essentially
was
the
problem.
Every
time
config
comes
down
and
you
have
a
large
number
of
routes.
We
have
to
do
all
of
this
processing
on
those
routes.
Now,
just
take
a
look
at
you
know
one
of
the
the
for
Loops.
If
you
had
a
very
few
number
of
routes,
say
five
thousand
six
thousand
routes
that
for
Loop
will
execute
very
fast.
E
You
are
actually
not
spending
too
long
on
CPU,
but
if
you
have
50
000
routes,
we
are
now
talking
like
a
10x
time
for
the
five
thousand
one
right,
so
so
things
get
linearly
worse
as
the
number
of
routes
increase
so
effectively.
We
are
bottlenecking
on
CPU
at
this
point
and
the
simplest
fix
we
could
do
literally
in
between
all
of
those
CPU
intensive
portions.
We
introduce
this
thing
known
as
yield,
which
is
a
point
of
this
talk.
E
E
So
the
big
thing
to
note
is
all
the
heavy
work
is
done
by
these
processes.
We
call
them
workers.
These
are
the
ones
that
are
receiving
your
requests
that
are
running
the
plugin
code
that
are
sending
the
request
forward,
etc,
etc.
All
of
these
workers
there's
usually
one
per
CPU.
So
if
you
one
per
core,
if
you
have
eight
cores
on
a
system,
you
will
get
almost
always
you
will
have
eight
workers
running.
E
All
of
them
are
single
threaded.
That's
the
main
point
here.
This
thing,
the
multi-process
but
single
threaded,
so
there's
only
one
sequence
of
execution
per
worker.
What
what
actually
happens
in
a
worker?
What
does
nginx
do
so?
We
are
almost
in
engine
pure
engine
accelerator.
At
this
point.
What
what
exactly
does
engineering?
Why
is
it
so
cool?
Why
is
it
so
fast
right,
nginx
is
effectively
a
gigantic
Loop.
We
call
it
the
event
Loop
and
it
it
it
broadly.
Does
this
a
lot
of
the
time
is
just
waiting
for
some
interesting.
E
E
D
E
Called
out
the
big
steps
that
happen
in
the
event,
Loop
and
some
of
these
steps
actually
are
very
important
to
understand
how
yield
Works
within
within
an
nginx
system,
but
here's
a
big
thing,
people
weight.
So
that's
that's
usually
for
a
lightly
loaded,
nginx,
that's
what
that's
where
it
will
be
sitting
most
of
the
time.
It's
waiting
for
the
kernel
to
wake
it
up,
saying
there
is
some
activity
available
you
you
need
to
do
something.
E
The
point
here
is
once
you
come
out
of
this
e
pool.
All
of
this
stuff
is
on
CPU
everything
other
than
that
accept.
Of
course,
everything
else
is
on
CPU.
So
the
way
to
read
this,
this
diagram
come
out
of
wait
for
network
activity,
kernel
wakes
you
up,
and
essentially
you
are
now
processing
all
the
all
the
I
o
That's
now
available
to
process
these
all
of
them
are
literally
looping
through
or
through
the
all,
the
events
that
are
generated
from
e-pol
one
at
a
time
and
that's
crucial.
E
D
C
E
E
Interesting
note:
there
is
one
more
data
structure.
It's
a
red
black
tree,
that's
called
timers.
We
are
not
going
to
go
into
this
again.
You
know
our
breadth.
First,
walk
of
a
tree
looks
like
you
are
essentially
doing
one
after
another,
each
node
right,
so
that
the
linear
movement
is
the
most
important
thing
to
understand.
So
there's
no
multi-threading,
it's
single
threaded,
each
client
gets
serviced
one
at
a
time
and
the
idea
is
that
you
do
something
very
quickly
go
to
the
next
one
and
the
next
one
and
the
next
one
and
the
next
one.
E
If
you
spend
too
long
on
any
one
client,
you
have
throughput
issues.
You
have
latency
problems,
okay,
so
what
does
yield
do
we
added
yield
and
yield
was
doing
magic.
So
this
is
just
a
yield
code.
Again,
you
don't
have
to
worry
too
much.
There's
some
debouncing
going
on,
but
the
big
thing
is
the
nginx
dot
sleep.
That's
an
open,
Rusty
function
that
is
doing
some
magic
for
us.
Let's
go
one
more
level.
Okay,.
E
I
talked
remember:
I
talked
about
the
events,
getting
posted
a
queue
getting
created,
that's
exactly
what
is
happening,
so
the
yield
code
that
we
showed
you
that
we
fixed
our
latency
by
adding
yield
yield
is
calling
this
C
method,
which
is
just
posting
an
event
that
is
adding
to
a
queue
right
and
this.
This
is
the
magic,
but
is
it
really
magic
here,
I
just
added
something
onto
a
queue?
How
exactly
did
that?
Allow
me
to
yield
like
is
that
is,
that
is
that
part
is
very
important
to
understand.
E
In
the
Sleep
code
we
are
just
adding
to
a
data
structure.
We
are
still
on
CPU.
How
exactly
does
that
work?
I
mean
what's
going
on
right
here
is
the
magic
there
was
the
last
line
called
Lua
yield
and
for
folks
who
work
with
node.js
Etc
other
yieldable
systems.
They
probably
understand
how
this
is
implemented,
but
essentially
core
routines.
E
So
all
of
the
lower
jet
implementation,
the
integration
that
happens
with
nginx,
is
all
about
core
routines,
and
it's
all
Cooperative,
you
know,
processing
what
the
yield
is
actually
doing
and
see.
This
is
if,
if
I
follow
the
code
all
the
way,
this
is
where
yield
gets
called:
okay
inside
the
nginx
dot
sleep.
You
are
here
at
this
point,
we'll.
E
D
E
You
compile
it
and
you
disassemble
it
and
you're
going
to
find
this
block.
I
hope
my
mouse
yeah
you're
going
to
find
this
block.
You
can
all
try
it
on
your
laptops
of
whatever
kind.
This
is
just
from
my
machine,
my
M1,
so
this
is
the
arm
specific
thing,
but
you
will
find
that
GCC
and
most
compilers
emit
that
information
known
as
unwind
info.
E
So
you
know
in
C,
plus
plus
etcetera
when
an
exception
gets
thrown
the
you
can
catch
an
exception
in
another
function,
further
up
exactly.
How
does
the
execution
resume
there
without
going
into
too
much
detail
that
unwind
info
literally
lists
down
every
single
function
and
all
the
places
in
the
function
where
you
can
resume
execution
and
any
time
you
call
that
method?
I
just
showed
you
the
unwind
method,
this
one
unwind
race
exception.
E
This
is
actually
part
of
an
ABI
by
the
it's
called
system:
five,
the
system,
Phi
ABI
for
most
processes
and
the
C
libraries.
It
insists
that
you
have
this
table.
So
when
your
compiler
and
Linker
generate
this
output,
they
will
all
have
this
information
and
that
unwind
function
is
actually
available
in
your
C
library
or
your
C,
plus
plus
standard
library,
and
when
you
call
that
it
knows
how
to
read
all
this
funky
information
and
actually
walk
back
checking
whether
an
exception
can
be
caught
or
not.
So
this
is
known
as
stack
and
winding.
E
You're
short
circuited.
All
this
note,
you
are
building
the
new
router
of
the
config
came,
you
are
deep
inside
you
did
US
you,
you,
you
called
yield,
which
called
the
the
funky
sleep
which
finally
called
yield
and
boom
you're
back
up
here,
where
you
triggered
the
score
routine
and
that's
the
short
circuit
right.
That's
how
you
have
now
kind
of
paused.
You
were
doing
this
complicated
for
loops
and
all
that
I
showed
at
the
starting.
E
E
Some
funky
code
doesn't
matter,
it
basically
says
run
me
later,
don't
run
me
now,
wait
a
little
run
me
later
and
boom.
We
are
back
here,
but
later
we
were
processing
stuff.
We
had
all
that
stack.
We
unmount
everything
we
have
come
down
here.
We
actually
actually
we
skip
one
whole
iteration.
We
come
down
here
and
we
are
back
here
and
the
router
continues
getting
built
right,
so
this
finally
makes
Kong
rather
happy.
So
what
I
was
showing
here
less
about
the
yielding
less
about?
All
of
that?
E
What
I
was
showing
to
you
is
how
nginx,
lower
jet
open
resty
cooperate
to
do
heavy
CPU
activities
without
actually
killing
the
throughput
or
killing
the
latency
right.
If
you
do
something
heavy
on
the
CPU,
while
you
are
servicing,
you
know,
100
000
requests
a
second
or
something
you're,
just
going
to
tank
all
the
requests
and
what
I
just
took.
C
C
E
The
beginning
great
question,
yeah
yeah,
so
the
question
is
I,
was
I,
was
building
a
router
on
the
CPU
I
reached
a
certain
step.
I
am
yielding
so
all
the
stuff
that
I
did
must
be
saved
and
I
must
resume
from
that
point,
so
that
magic
is
actually
happening
here
now
you
may
see
that
I've
actually
snipped
out
some
code,
because
I
cannot
get
the
whole
thing
on
on
the
page,
but
all
the
state
of
open,
resty,
the
lower
jet
state
is
actually
maintained
here
this.
E
This
is
essentially
a
symbol
table
or
environment
as
they
call
it
or
table.
That's
you
know,
so
so
this
environment,
it's
a
core
routines
environment.
That's
this
environment
eventually
makes
it
into
this
object,
and
this
object
is
the
one
that
is
getting
posted
to
the
queue.
So,
basically,
you
have
created
an
object.
It's
in.
D
C
C
E
Yes,
absolutely
stopping
especially,
is
expensive,
resuming
is
a
little
less
expensive
stopping
because
you
literally
have
to
go
through
every
frame
on
your
stack.
If
you
remember,
I
had
skipped
slide.
Oh
here
you
go.
No,
not
this
one
yeah
this
one,
this
one
on
the
left
hand
side
are
the.
Is
the
host
stack,
so
the
C,
the
C
runtime
stack,
and
this
is
your
lower
code.
The
lower
code
is
not
actually
set
up
as
frames
in
your
process.
E
These
are
special
data
structures
that
lower
jet
maintains
when
it
unwinds
it
has
to
process
this
this
this
come
here
go
in
manage
all
of
this
come
back
and
then
so
it's
very
costly.
Yes,
it
is.
It
is
costly
and
not
a
terribly
good
idea,
which
is
why
we
we
do
this.
We
do
this
debouncing.
If
you
call
yield
inside
a
tight
Loop,
we
actually
will
not
yield
we'll
just
debounce
you.
This
is
why,
because
it
is
costly,
so
you
have
to
balance
it
where
you
call
where
you,
where
you
call
it.
E
That's
a
fantastic
question,
so
SMX
is
a
completely
separate
subsystem
from
from
the
lower
jet
or
open
SD
subsystem.
So
nginx
has
the
notion
of
modules,
and
so
nginx
takes
a
request.
This
this
event
Loop
that
yeah,
let's
just
this,
this
is
good
enough.
This
event
Loop
us
as
connections,
go
through
this
event,
Loop.
That
is
this,
this
this
notion
of
phase
and
it's
a
very
common
thing.
Most
web
servers
have
this.
You
are
in
the
access
phase,
you're
in
the
rewrite
phase
and
so
forth
and
so
forth.
E
All
those
phases
what
engine
extra
says
it
basically
says:
hey.
Is
there
any
module
out
there
that
wants
to
handle
this
request
in
this
phase
and
so
open
resty
steps
up
with
low
adjut
and
says
I
would
love
to
and
and
it
handles
that
request
and
it
and
in
inside
open
stcs
is
that
some
Concord
that
wants
to
run
for
that
and
so
forth,
once
open
resty
is
done,
nginx
is
going
to
ask:
is
there
any
other
module
available.
E
In
turn,
turn
around
and
say:
is
there
any
filter?
That's
registered
here
that
right?
So
so
what
you
will
see
in
the
initial
in
about
six
or
seven
weeks,
the
first
drop
of
this
should
come
out.
What
you
will
actually
see
is
all
your
lower
code
runs
your
plugins
run
first
and
then
your
golang
rust
or
whatever
vasm,
compile
filters.
They
will
run
next.
So
that's
the
net
effect
you
will
see.
Now
later
we
are
actually
going
to
interview
both
together.
That's
really
complicated
engineering,
I'm,
really
looking
forward
to
that.
E
It's
callbacks
all
the
way.
So
so
folks,
one
of
the
one
of
the
things
about
about
nginx
is
the
thing
is
damn
fast,
but
you
need
to
understand
how
it
works.
You
need
to
give
it
some
respect
and
open
registry
is
a
fantastic
piece
of
engineering
that
makes
it
easy
all
the
score,
the
core
routine,
the
yield
stuff.
There's
a
lot
more
there
around
core
routines
core
sockets.
It
lets
you
program
without
realizing.
You
are
in
a
in
in
this
kind
of
I
o
Loop
in
an
event
Loop.
E
You
are
effectively
doing
synchronous
programming
without
thinking
about
all
these
async
callbacks
and
whatnot
I'm
sure
there
are
parallels
of
this
in
other
other
runtimes,
but
the
way
it's
executed
in
nginx
and
the
way
it's
executed
using
open,
resty
and
the
fact
that
the
lower
jet
instant
play,
which
is
in
many
cases
it's
it's
faster
than
some
of
the
assembly
generates,
is
faster
than
the
than
what
your
compilers
would
generate.
So
faster
than
C
double
quotes
right.
It's
really
really
really
beautifully
engineered.
E
E
And
and
I
I
remember,
my
product
guy
was
on
the
call
he
had
to
take
the
call
and
they
were
all
very
serious
and
they
were
like.
Oh
you
know,
we
ran
this
much
heavy
testing
against
all
our
API
stack.
Kong
was
there.
It
was
not
good
I'm
like
what
happened.
Everything
melted
only
Kong
remains
standing.
So
like
okay,
great
thank
you.
So
so
it's
also
this
philosophy.
By
the
way
this
I
have
to
say,
I,
keep
saying
to
my
Engineers.
Also
there's
this
philosophy.
Nginx
comes
from
the
philosophy
of.
E
And
you
give
me
some
CPU
I'm
going
to
take
all
of
it.
Getting
your
CPU
close
to
100
is
a
good
thing.
You
know
I'm
saying
this,
in
contrast
to
some
of
the
more
modern
approaches
of
thinking
about
problem.
If
you
hit
70
CPU
spin
up
five
more
machines,
no
nginx
will
do
its
level
best
to
take
all
the
resources
you
have
given
it
because
it
wants
to
use
every
possible
piece
every
possible
cycle
to
push
as
much
throughput
as
it
can.
That's
what
it's
designed
for.
D
E
Close
so
see
this,
this
is
the
nginx
level
API
most
of
the
time.
The
events
that
you're
posting-
they
are
just
simple
structures
in
memory
there
is
no
I
mean
Loa
only
comes
in
the
picture
if
there
is
actual
like
like
the
open,
resty
module,
the
lower
nginx
module
had
registered
for
something
interesting,
and
you
had
some
code
firing
and
you
decided
to
yield
or
or
decide
to
do
things
like
you
wanted
to
start
a
timer
like
you
used
an
underlying
nginx
facility.
Only
then
do
we
need
to
save
the
state
and
do
that.
E
Yes,
absolutely
absolutely
it
does
for
those
of
you
who
do
run
Kong.
Sometimes
you
may
have
seen
errors
like
we
run
out
of
timers
too
many
active,
timers
and
and
so
forth.
Basically,
what
that
means
is
the
the
this
data
structure
is
eating
memory.
This
is
in
memory.
You
can
post,
they
can
make
it
so
big
that
it
actually
starts
consuming
a
lot
of
your
memory,
and
it
takes
too
long
to
finish
this
iteration
remember
that
each
of
these
is
a
loop
that
has
to
go
through
the
full
data
structure.
E
E
Okay,
cool.
Thank
you
first,
thank
you
for
letting
me
do
this.
It
was
fun
learning
about
this
and,
to
be
very
honest,
we
were
not
sure
this
is
our
first
attempt
at
Kong
internals,
and
how
deep
should
we
go?
There
was
much
debater
about
whether
code
should
be
pasted
on
the
screen
or
not
I
I,
I
I
think
it
was
worth
it
I
hope
that
some
of
you
understand
the
the
physics
behind
how
how
this
works.
E
It's
it's
a
beautiful
piece
of
engineering,
nginx,
open
history
is
brilliant
and
lower
jet
is
just
out
of
this
world.
You
know,
if
you
guys
have
the
time
you
should
at
least
read
up
on
how
how
some
of
that
is
implemented.
Some
of
the
this
stack
Powers
a
huge
amount
of
traffic
on
the
internet,
a
huge
amount
right
so
worth
studying
this
a
bit
more
cool.
Thank
you.
So
much.
E
By
doing
why
do
I
say
it's
huge
amount
of
traffic,
all
all
the
all
the
big
ones
in
in
in
China,
Shopify,
Spotify,
I
can't
even
properly
tell
you
all
the
big
names
that
are
probably
running
this,
but
nginx
is
a
rock
solid
piece
of
engineering.
It
is
in
C
it's
a
little
hard
to
work
with
directly
open
industry
makes
it
easy
and
that's
why
that
stack
is
used
by
very.
C
F
F
Yeah
somewhere
around
2004,
like
2002
right,
like
some
yeah,
so
almost
20
20
years
right
now,
Kong
is
eight
years
old.
So
that's
not
a
whole
lot
of
time,
but
somehow
Kong
is
built
on
top
of
giants
which
makes
it
20
years
old
right.
So
when
you
work
on
Kong,
you
have
to
take
into
account
nginx
internals,
like
he
went
down
into
right.
So
it's
not
like
a
project
is
eight
years
old.
It
looks
like
that,
but
it's
actually
much
older
than
that
right
now.
F
I
joined
Kong
about
five
years
ago,
but
I
was
a
Kong
user
before
I
joined,
because
Kong
is
open
source.
So
you
know
the
power
of
Open
Source
and
we
live
in
the
land
of
startups
right.
The
city
of
startup
San
Francisco
here
so
when
I
joined,
Kong
Kong
was
actually
pretty
small.
We
were
I,
don't
know
we're
probably
15
10
15
Engineers,
you
know
trying
to
you
know
get
to
like
okay.
Can
we
get
this
really
good
product
out
there?
F
And
you
know
everybody
will
go
mad
and
buy
us
and
will
be
rich
right.
The
reality
is
quite
different
right.
The
reality
is
that
you
have
to
put
in
week
in
and
week
out
of
a
lot
of
effort
and
then
slowly
and
slowly
you
make
decisions
that
in
life
you
later
regret
right
here.
We
don't
regret
it
actually.
So
we
are.
F
We
have
come
a
long
way,
so
we
when,
when
I
was
using
Kong
I,
was
sort
of
trying
to
figure
out
who
actually
can
use
Lua
and
I
got
into
this
team,
and
everybody
was
like:
what's
Lua,
we
don't
want
to
deal
with
it.
So
I
was
like
okay,
I
mean
I
was
a
naive
engineer.
I
was
like
okay,
I'll,
take
it
up
and
then
I
started
contributing
to
Kong,
and
you
know
fast
forward,
I'm
here
and
suddenly
Kong
because
of
the
power
of
Open
Source.
We
suddenly
have
probably
more
than
50
000
users.
F
Because
it's
open
source
right
like
we,
don't
even
it's
very
hard
to
figure
out
some
of
these
names,
you
probably
have
heard
of
them,
have
interacted
with
them.
You
know
there
are
vaccine
providers.
There
are
stock
exchanges,
Banks
everybody
uses
Kong.
So
over
the
last
five
years,
what
I
saw
was?
Okay,
I
started
writing
code.
Okay,
let's
get
this
out,
let's,
let's
experiment
a
bit,
and
now
we
are
at
a
position
where
the
stakes
are
too
high.
F
Right
Kong
runs
at
such
a
critical
infrastructure
component
that
when
Kong
goes
down,
somebody
can
actually
go
to
jail
right,
very,
very
hard
regulatory
domains
right
so
Stakes
are
very
high,
which
means
that
reliability
performance
everything
is
super
critical.
You
know.
So
that's
that's
the
backdrop
of
why
we
had
to
go
down
into
details
and
sort
of
things
that
saju
talked
about
right
things
that
Victor
talked
about.
Okay.
Why
is
debugability
important
right
when
you're
losing
millions
of
dollars
a
minute
want
to
fix
your
systems
fast?
F
All
right,
so
we
will
go
through
a
brief
overview
and
I
want
to
give
you
a
little
bit
of
insight
into
how
how
we
look
at
an
open
source
project
that
we
have
been
leading.
You
know
one
of
the
largest
open
source
project
in
this
space
and
how
we
have
now
changed
our
thinking.
Now,
where
do
we
focus
right?
There's
a
again,
a
very
open
engineering
talk
here.
So
interrupt
me.
F
If
you
have
questions,
don't
don't
wait
till
the
end
at
all,
so
this
is
something
that
we
talked
about,
which
is
called
the
hybrid
mode,
so
you
refer
to
it.
Victor
referred
to
it.
So
that's
the
control
plane,
that's
the
database
postgres,
and
this
is
the
data
plane.
This
is
where
the
magic
happens.
That's
our
value.
Add
Pro
traffic
comes
in
traffic,
goes
out.
Everybody's
heard
of
nginx
Apache
use
some
sort
of
web
servers
right.
F
Okay,
so
that's
the
meat
inside
the
data
plane
box.
So
what
we
are
going
to
look
is
just
the
bottom
part.
We
are
not
going
to
look
at
the
control
plane
again.
We
have
got
workers,
so
each
worker,
the
exagger
explained,
is
a
Linux
process
right.
It
is
an
actual
process
and
it
has
some
data
structures
and
some
core
components:
okay,
how
many
people
work
with
microservices
today
here?
F
Okay,
a
good
60,
70
percent
of
the
crowd,
others
are
hopefully
migrating
to
microservices.
So
what
we
have
here
is
on
the
left
side.
We
have
the
router
which
receives
a
request
and
tells
okay.
Where
is
this
request
going
right?
So
if
you
have,
let's
say
a
Billings
API,
an
inventory
API
and
maybe
a
user's
API
right.
So
if
I
come
to
slash,
V1,
slash
billing?
Okay,
you
are
going
to
the
Billings
back
end
right.
That
decision
making
happens
in
the
router.
F
Then
this
is
where
the
magic
of
Kong
lies,
where
you
run
a
bunch
of
plugins
right.
So
those
are
rate
limiting
plug-in.
You
know
open
Telemetry
plugin,
the
various
plugins
that
you
know
Victor
set
up.
You
know,
injector
header,
remove
a
header,
you
know,
so
those
are
all
running
through
the
traffic
right.
That's
where
conch
major
value
addition
lies
and
then
finally
we
have
the
load
balancer.
F
So
the
load
balancer
is
what
enables
you
to
put
a
single
Kong
in
front
of
all
of
your
infrastructure
and
then
send
traffic
to
n
number
of
backends.
So
these
could
be.
You
know
you
are
running
20
containers
to
solve
your
traffic
or
you're
running
five
virtual
machines.
You
know
or
you're
running
three
kubernetes
pods,
you
know
whatever
you
have
so
so
that's
the
three
main
areas
of
Kong
like
so
that's
the
router.
F
So
what
we
were
facing
was
some
of
the
challenges
that
briefly
suju
talked
about
where
we
are
facing
issues
where
we
have
20
000
routes
configured
in
a
single
instance
of
conch
right
I
mean
some
of
our
largest
customers
go
even
beyond
that.
That
means
that
you
have
to
look
up
a
route
table
and
figure
out
which
of
those
20
000
services
do
I,
select
and
proxy
traffic
too
right
now
that
can
take
time,
and
this
has
to
execute
on
every
request
right
and
the
numbers
we
are
throwing
around.
F
The
second
thing
is,
as
these
number
of
routes
increase,
because
as
the
organizations
you
know
they
deploy
more
services,
you
have
already
some
Legacy
the
memory
footprint
of
conch
will
increase
right
and
if
you
are
running
like
at
a
large
scale
and
con
starts
heating
up
gigabytes
and
gigabytes
of
memory,
that's
direct
cost
right.
That's
a
higher
AWS
mail
for
you
that's
much
harder
to
scale
right
because
you
need
bigger
and
bigger
machines
right
and
finally,
there
are
limits
to
what
our
old
router
could
do
right
so,
which
was
again
written
in
Lua.
F
Lua
is
a
dynamic
language.
Logit
is
awesome,
but
there
are
limits
to
how
much
you
can
push
a
jit
right.
Those
who
have
tried
to
tune
the
jvm
probably
have
some
good
experience
and
horror
stories
to
share
tuning
jits
and
actually
making
them
work
are,
are
not
an
easy
thing.
So,
if
you
introduce
this,
we
have
some
performance
numbers
I'm,
going
to
show
them
towards
the
end.
The
most
important
thing
we
are
transitioning
into
more
safer
languages
and
at
least
that's
what
we
are
willing
to.
F
So
this
is
again
open
source
peace
is
an
open
source
code
base
written
completely
in
Rust,
we
used
very
different
kind
of
data
structures
that
support
us
do
not
recompute
the
entire
data
structure,
so
you
saw
we
had
four
Loops
in
the
earlier
presentation.
You
know
where
we'd
have
to
like.
Okay,
take
a
look
at
all
the
hundred
thousand
routes
and
build
a
new
data
structure
so
that
we
can
map
those
requests.
F
Now,
if
you
add
one
new
route,
you
have
to
rebuild
the
entire
data
structure,
which
takes
time
sits
on
the
CPU,
introduces
latency
and
boom.
You
have
that
graph
that
we
started
with
right,
like
P99,
just
climbing
up
and
up
and
up
what
we
are
doing
here
is.
We
are
trying
to
go
to
sync
to
con
change
these
data
structures
in
an
incremental
fashion.
F
Hey
you
added
something:
okay,
we'll
just
change
the
data
structure,
insert
the
right
node
at
the
right
place
in
the
tree
and
call
it
done
right
so
that
it's
a
plus
CPU
you
get
more
throughput
and
you
reduce
your
latency
all
right.
So
the
next
thing
we
have
already
talked
about
this
is
Lua.
Plugins
Lua
is
how
many
have
heard
of
Lua
or
had
heard
of
Lua
before
this
Meetup
yeah
I
mean
that's.
F
A
pretty
good
number
actually
didn't
even
expect
that
so
Luis
is
a
pretty
popular
languages,
I
mean
if
you're
running
or
playing
games.
You
know
pretty
popular
in
that
space.
Very
Niche,
embeddable
language-
you
can
pick
up
most
of
the
language
in
probably
15
minutes.
It's
it's
not
there's
nothing
more
to
it.
It's
very
simple
language.
F
F
How
many
are
have
heard
of
go,
yeah,
JavaScript,
even
more
hands
right,
so
these
are
more
popular
languages
and
one
of
the
core
key
aspects
that
I
explained
in
the
earlier
side.
Con
can
execute
your
business
logic
right.
So
you
know
I
want
to
remove
a
header
or
maybe
I
want
to
send
some
logs
to
Splunk.
You've
got
some
flux
Splunk
in
the
audience
or
maybe
I
want
to.
F
You
know,
capture
the
request
response
bodies,
and
you
know
forward
it
to
some
analytics
pipeline,
whatever
you
want
to
write
some
code
and
you
want
to
put
that
into
your
API
Gateway,
without
sacrificing
the
power
of
Lua,
Jet
and
nginx
that
we
talked
about
right.
So
how
do
we
do
that?
So
this
is
where
vasm
comes
in.
Vasm
stands
for
webassembly,
it's
a
new
standard.
F
You
know
how
jvm
is
compiled
ones.
You
know
Run
Anywhere,
you
just
take
the
same
object
files.
You
know
deploy
anywhere.
The
jvm
will
just
in
time
interpret
it
and
run
it
right,
but
it
is.
It
is
a
bytecode
format
that
it
compiles
to,
and
then
you
interpret
that
and
you
know,
gets
compiled
down
to
the
machine
code
right
web
assembly
originally
was
introduced
on
the
browser
side
right
where
browsers
run
in
a
very
secure
and
a
trusted
environment,
they
don't
have
access
to
your
file
system.
They
don't
allow
to
run.
F
You
know
a
lot
of
understate
code.
You
know
very
tightly
sandbox,
because
if
somebody
escapes
the
browser
they
have
access
to
your
systems.
Right,
like
the
security
boundaries,
are
tight
between
websites
right,
so
vasam
was
a
way
to
you
know,
compile
any
code
to
again
a
bytecode
format,
an
intermediate
format
which
then
gets
executed
down
into
machine
format.
F
It
runs
on
your
CPU
right,
so
it
could
be
x86
armor
whatever
so
that
started
on
the
browser
side
and
now
has
moved
over
to
the
server
side
and
there's
no
quite
some
again
research
going
on
a
lot
of
big
players
in
this
space.
What
we
have
introduced
is
you
can
run
open
Rusty.
We
run
open,
Rusty
and
run
Lua.
We
are
introducing
something
called
the
vasm
VM
and
now
you
can
develop
plugins
in
any
language
that
supports
spasm
right
and
there
are
I
think
about
seven
languages.
F
Now
that
compile
to
Azam
it's
increasing
day
by
day.
The
most
popular
in
the
group
is
probably
go
here.
Considering,
like
you,
know,
an
infrastructure
product.
So
we
are
go.
We
we
are
working
on
this
one.
We
have
been
working
on
this
one
for
a
while.
We
have
something
coming
out
pretty
soon,
where
you
can
actually
write
a
pretty
performant
plugin
in
you
know,
go
or
rest
or
you
know
other
languages
coming
soon,
and
then
you
can
hook
into
the
request
response
lifecycle.
So
you
can
read,
request,
read
response,
you
know.
F
If
you
want
to
do
some
validations
using
open,
API
spec,
you
can
do
that
right.
Any
all
of
that
will
be
possible
the
most
important
aspect.
Again,
we
are
an
open
source
company.
We
are
going
to
work
well
with
the
envoy
proxy,
so
whatever
filters
or
extension
logic
that
you
write
against
Envoy,
you
can
transplantly
transparently
be
migrated
into
Kong.
F
So
this
is
coming
out
later
this
year.
This
is
expected
to
give
you
a
lot
more
capabilities.
In
terms
of
you
know.
You
have
larger
ecosystems,
you
don't
you
can
just
go
ahead
and
use
you
know
some
llm.
You
know,
library
that
somebody
has
returned,
compile
it
down
to
vasm
and
just
use
it
right
and
suddenly
now
you
have
some
machine
learning
in
your
gateway
itself.
If
that's
something
that
you
want,
the
last
piece
is
load.
Balancer.
F
F
What
was
that
elb
elastic
load
balancer
by
AWS,
others,
F5
right?
A
fantastic
example
has
been
around.
Probably
the
only
name,
I
remember
before
the
cloud
days:
I
guess
right.
Ships
in
a
box
put
an
F5
in
front,
and
you
know
it
will
handle
any
kind
of
beating
that
you
give
it
right
and
like
reliability
right
plus.
So
it
does
that
plus
it
gives
you
this
magical
capability
where
you
send
all
traffic
you
wanted
to
it,
and
then
it
will
distribute
it
to
n
backends.
F
You
have
right
so
now
you
have
oh,
my
CPU
is
running
out
on
one
VMO
I'll,
just
spin
up
four
VMS
or
I'll.
Just
deploy
five
machines,
split
traffic
easier
said
than
done.
Splitting
traffic
carefully
is
very
hard.
What
Matrix
you
want
to
optimize
for
it
that
matters.
The
algorithms
have
become
much
more
sophisticated,
especially
with
large
traffic
profiles.
F
F
We
have
started
adding
support
for
new
some
new
features,
so
we
just
recently
shipped
what
we
call
latency
based
routing.
So
if
you
want
to
you
know
optimize
for
latency
in
your
gateway,
where
or
or
in
a
traffic,
where
you
know
the
least
loaded
backend,
when
a
backend
gets
loaded,
it
generally
becomes
slow,
because
you
know
you
don't
have
enough,
I
o
or
enough
Network
or
enough
CPU,
and
you
want
to
load
balance
based
on
just
latency,
like
I,
want
to
give
my
lowest
like
my
customers
as
low
as
latency
as
possible
right.
F
So
you
can
do
that.
We
are
adding
support
for
new
algorithms.
We've
got
better
short
circuiting.
Algorithms
that
we
are
currently
thinking
about.
The
most
important
piece
is
how
we
rebuild
the
router
and
I
talked
about.
You
know
how
we
incrementally
rebuild
the
router
now
to
make
sure
that
you
know
we
don't
eat
up
a
lot
of
CPU,
not
a
lot
of
memory.
We
want
to
do
the
same
with
load
balancer,
it's
a
complicated
project,
but
if
anybody
is
interested
you
know
definitely
come
talk
to
us
again.
F
It's
an
open
source
project
so
would
love
to
hear
you
know
what
you
think
about
it.
You
know
there
are
possibilities
for
contributions
as
well
all
right,
so
that
was
enough
of
details.
It's
inside
the
data
plane
at
this
layer
right.
This
is
the
layer
where
saju
started
with
where
we
said:
hey,
there's
a
big
Json
blob.
You
know
like
that.
Json
could
be.
F
F
What
if
we
could
just
not
have
that
problem
right
like
why?
Why
should
we
even
think
about
hey,
we
have
to
go,
do
some
yielding
and
things
like
that
or
what,
if
we
just
don't
have
that
problem
like
why?
Why
do
we
need
to
and
most
important
point
whenever
an
update
happens
here
and
an
update
gets
stored
in
the
database?
We
take
an
export
of
the
entire
database
and
then
shove
it
down
to
the
data
plane.
That's
a
that's
big
right!
So,
anytime,
you
add
a
route
to
20
000
routes.
F
We
have
the
twenty
thousand
and
one
routes
coming
down
here
right.
So
oh
hunked
updates.
What's
that
I
missed
the
C,
so
what
we
want
to
do
is
we
want
to
do
small,
chunked
updates.
You
know
how
how
we
can
do
replication
from
essentially
this
database
to
down
here
in
the
data
plane.
So
we
have
started
introducing
some
foundational
Technologies
for
that
Kong.
F
Now
the
data
plane
runs
on
top
of
something
called
lmdb,
lightning,
mapped
memory,
databases,
that's
that
is
essentially,
you
know
you
m
map
a
file
and
let
the
operating
system
take
care
of
paging
in
the
the
file
that
lives
on
the
disk
right.
So
that
gives
us
transactional
capabilities,
and
now
that
gives
us
capabilities
to
hey
something.
Changes
in
the
database
just
send
that
specific
update
down
to
the
data
plane,
and
that
way
you
get
rid
of
all
kinds
of
for
Loops.
F
You
know
of
N
N,
squared
problems,
kind
of
right,
and
now
you
are
just
at
o
of
one
for
most
parts
right
and-
and
you
solve
a
lot
of
those
problems.
So
this
is
this
is
like
zooming
out.
We
have
talked
about
the
router.
We
have
talked
about
plugins
and
we
have
talked
about
load,
balancer
right
and
all
of
these
problems
that
I
talked
about.
They
get
multiplied
by
n.
Okay,
we
do
not
forget
that
there
are
n
worker
processes
right
for
everything
we
talk
about.
F
That
gets
multiplied
by
the
number
of
cores
that
you
have
now.
There
are
beefier
machines.
I
started
out
when
there
were
only
one
core,
I
didn't
even
know
what
multicore
was
or
SMP
was
the
term
I
don't
know
if
they
call
that
yeah
I
thought
that
too
and
like
yeah
it
used
to
be
like
you
go
up
in
gigahertz.
F
You
do
not
scale
out
right,
so
we
want
to
do
incremental
synchronization
so
that
we
don't
have
any
blocking
on
the
CPU
as
we
realize
you
know
we
blocking
on
the
CPU
in
nginx
land,
but
you
know
don't
do
it,
so
we
want
to
reduce
the
Reliance
on.
You
know
full
configuration
loads.
So
essentially
you
the
the
moment
you
are
not
sending
out
a
10
megabytes
Json
and
instead
sending
out
like
a
100,
byte
Json
document.
You
suddenly
are
optimizing
for
the
you
know
load
on
the
database,
because
you
are
doing
that
export.
F
F
We
are
probably
going
to
talk
over
beers
after
this
event
on
the
same
topic
again
welcome
to
join
us
yeah.
So
this
is
all
like
about.
Like
you
know,
okay,
these
are
things
that
we
have
now
started.
Taking
very
much.
You
know
a
big
Focus.
So
where
are
we
now?
So?
This
is
another
graph,
don't
focus
on
the
numbers,
but
what
what
matters
here
is,
as
the
number
of
routes
were
added
are
throughput
reduced,
and
that
was
because
of
you
know.
Cache
is
maxing
out.
Caching,
cash
is
being
you
know.
F
F
This
one
is
the
a
very
early
preview
of
the
change
that
we
made
with
the
new
router
that
I
talked
about
written
in
Rust,
where
the
the
growth
rate
of
memory
of
Kong,
the
slope
of
this
line
and
the
slope
of
this
line
is
different
right.
So
when
you
increase
the
number
of
routes
in
Kong,
the
memory
usage
will
increase,
I
mean
I
mean
it
is.
It
is
an
open
problem
to
a
certain
extent,
but
the
constant
so
K
into
o
of
n.
Then
the
K
is
lower
right,
so
the
slope
is
lower.
F
That
means
that
you
know
instant
saving
right
like
if
you
move
from
old
version
to
new
version,
so
that's
it
I
mean
we
have
been
focusing
on
something
that
a
term
that
internally
is
called
robustness
right.
Where,
because
a
lot
of
people
or
a
lot
of
customers,
trust
Kong
to
perform
well
in
adverse
situations,
it
should
not
be
possible
for
Kong
to
you
know
you
cannot
take
down
Kong.
You
know
it
should
perform
at
a
very
reasonable
level
in
an
expected
fashion.
F
So
we
are
going
doing
a
critical
data
structures
are
being
Rewritten
again
an
exciting
time
to
work
at
Kong.
We
are
shooting
for
more
debuggability.
So
what
Victor
talked
about
again
an
open
Telemetry?
That
is
to
give
you
more
insight
when
you
try
to
use
products
like
like
our
products
as
to
how
is
the
product
doing
and
if
suddenly,
let's
say
Kong
starts
getting
slow.
What
is
the
root
cause
of
slovenation?
How
can
you
you
know
tweak
that
look?
Maybe
it's
my
DNS
server
that
is,
and
it's
getting
loaded
okay.
F
But
how
do
you
figure
out
that
it's
your
DNS
server,
you
know
like?
Okay?
Can
I
get
visibility
into
how
long
DNS
takes
time
so
that
that
that's
our
Focus?
So
this
this
is.
This
talk
is
mostly
about.
You
know
how
how
I
have
also
come
to
realize
that
you
know
how
we
mature
as
an
organization,
how
you
know
the
impact
of
the
product
changes.
You
know
in
early
years
you
know
you're
trying
to
get
to
build
something
that
you
want
to
ship
a
value
like
you
know.
F
F
Why
did
like
my
bug
will
result
in
you
know
sleepless
nights
for
some
people
right
so
so
that
approach
has
changed
and
that's
where
I
say
you
know
like
Kong
is
eight
years
old,
but
I
mean
staying
with
Kong
for
a
while
I,
don't
know
how
old
I
am
now
so
yeah.
So
that's
it.
That's
that's
the
talk,
love
to
talk
more
about
it.
I'm
really
I'm
active
on.
You
know
Twitter
with
responding,
so
you
can
contact
me
on
Twitter.
That's
my
email.
F
If
you
need
any
help
with
Kong,
we
have
got
cognation
where
we
discuss
Kong.
We
have
got
a
healthy
Community.
Who
can
respond
to
your
request
if
you
are
running
into
issues?
Any
kind
of
API
issues
that
you
are
running
into
the
community
is
pretty
focused
in
the
API
space
and
we've
got
the
repository.
Pretty
easy,
Kong
go,
follow
it
check
it
out,
give
it
a
GitHub
star.
F
If
you
like
the
the
Meetup
and
always
looking
for
contributors,
we
have
some
healthy
contributions
going
on
across
the
product,
so
probably
in
every
space
you
know
uis
apis
backends
want
to
hack
luajit
want
to
write
custom
plugins,
we
have
something
or
the
other
going
on.
Victor
can
also
talk
more
about.
You
know
we
have
pretty
cool
swag
with
open
source.
Community
engagement
and
open
source
contributions
usually
amount
to
something
good
on
your
profile
too.
So
that's
it.
Thank
you
so
much
any
questions.
F
Yeah
I
mean
so
we
are
pretty
active
in
the
kubernetes
community.
We
are
trying
to
do
our
share
to
make
it
easier
to
use.
We
have
been
successful
to
some
extent
right.
We
have
got
a
good
mind,
share,
there's
Gateway
API,
you
know
you
can
run
Kong
in
kubernetes
much
more
easily
now
compared
to
you
know,
if
you
try
to
run
some
other
Legacy
gateways
so
that
so
con
comes
packaged
in
some
kubernetes
providers.
Con
can
do
kubernetes.
Other
aspect
is
nobody.
F
F
F
A
A
Yeah
we
we
have
the
the
project
that
we
develop
in-house
and
we
contributed
this
to
cncf.
It's
called
Cuba
it's.
What
was
at
the
time
when
we
contributed
this.
It
was
the
only
Envoy
based
service
mesh
that
was
in
cscf
and
istio,
followed
and
joined
cncf,
but
how
many
service
measures
right
now,
since
you're,
like
five
or
six
different
service,
meshes
in
sincere?
How.
A
A
Any
any
questions,
any
like
suggestions,
maybe
like
What
folks,
would
like
to
see
in
future
meetups
that
maybe
we
can
bring
some
I,
don't
know
server
smash
people.
We
have
a
like
a
big
part
of
the
the
front-end
style
people
as
well
like
we
do
have
a
con
components
which
is
like
a
UI
library
for
JavaScript
based
on
The
View.
It's
pretty
awesome
like
you
can
go
and
just
Google
konk.
D
Yep
so
I
have
a
small
question
about
the
graph
you
showed
I
think
it
was
like
slide.
24.
25
yeah,
oh
no,
the
one
before
yeah
this
one.
How
can
it
be
more
performance
with
more
routes
like
the
blue,
like
I,
can
see
how
the
new
version
with
like
using
diffs
can
be
less
performance
when
you
have
a
little
bit
of
roots
like
low
count
of
roots?
But
how
can
it
be?
How
can
it
increase.
F
F
E
Is
actually
a
multivariate
answer,
one
this
graph
say
compared
to
the
graph
I
was
showing.
There
are
two
separate
things
that
in
the
the
the
version
that
you
see
the
red
line,
is
it
has
a
label
2.8.
So
that's
the
old
version,
3.0
3.0
is
a
three
x
is
the
new
new
series.
The
two
eight
series
is
essentially
taking
all
your
routes,
processing
them
and
building
a
big
table.
It's
actually
a
lookup
table,
and,
and
then
you
use
various
techniques
to
find
your
route
and
then
apply
3.0.
The
router
is
dramatically
different.
E
That
router
is
actually
a
mini
for
lack
of
better
word,
I'm,
just
going
to
say
virtual
machine.
It's
literally,
we
came
up
with
a
DSL
with
which
you
can
express
your
routes,
and
that
blue
line
is
literally
that
that
engine
or
other
we
call
it
the
rust
ATC
router
that
router
executing
and
how
fast
it
can
execute
across
routes
actually
that
graph.
E
There
are
many
other
things
that
are
happening
there,
but
we
are
not
even
comparing
apples
to
app
one
is
building
tables
and
and
kind
of
indexing
into
them.
The
other
one
is
actually
running
programs,
so
in
com
3x
onwards,
the
routes
are
actually
mini.
Programs
which
I
mean,
of
course,
that's
not
how
we
expose
it
to
end
users,
but
there
are
literally
many
programs
that
are
executed
by
this
engine
sitting
inside
of
things.
So
that's
now
the
performance
characteristics
of
that
engine.
Why
you
know
you
see
the
S?
E
F
Plus
the
other
thing
to
observe
this
x
axis
is
actually
oh
sorry.
The
y-axis
is
actually
not
that
big
it.
It
looks
that
huge,
but
it's
it's
also
some
some
variance
here
that
will
explain
it
or
some
deviation
so
yeah,
but
the
most
important
aspect
is
like
how
this
crashes
down
and
this
stays
on
the
top.
E
E
Just
wanted
to
make
this
comment.
It
came
to
mind
while
Harry
Harry
was
just
kind
of
describing
our
customers
see
a
lot
of
you.
You
work
in
SAS
environments
or
aspire
to
build
SAS
companies,
large
ones.
Hopefully,
Kong
is
in
this
very
interesting
place
where
we
actually
are
also
building
our
SAS.
Like
most
companies
of
our
open
source
infrastructural
player,
like
confluent
elastic
Etc,
we
are
a
bit
like
them
right.
They
all
solve
for
one
piece
of
the
infrastructure
puzzle.
E
We
are
the
front
door
of
your
infrastructure
and
that's
what
we
solve
for
see
all
this
fascination
with
the
Lua
Jet
and
performance
and
caching,
cache
misses
and
optimizations.
We
have
to
do
this
because,
unlike
a
lot
of
New
Gen
companies
out
there,
we
are
actually
building
a
brick
that
is
used
by
people
to
build
their
SAS,
and
we
cannot
afford
to
not
be
robust
now
again
without
taking
names
here.
E
Most
of,
if
you
exclude
Amazon
us
and
I
can
take
names
and
exclusions.
So
if
you
exclude
Amazon
almost
all
the
time
you
buy
something
online
through
a
e-retailer
you're,
going
through
Kong,
a
single
API
drop
for
a
banking
customer,
like
literally
one
API
you're
talking
you're
talking
millions
of
API
calls
happening
a
single
API
going
down
or
con
going
down
or
having
low
throughput
for,
say,
10
seconds,
it's
a
million
dollars
gone
right
there
boom.
So
the
expectations
on
software.
At
this
layer
I
mean
it's
not
just
Kong
infrastructure.
Most
people
expect
that
be.
E
E
With
your
permission
that
we
were
able
to
geek
out
a
little
bit
today,
it
just
goes
to
show
the
kind
of
engineering
we
have
to
do
here
at
the
same
time,
you're
building
a
sas2.
So
we
we
understand
that
part
of
the
story
too.
We
are
what
our
CEO
calls
a
dual
train
company.
We
are
building
the
infrastructure
piece
that
goes
to
other
people's
data
centers
and
we're
building
our
own
SAS.
So
it's
it's
a
very
hard
thing
for
a
company
to
do
these
two
essentially
different
business
models
right.
E
F
Thank
you
so
much
for
taking
the
time
out
and
coming
here
and
again
allowing
us
to
just
geek
out
and
talk
to
you
about
things
that
we
find
interesting.
We
host
these
meetups
on
some
regular
Cadence.
So
if
you
are
interested
in
talking,
please
come
talk
to
me.
Saju
or
Victor
just
find
us-
and
you
know
talk
to
us
again.
We
we're
very
much
you
know
in
terms
of
open
development,
that's
where
we
want
to
be
so.
If
you
have
an
interest
in
any
of
these
projects,
come
come,
find
us.
Okay!
Thank
you.