►
From YouTube: Achieving a Highly Available and Scalable Multi-Tenant... - Ansu Ann Varghese & Aleksander Slominski
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Achieving a Highly Available and Scalable Multi-Tenant Eventing Backend Architecture in Your Serverless Products - Ansu Ann Varghese & Aleksander Slominski, IBM
Knative Eventing provides a serverless experience with autoscaling but its backend components do not provide scaling and high availability. And today there is a need for large scale customers to implement highly available and scalable event sources. Specifically, in regards to two dimensions: adjust backend deployments based on workload and adjust deployments based on the number of source instances in a multi-tenant environment. It is expected that a serverless platform consumes the least amount of cluster resources while being scalable and resilient. There are no generic sustainable mechanism supporting Knative components with different backing technologies. We have developed a new reusable, customizable and pluggable architecture that schedules resources on the best dataplane pod considering all the highest priorities. In this talk, we will show how the new Eventing scheduler has been used in Knative Eventing for scaling KafkaSource’s dataplane and ensuring consumers are highly available, and how you can use a similar design for any of your custom resources. We will also show how various Kafka consumer settings were tuned to increase throughput in a real production environment.
A
A
This
is
a
basic
must
requirement
for
most
Enterprise
organizations,
Building
Solutions,
on
on
top
of
k-native,
so
we
have
thus
implemented
an
Eventing
scheduler
that
gives
you
a
scaling
solution
and
distributes
consumers
across
the
data
plane.
Pods.
Based
on
your
priorities,
we
will
then
go
through
a
demonstration
of
this
new
feature
using
a
Kafka
Source
installation
on
IBM
cloud
kubernetes
service.
A
This
is
a
very
active
area
with
a
lot
of
interest
from
the
Upstream
community
and
from
organizations
Building
Products
based
on
k-native
Eventing.
There's
work
currently
being
done
to
expand
the
scalability
using
this
scheduler
into
other
k-native
components
and
also
integrating
these
resources
directly
with
the
the
kdel
auto
scaler.
A
Now,
as
K
native
is
built
around
a
serverless
experience,
users
expect
that
Services,
when
not
used,
will
be
scaled
down
to
zero
and
when
used
to
scale
up
and
down
corresponding
to
the
amount
of
events
that
need
to
be
pulled.
So
our
back-end
deployments
must
have
the
capability
to
Auto
scale
up
or
down
as
the
workload
demands
change
to
accommodate
for
faster
or
for
idle
processing.
A
Third
point:
the
backend
data
plane
and
controller
should
help
with
providing
a
maximum
throughput
and
high
compute
density
on
the
running
cluster.
Next,
we
should
be
providing
highly
available
and
resilient
support
for
our
resources
across
all
failure.
Domains
in
a
multi-region
environment
such
that,
when
there
is
a
failure,
recovery
is
quick
and
disruption
is,
is
minimal.
A
A
So
all
of
these
expectations
that
we
discussed
lay
out
the
foundation
for
implementing
this
new
Eventing
scheduler
So.
Currently,
each
dispatcher
backend
replica
instantiates
one
consumer
for
each
candida
resource
and
with
an
increasing
number
of
resources,
the
dispatcher
resources
need
to
be
increased
as
well.
We
also
need
to
be
able
to
configure
how
many
consumers
we
need
to
run
for
a
specific
resource
in.
In
addition,
users
today
don't
have
a
way
to
configure
parallel
deliveries
to
to
increase
throughput.
A
A
This
new
Eventing
scheduler
is
a
also
a
k-native
generic
component.
It
today
lives
in
the
k-native
Eventing
repository
along
with
all
of
its
sub-components,
and
it's
a
plug-in
implementations.
It
can
be
customized
to
use
with
your
custom
resources
and
sources
and
is
meant
to
be
a
reusable
framework.
It's
not
specific
to
any
existing
K
native
implementation.
A
A
The
scheduler
also
allows
many
scheduling,
features
to
be
implemented
as
plugins,
while
keeping
the
core
of
the
scheduler
simple
and
maintainable
scheduling
happens
in
a
series
of
stages.
First
is
the
filter
stage,
and
these
filter,
plugins
known
as
predicates,
are
used
to
filter
out
the
pods
where,
where
a
replica
cannot
be
placed,
the
next
stage
is
scoring
the
score
scoring
plugins
also
called
as
priorities
provide
a
score
to
each
pod
that
has
passed
the
filtering
space
scheduled
will
then
select
the
Pod
with
the
highest
weighted
score
sum.
A
So
next
the
scheduler
can
handle
recovery
from
unexpected
domain
failures
or
planned
worker
restarts
due
to
some
choices
in
its
design
data
plane,
replicas
used
for
scheduling.
Consumers
are
actually
part
of
a
stateful
step
set
architecture
whose
pod
anti-affinity
rules
is
such
that
it
it
constrains,
which
node
each
replica
in
the
stateful
set
is
allowed
to
be
scheduled
on.
The
scheduler
also
relies
on
the
the
sticky
identity
of
pods
in
a
stateful
set.
A
A
A
A
Okay,
so
each
Kafka
Source
instance
is
associated
with
the
creation
of
a
new
consumer
group
which
behaves
as
a
virtual
pod
within
the
Eventing
scheduler
framework,
and
it
can
be
scaled
up
and
down
by
increasing
or
decreasing
the
number
of
virtual
pod
replicas
for
maximum
distributed
processing.
It
can
be
manually
scaled
or
it
can
be
Auto
scaled
by
by
Cada.
A
Now
the
job
of
the
scheduler
is
to
place
these
virtual
replicas
onto
real
kubernetes
data,
plane
pods
that
you
see
at
the
very
bottom
and
and
then
come
up
with
a
set
of
placements,
which
is
then
propagated
back
to
the
Kafka
Source
status,
fields
and
and
a
new
consumer
resource
is
created
for
each
of
those
data,
plane,
pod,
pods
and
and
they're
bound
one
to
one
on
the
right
side
of
the
screen.
You'll
see
a
sample
scheduler
config
map.
A
A
The
the
scheduler
and
how
it
picks
placements
is
very
much
configurable
in
terms
of
choosing
the
strategies
that
is
important
to
you
and
also
being
able
to
easily
plug
in
these
new
strategies,
and
such
flexibility
helps
in
separating
concerns
for
different
Persona
and
for
different
environments.
For
example,
a
developer
in
a
local
environment
may
need
a
different
scheduling
setup
from
an
SRE
in
a
production
environment.
A
A
Some
other
benefits
to
keeping
the
scheduler
component
separate
is
having
resource
controllers
that
are
easily
extensible
and
Loosely
coupled
from
one
another.
Now
this
goes
without
saying
that
it,
the
scheduler,
provides
a
a
simpler
way
of
supporting
High
availability
and
scalability
for
all
your
resources.
Many
times
event
meshes
rely
on
not
just
1K
native
component,
but
a
collection
of
them
with
one
code
package
to
maintain
for
the
scheduler.
It
makes
this
whole
thing
more
sustainable
across
different
K
native
resources.
A
A
Okay,
so
let's
switch
screens
to
our
terminal
okay.
So
let
me
give
you
a
look
into
what
the
Kafka
source
custom
resource
looks
like
so
here,
I
have
a
Kafka
Source
specified
that
in
the
specification
has
information
about
the
Kafka
event,
streams.
Instance,
you
can
put
in
your
information
about
your
bootstrap
servers
and
also
a
secret
containing
that
will
help
authenticate
to
your
Kafka
Source.
In
addition,
I
have
specified
12
consumers
to
as
just
to
just
start
started
with
okay.
A
A
A
Okay,
so,
as
you
can
see
here,
this
is
showing
the
placements
information
in
the
status
section
of
the
Kafka
source
and,
as
you
can
see
here,
the
12
replicas
have
been
equally
distributed
among
three
different
dispatcher
pods
and
the
way
the
state
full
set
architecture
is
defined.
These
pods
live
on
different
nodes
in
different
zones,
so
this
satisfies
h,
a
okay.
A
A
Okay,
so
what
you
see
here
is
that
cada
is
in
control
now
and
it
has
Auto
scaled.
The
12
replicas
that
we
had
originally
request
requested
down
to
zero,
because
the
Kafka
Source
currently
is
idle
and
it's
not
processing
any
events.
I
haven't
sent
any
events
yet
so
let
me
go
ahead
and
call
my
event
producer
script
to
to
send
some
events
I'm
going
to
do
just
10
events.
A
Okay,
so
all
10
messages
have
been
sent
and
let's
see
what's
now
going
to
happen
to
our
consumer
group,
okay.
So
what
you
see
here
is
that
cada
has
now
increased
the
number
of
scalable
replicas
on
the
Kafka
Source
from
zero
to
one
so
that
there
is
a
consumer
replica
doing
the
processing
of
these
events
and
to
confirm
that
the
events
are
being
received
at
the
sink.
A
We
can
quickly
just
take
a
look
at
the
sync
which
is
just
an
event
display,
and
here
you
see,
the
events
that
I
sent
to
my
Kafka
topic
have
been
received
and
are
being
logged
in
my
sync
service,
yeah,
okay
and
once
the
processing
of
these
10
events
are
over,
we
should
expect
again
Keda
to
bring
back
the
replica
account
down
to
zero
there.
You
go
Okay,
so
we've
seen
the
whole
flow
from
Kafka
topics
to
the
Kafka
source,
with
the
autoscaler
in
action
and
the
messages
have
been
processed
and
received
at
the
sync
location.
A
A
A
A
A
This
also
makes
applications
more
resilient
to
Lost
events.
However,
it
might
result
in
duplicate
events
being
sent
among
other
performance
issues
like
events
in
a
queue
taking
too
long
to
be
processed
or
not
having
enough
concurrent
clients
to
process
events
synchronously
and
we've
all
seen
all
this,
but
what
I
want
to
share
with
you
is
that
the
Eventing
Kafka
broker
data
plane
today
exposes
some
of
these
Kafka
consumer
configurations
through
config
maps
that
can
be
tuned
to
suit
your
specific
workloads.
A
These
are
some
of
the
parameters
that,
when
tweaked,
showed
a
significant
Improvement
in
performance,
especially
to
lower
the
number
of
duplicate
messages
and
to
speed
up
concurrent
processing
yeah
next
slide.
If
you're
still
interested
in
the
work
here,
it
might
help
to
know
that
there
is
work
currently
being
done
to
expand
scalability
using
the
scheduler,
with
other
k-native
components
like
triggers
and
channels
we're
also
integrating.
Many
of
these
kafka-backed
k-native
resources
directly
with
the
cada
Auto
scaler,
so
direct
integration
is
guaranteed
to
really
simplify
further
and
operators,
experience
whose
job
is
to
manage
various
installations.
A
This
brings
us
to
the
end
of
this
talk.
Thank
you
so
much
for
attending
and
taking
the
time
to
listen
to
what's
going
on
in
this
space
to
have
scalable
sources
and
to
support
our
keynative
users
in
a
production
setting.
It's
also
been
a
great
time
collaborating
with
our
red
hat
serverless
team
partners
on
bringing
the
solution
to
completion
and
also
members
of
the
IBM
research
and
the
product
teams
at
IBM
cloud.