►
From YouTube: Volcano: Intro & Deep Dive - Klaus Ma, Huawei Cloud
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Volcano: Intro & Deep Dive - Klaus Ma, Huawei Cloud
Volcano is a system for running high-performance workloads on Kubernetes. It features powerful batch scheduling capability that Kubernetes cannot provide but is commonly required by many classes of high-performance workloads such as ML/DL, big data application and Bioinformatics/Genomics. During this session the Volcano maintainers will introduce the project for newcomers and then provide updates on the latest and greatest features landing in Volcano. If you are interested in running high-performance workloads in Kubernetes, this session is for you.
A
This
is
klaus,
I'm
the
founder
of
volcano
and
cool
batch,
and
I
used
to
be
the
co-leader
of
sixth
gathering
and
taking
the
lead
of
cncf
tagalong
time.
Volcano
is
a
cloud
native
batch
system
for
intelligent
workload
such
as
hpc,
ai
and
big
data.
It
was
promoted
to
sincere
fingerprinting
project
earlier
this
year,
so
it's
my
pleasure
to
give
a
introduction
and
deep
dive
on
this
project.
A
A
A
So
more
and
more
organizations
are
leveraging
cloud
native
technical
to
build
a
unified
platform
for
our
workloads,
but
there's
still
some
gap
in
the
cloud
native
ecosystem.
So
we
built
a
cool
batch
at
2017
to
handle
scattering
gap.
The
then
we
build
the
volcano
based
on
the
cool
batch
to
handle
all
the
other
related
gaps.
A
Here
is
a
major
gap
in
the
cloud
native
ecosystem
for
batch
workload.
The
first
one
is
the
job
management.
It's
a
common
requirement
to
have
different
product
template
and
a
fun
green
lifecycle
management
in
a
job.
It's
a
complex
and
hard
to
maintain
to
build
a
cr
to
build
crd
for
different
type
of
job.
A
The
second
gap
is
about
the
scheduling
such
as
the
priority
functioning
gun
capability
resolution
and
backfill,
and
so
on.
The
third
gap
is
to
support
a
different
workload
in
common
drop
of
crd,
to
reduce
the
complexity,
complexity
and
material
effort.
The
next
one
is
dynamic:
resource
scheduling,
dynamic
resource
sharing
between
the
different
attendant
tenants,
such
as
queue.
The
last
gap
is
the
board
performance,
most
of
the
intelligent
workload
required
hazard,
put,
for
example,
without
a
requirement
to
dispatch
eight
thousand
poles
per
second
for
spark
drop.
A
Here
is
the
overview
of
volcano
project.
Volcano
includes
several
components
for
multi-cloud
scenario.
We
have
a
federation
sub-project
in
pipeline
to
balance
the
results
between
between
the
different
cluster
in
in
each
individual
cluster.
We
have
introduced
several
series,
such
as
the
job
for
common
batch
workload
and
a
queue
for
resource
sharing,
and
the
controller
will
help
to
manage
the
lifecycle
of
this
crds.
A
Currently
there
are
more
than
contributors
in
the
community
and
more
than
50
enterprise
users
adopt
a
volcano
in
the
in
their
product
environment.
We
are
going
to
have
a
release
every
three
months
and
the
last
latest
release
is
the
1.5
and
we
already
have
there.
There
is
some
example
of
of
users
such
as
we
already
use
our
volcano
in
huawei
cloud
in
our
product
environment,
and
we
also
work
with
aws
and
some
other
team
to
build
to
build
volcano
in
their
products.
A
In
batch,
as
we
know
in
batch
system,
there
are
several
concepts
which
are
important
to
the
high
level
design.
The
first
concept
is
the
is
the
job
for
batch
decision
is
usually
our
common
job
specification
for
all
kinds
of
workload
such
as
web
service,
ai
metadata
and
so
on.
So
in
cloud
native
ecosystem,
it's
required
to
introduce
multiple
poll
templates
and
the
thundering
fun
green
error
handling.
A
The
second
one
is
about
tennis.
Traditional
batch
systems
introduce
a
user
as
tenants
and
kubernetes
use
the
namespace
as
tenants
based
on
my
understand
and
to
avoid
system
overload.
Resource
limits
is
required
to
the
batch
system.
For
example,
the
slums
support
the
job,
kills
quality
of
service
and
kubernetes
support
quota
to
control
the
results.
A
A
A
A
Here
is
the
npi
example
mpl
wrong
and
the
worker
are
using
different
port
template
with
a
different
command
line.
Volcano
also
supports
fun.
Green
lifecycle
measurement,
for
example,
restart
the
whole
job
when
when
one
of
task
was
failed,
volcano
will
also
build
the
job.
Plugin
in
volcano
is
to
help
users
to
do
a
customer,
customized
enhancement
for
the
to
the
job.
For
example,
ssh
plugin
is
used
to
configure
accession
access
automatically
for
mpi
drop.
So
we
don't
you.
A
A
A
A
A
The
scheduler
will
reclaim
two
cpus
from
q1
based
on
weight
of
them,
so
the
new
job
in
q2
gathered
two
cpu
to
run.
In
addition,
we
support
a
guaranteed
enqueue
to
reserve
results
for
high
priority
jobs
and
support
the
capacity
in
queue
to
avoid
system
overload
next,
okay,
okay.
Next,
the
next
one
is
about
fair
share.
Fair
share
is
a
common
requirement
for
elastic
or
streaming
job
like
spark.
However,
in
kubernetes
the
small
part
submitted
some
more
post
possible
possibility
to
get
more
resources.
A
A
The
small
job
may
get
to
start
starving
without
version
scheduling,
volcano
ensure,
big
drop
and
small
drop
gather
result
results
fairly
with
drf
algorithm,
but
only
job
fair
share
is
not
enough.
Suppose
one
namespace
or
submit
a
loss
of
job
to
the
queue
compared
to
the
other
namespace.
It
would
possibly
occupy
the
most
of
the
queue
results.
A
A
A
A
A
Today
I
machine
learning
workload
had
a
higher
demand
for
gpu
compared
to
the
traditional
workloads.
Gpu
is
a
as
a
expensive
result.
How
to
improve,
improve
the
gpu
utilization,
is
a
hot
topic
and
and
is
important.
You
have
training,
can
dynamically
adjust
the
number
of
instances
involved
in
the
training,
greatly
improving
the
utilization
of
gpu
results,
especially
on
the
public
cloud.
It
can
be
worked
with
a
sport
instance
to
get
a
lower
cost
and
improve
the
training
efficiency.
A
Firstly,
let's
see
what
elastic
drop
is
like
the
left
picture
show
show
a
volcano
drop,
the
mean
available
infer
the
job
has
five
posts
at
least,
and
the
replica
infrared
input
to
the
job
has
a
temples.
At
most,
the
job
gather
running
will
fail
posts
allocated
and
then
the
job
will
get
more
posts
if
they're,
if
there
are
enough
gpu
results
for
them
to
run.
Here
is
another
scenario.
A
As
you
know,
the
influence
the
service
always
have
a
large
gpu
utilization
compared
to
the
training
workload,
as
people
tend
to
call
call
located
in
inference
service
with
the
last
training
workload
to
improve
their
utilization
in
the
right
in
red
pictures.
So
the
influence
job,
the
influence
job
2
with
the
high
priority
with
high
profit,
will
prompt
elastic,
pulled
from
the
training
drop.
One.
A
Another
another
feature
is
about
the
task
top
logic,
and
I
will
wear
the
schedule
this.
This
page
shows
how
the
task
top
logic
and
io
aware
scheduling,
have
the
distribu
distributing
the
training
for
some
gpu
training
case
that
the
data
exchange
between
the
tasks
costs
a
lot
of
time
and
become
the
bottom
night
of
training.
If
the
time
of
the
data
exchange
could
be
reduced,
the
training
performance
can
be
improved.
A
A
There
are
three
nodes
in
the
testing
cluster
and
we
submit
three
training
jobs
with
two
ps
and
four
worker
posts.
We
get
three
different
placements
at
first.
The
results
are
random.
With
the
default
default
scheduler.
As
we
know,
the
group
c
is
the
best
brightest
that
we
want
with
the
task
top
logic
scheduling.
We
are
able
to
get
a
stable
result
as
a
group
c.
As
far
as
I
know,
some
users
are
using
authentic
and
anti-inflammatory
features
to
achieve
this
goal.
Wireless
complexity,
increased
cluster
scalability
and
the
profit
to
performance
is
not
good
enough.
A
A
A
In
the
left
feature
at
the
moment
of
time,
one
t1
users
submit
a
big
drop
sub
me.
A
big
drop
drop
one
with
a
gun
and
a
small
job
job
too.
The
smart
job
get
allocated
and
the
big
job
keep
hunting
due
to
you
due
to
not
enough
results
as
a
more
open
time
too,
a
new
smart
job
three
was
submitted
and
get
a
locket.
The
big
job
keep
pending
because
there's
still
not
enough
results
with
the
time
with
time
go
on
the
big
job
keep
starving.
A
If
the
release
results
always
can
understand,
certified
skunk
and
use
the
same
in
the
small
small
drop,
the
srs
scheduling
a
law
to
configure
the
job
so
that
it
is
completed
it's
completed
on
the
time
and
reduce
the
risk
of
mis
missing
deadline
necessary,
wait.
Time
is
the
maximum
time
that
one
job
some
should
stay
in
pending.
When
sra
wait
time
reached
sra
plan,
he
moved
the
pending
job
to
the
united
states
and
started
to
reserve
resources
for
the
job
until
the
jobs
requirement
is
satisfied.
A
The
next
one,
the
next
important
feature
is
about
newmar
feature
for
con
for
computing
computer
in
intensive
and
performance
demand
at
the
job
user
would
want
to
consider
user
exclusive
cpu
results
which
can
reduce
the
laws
to
back
contact.
The
switch
foot
job
running
on
the
pneuma
based
process
is
also
better
to
have
the
cpu
applying
to
the
port
on
the
sim
newman
node,
which
reduce
the
communicating
loss
and
improve
the
improved
training.
A
Currently,
topological
management
and
cpu
management
provide
a
topological
scheduling
as
a
node,
so
the
problem
will
improve
the
scheduling.
The
slope
in
the
scheduler
cannot
perform
the
top
logic
scheduling
it's
not
aware
of
the
top
logic
and
the
cpu
allocation
on
the
node.
Therefore,
the
container
may
be
refused
by
the
corporate,
because
I
know
the
result
is
less
than
the
top
logic
minor
requirement.
A
A
A
As
the
later,
an
announcement
spark
operator
provide
another
way
to
have
wrong
to
have
running
spark
on
kubernetes
as
well.
However,
in
a
long
time,
sparrow
unknown
had
the
lack
of
batch
scheduling
batch
batch
scheduling
features
later
over
the
last
year,
we
started
to
work
with
the
spark
contributor
to
support
customer
budget
scheduling
for
spark
spark
with
scale
spark
with
volcano,
provides
the
best
scheduling
capability
like
job
priority,
qr,
fair
share
resource
resolutions
and
so
on.
It
will
be
released
in
spark
3.3
version.
A
Another
important
feature
is
about
collocation.
Multiple
server
being
deployed
together
has
become
common
in
recent
years.
Online
web
service
and
offline
analysis
service
has
their
different
features.
Online
services
requires
new
results,
but
sorry
online
service
require
fuel
results,
but
they
are
very
demanding
for
the
latency,
such
as
recommendation
and
the
other
service.
A
Offline
analysis
service
is
a
computing
intensive,
require
require
more
results,
but
the
traffic
stays
stable
and
they
don't
and
they
don't
demand
a
faster
response
like
online
service.
If
online
service
and
offline
workload
can
be
deployed
in
a
in
a
hybrid
way,
the
resource
can
be
fully
used.
Another
pinpoint
is
our
subscription
and
our
subtraction
of
the
results
users
would
like
to
ensure
their
scalability
they
so
they
tend
to
apply
the
exclusive
results
proposed.
A
However,
kubernetes
scheduling
relay
on
the
stacked
port
requirement
to
in
the
scheduling
this
leads
to
high
resource
allocation
range
and
low
resource
usage.
Typically,
the
average
credit
cpu
resource
usage
could
be
less
than
15
percent.
This
is
a
huge
risk
of
resource
to
resolve
this
pinpoint
volcano
has
done
some
research
done.
Some
research
investigations,
for
example,
based
on
the
monitoring
and
usage
volcano,
can
perform
dynamic
scheduling.
A
A
A
Taser
scheduling
take
scheduler
for
the
example,
we
improved
the
scheduling
throughput
to
one
point,
one
one
point:
five
thousand
pounds
per
second,
but
adopt
equivalent
cash
batch
bonding
and
sick
bonding,
and
they
also
depends
on
the
some
enhancement
to
the
kubernetes
to
the
cube
api
server.
Yeah
here
is
some
user
case
from
from
volcano.
A
Is
the
one
of
turbo's
social
media
and
e-commerce
company
in
china?
Many
people
use
their
applications
in
mobile.
They
have
100
million
active
users
per
month
and
the
main
workload
they
are
running
to
provide
recommendations
for
the
end
user.
They
need
to
refresh
the
model
every
several
minutes
and
also
they
have
some
online
service
to
do
a
reaction
when
users
refresh
their
their
nose.
The
challenge
is:
they
have
a
large
cluster
with
nearly
a
1000
nodes
and
the
model
have
one
100
billion
parameters.
A
When
jobs
have
hundreds
of
ps
and
worker
posts,
I
actually
they
want
to
pass
the
resource
allocation.
The
user
adopts
the
task,
top
logic
scheduling
and
we
have
20
performance
improvement.
They
also
use
volcano
sla
base,
based
scattering
to
prevent
a
large
to
prevent
large
drops
from
the
starvation.
A
A
A
For
the
public
adoptions,
we
we
do
get
a
lot
of
user
options,
especially
for
the
people
are
running
in
running
the
ai
big
data
live
sentence.
Science
transcoding
workload
on
kubernetes
here
are
part
of
adoption
using
volcanoes
product
environment
from
the
code
diversity
university.
In
recently,
we
can
find
that
we
got
a
good,
devastating
community
development.
A
Here
is
the
release
journey
of
volcano.
We
have
released
more
than
16
major
version
since
the
things
the
way
the
project
start
at
the
early
stage,
we
developed
a
site
of
scheduling
policy
to
support
the
patchwork
load
then
integrate
with
the
ecosystem,
such
as
the
kubernetes,
sorry
with
the
flow
and
the
spark
operators
and
algo,
and
so
on
later
we
found
that
there
are
lots
of
gap
on
the
job
on
the
turbo
management.
So
we
support
quite
a
lot
of
time
to
enhance
the
job
management
to
to
upstream.