►
Description
Volcano – Cloud Native Batch System for AI, BigData and HPC - William(LeiBo) Wang, Huawei Cloud Computing Co., Ltd
Volcano is a cloud native batch system which is also the first batch computing project in CNCF. The major use cases are in the field of high-performance computing (HPC), such as big data, AI, Gene computing. Volcano offers job based fair-share, priority, preemption, reclaim, queue management abilities which are important for HPC users. It has integrated with computing ecosystem like spark-operator, fink-operator, kubeflow, Cromwell in big data, AI and HPC computing domains. This year Volcano is also being integrated to spark with it's custom batch scheduler natively. And many new features are being developed by contributors. f.g. co-location, elastic training, vGPU, throughput optimization and multi-cluster scheduling for HPC users.
The community has helped more than 50 users to deploy Volcano in their production environments around the world since it is open-sourced in 2019. William(Leibo) Wang who is the tech lead of Volcano community will present the latest features, use cases, progress, roadmap and best practices. He will also show how to accelerate AI training, serving, big data analysis and how to improve cluster utilization based on Volcano and other cloud native projects for users.
A
Hello:
everyone
I'm
william
from
huawei
cloud,
it's
a
great
honor
for
me
to
participate
in
this
event
today.
My
topic
is
volcano
clonative,
computing
system
for
ai,
big
data
and
hpc.
A
briefly
introduction
upon
me:
I'm
tech,
lead
or
volcano
community,
and
my
work
is
focusing
on
batch
scheduling,
performance.
Ai,
big
data
acceleration.
A
Here
is
the
agenda
today,
I'll
first
introduce
the
trend
of
batch
computing
and
challenges
we
are
facing,
and
then
I
like
to
talk
about
clonitive
project
of
kindle
later,
I'm
gonna
show
you
how
volcano
addresses
these
challenges
in
quantities
then
I'll
show
several
use
cases
and
that
how
user
built
their
platform
with
volcano?
Finally,
I
will
introduce
the
community
first.
Let
me
start
with
the
batch
computing
development
journey.
There
are
three
different
lines
showing
three
different
domains.
A
A
People
were
also
talking
about
how
to
better
deploy,
hpc
application
on
coordinates
in
the
big
data
domain.
As
you
know,
spiral
kind
of
link
have
already
of
it
officially
support
for
quantities
as
in
a
domain.
The
ef
framework,
such
as
tensorflow
copy,
the
accumulator
most
of
them,
are
running
on
top
of
kubernetes
right
now,
more
and
more
users
would
like
to
migrate
their
batch
workload
furnace
to
unify
the
platform
and
see
this
would
bring
two
major
benefits.
First,
multiple
resource
managers
can
be
unified.
Users
do
not
need
to
maintain
er
or
methods
anymore.
A
A
A
The
second
thing
is:
kubernetes
has
limited
scheduling
policy,
especially
compared
to
those
systems
like
er
or
slurm
of
sg
sort
of
systems,
for
example,
the
job
property
preemption
fair
share,
reservation
topology
are
not
supported
in
coordinates
the
community.
The
computing
framework
in
different
domains
are
turning
to
run
better
on
kubernetes,
develop.
They
developed
their
operators
such
as
tfl,
printers
back
operator,
etc.
A
A
Kubernetes
can
isolate
resources
by
namespace,
but
lack
of
dynamic
resource
sharing.
Kubernetes
has
no
concept
of
q.
The
results
cannot
be
used
as
an
effective
way.
Large
challenge
is
the
heterogeneous
computing.
Many
companies
have
their
accelerating
hardware,
such
as
gpu
tpu,
and
you
how
to
schedule
these
resources
uniformly
and
provide
the
best
resource
allocation
called
the
natively.
A
A
So
that's
why
we
started
working
on
the
volcano
project.
Here
is
the
volcano
overall
architecture.
We
are
trying
to
provide
more
reached
rich
advanced
scheduling
policies,
especially
to
help
reduce
batch
workload
better
running
on
top
of
kubernetes,
as
the
figure
shows
volcano
is
not
just
a
scheduler.
It
has
job
and
job
controller
to
support
the
enhanced
job
lifecycle
management.
It
has
queue
to
offer
resource
planning
sharing
for
multi-tenants
for
the
analyzing
hardware,
we
are
look.
A
We
are
working
on
to
support
heterogeneous
device
to
better
management,
manages
the
s86
arm,
mpu,
etc,
and
working
with
kubernetes
to
improve
the
performance
for
the
back
batch
workload
from
the
graph.
We
can
see
volcano
engaged
deeply
with
the
upstream
computing
frameworks.
So
far
we
have
supported
almost
all
of
the
popular
mainstream
framework.
A
Volcano
become
open
sourced
at
june
2019
and
donated
to
cncf
in
april
2020
as
a
sandbox
project.
Volcano
released
a
feature
version
every
three
months,
and
currently
there
are
more
than
350
contributors
in
community.
Volcano
health
had
moved
to
incubator
in
april
2022
and
there's
more
than
50
enterprise
users
adopt
volcano
in
their
production
environment.
A
Before
we
get
into
the
detail
of
kino,
let's
look
at
some
very
important
concepts.
First
is
a
job.
What
do
what
does
the
multiple
time
port
template
mean?
Let's
take
mpijob
as
an
example,
people
expect
to
use
different
image
for
master
pod
and
work
port
in
each
npi
job.
The
job
needs
to
offer
this
kind
of
ability
to
allow
different
configurations
for
different
role
of
code
in
each
job.
A
The
second
is
the
username
space
and
the
resource
quota
in
kubernetes
namespace
is
used
to
isolate
resource.
It
is
mapped
or
regarded
as
user,
and
the
resource
quota
infers
to
up
up
limit
results
that
a
user
is
able
to
use
at
most
and
the
queue
is
widely
used
for
sharing
resources
between
different
users
or
resource
pers.
A
A
A
Let's
move
to
the
job
management
in
volcano
and
see
how
to
address
these
challenges
we
mentioned
previously.
Multiple
poll
template
is
supported
in
volcano
with,
with
this
feature,
volcano
job
is
able
to
unify
the
support,
most
mainstream
computing
frameworks,
lack
of
mpi
tensorflow,
her
word
patos,
etc.
A
Volcano
job
also
provides
an
extendable
job
plugin
to
allow
users
to
find
their
custom
behavior
and
provides
several
built-in
plugins
by
default.
Also,
the
volcano
job
and
scheduler
can
coordinate
to
do
more
of
transition
for
batch
workload.
For
example,
the
job
dependency
scheduling
did
aware
scheduling
and
to
improve
the
improved.
The
java
performance.
A
A
The
queue
is
mainly
used
to
share
resources
between
multi-tenants
and
resource
pro.
For
example,
one
queue
can
map
to
a
group
in
your
company
and
also
you
can
match
a
specified
kind
of
resources
like
gpu,
for
this
queue
in
the
cluster
and
also
your
user
is
allowed
to
configure
different
policy
for
different
queue.
It's
very
useful.
A
After
we
got
to
get
to
know
the
q
concept
in
volcano,
let's,
let's
talk
about
some
scenarios.
The
first
scenario
is,
if
you
have
two
teams,
how
to
share
resources
between
between
them
with
you.
Here
is
an
example.
There
are
two
kills
q1
and
q2,
which
are
mapping
to
the
r2
teams,
and
their
weight
ratio
is
two
to
one.
There
are
six
cpus
in
cluster
and
there
are
six
posts
in
q.
A
A
Next
fair
share
is
a
common
requirement
for
elastic
or
streaming
jobs
like
spark.
However,
in
kubernetes
small
post
submitted
the
more
possibility
to
get
more
resources.
There's
no
fair
share.
Volcano
profit
is
the
fashion
between
jobs
and
name
service.
This
from
the
graph,
you
can
see
the
user,
1
and
user
2
submit
small
job
and
big
job
to
the
sim
queue.
Smart
job
might
have
met,
get
starving
without
faster
scheduling.
A
A
Nowadays,
machine
learning
workload
has
hair
demand
for
gpu
compared
to
traditional
workloads.
Gpu
is
a
precious
resource.
How
to
improve
the
gpu
utilization
is
hard
part
hot
topic
and
has
great
value.
Elastic
elastic
training
can
dynamically
adjust
the
number
of
instances
involved
in
the
training,
greatly
improving
the
utilization
of
the
gpu
resources,
especially
on
the
public
cloud.
It
can
work
with
spot
instance
to
get
a
lower
cost
and
improve
the
training
efficiency.
A
A
The
job
has
templed
at
most,
the
job
gets
running
where
failure
to
get
allocated,
and
then
the
job
will
extend
more
because,
if
there's
more
free,
gpu
resources,
clearly
another
scenario,
as
you
know,
the
inference
inference
service
always
have
a
lower
gpu
utilization
compared
to
training
workload,
as
people
tends
to
co-locate
the
inference
service
with
elastic
training
workload
to
improve
the
utilization.
The
right
figure
shows
an
example:
the
infra
the
influence
job
2
with
hair
priority,
preempt
elastic
pulled
from
training
job
1
to
ensure
it's
sle.
A
This
page
shows
how
the
task
top
topology
and
io
aware
helps
the
distributing
training
for
some
gpu
training
cases.
The
data
exchange
between
tasks
costs
lots
of
time
and
become
the
botnet
of
the
training.
If
the
time
or
did,
exchange
could
be
reduced,
the
training
performance
can
be
improved
stock.
The
task
top
lodge
would
schedule
ps
and
work
on
the
sim
node
to
reduce
the
data
exchange.
Latency,
ps
and
worker
can
use
the
host
internal
network,
which
is
better
than
the
network
across
hosts.
A
A
As
you
know,
the
group
c
is
the
best
placement
that
we
want
with
the
task
topological
scheduling
we
are
able
to
get
stable
result
as
group
c.
As
far
as
I
know,
some
users
use
the
affinity
and
anti-affinity
features
to
achieve
this
goal.
However,
the
complexity
increases
with
the
cluster
skill
and
the
performance,
not
so
good.
We
also
do
some
research
on
the
io
aware
scheduling
the
io
awareness
scheduling
can
minimize
the
minimum
last
the
max
data
transfer
latency
and
get
even
better
performance.
A
A
In
a
real
production,
cluster
user
often
submits
multiple
kind
of
job
jobs,
smart
jobs,
big
jobs,
how
to
avoid
the
big
job
or
small
job
getting
starving
is
very
important.
Left
figure
shows
an
example
at
the
moment
of
t1
users
submit
a
big
job
with
gun
and
smart
job
too.
The
smart
job
get
allocated
and
the
big
job
keep
pending
due
to
insufficient
resources.
At
the
moment,
t2
a
new
smart
job
3
was
submitted
and
got
allocated
the
big
job
1
keep
pending
due
to
insufficient
resources
with
the
time
goes
on.
A
Spark
started
to
provide
support
for
kubernetes
in
2.3
version
in
2017
and
later
spa
computer
provides
another
way
to
help
run
spark
on
top
of
kubernetes
as
well.
However,
in
a
long
time,
spartan
companies
lack
of
bash
scheduling,
abilities
late
last
year,
we
started
to
work
with
spark
contributors
to
support
custom
batch
schedules
for
spark
supply
with
volcano
provides
the
batch
scheduling,
abilities
like
job
priority
queue,
fair
share,
resource
reservation,
etc.
A
As
the
amount
of
data
continues
to
grow
and
the
complexity
of
business
increases
user
require
communities
to
support
larger
skill
cluster.
We
also
spend
a
lot
of
effort
to
support
ten
thousand
news.
100
million
containers
in
one
cluster
by
optimizing,
the
container
network,
scheduling,
container
engine
etcd
and
hp
api
server.
The
red
part
lists
some
specif
specific
approaches.
A
A
Now,
let's
take
a
look
at
how
users
use
volcano
to
build
their
ai
platform
is
one
of
the
top
social
media
and
economy
inc
e-commerce
company
in
china.
Many
people
use
their
apps
in
mobile
phone.
They
have
100
million
active
users
per
month
and
the
main
workloads
they
are
running
is
to
provide
the
recommendation
for
end
users.
They
need
to
refresh
modules
every
several
minutes
and
they
have
some
online
service
to
do
immediately
reaction
when
users
refresh
their
their
notes.
A
The
challenge
is,
they
have
large
large
clusters
with
nearly
solid
nose
and
the
model
have
100
billion
parameters.
One
job
has
hundreds
of
peers
and
workers.
Actually,
they
want
the
best
resource
allocation,
the
user
adopt
the
task,
topology
scheduling
and
again
20
performance
increased,
and
she
also
used
the
volcano
sle,
based
on
based
scheduling,
to
prevent
the
large
job
from
starving.
A
This
is
another
case
of
financial
selector
sector
user.
They
initially
use
the
yard
to
sketch
schedule
batch
jobs
as
the
com
con
company
grows.
The
environmental
policy
also
changed.
Different
research
team
requires
container
deployment
to
avoid
the
environment,
confliction
and
dependency.
However,
the
colonies
lacks
fair
share,
scheduling
between
multiple
teams.
Also,
they
have
the
requirement
of
using
different
frameworks
such
as
tensorflow
pathage
mpi.
A
That
requires
to
install
different
kinds
of
operators
in
quantities
which
lead
which
lead
to
hair
cost
maintenance
and
learning.
The
user
looks
for
the
solution
and
found
volcano
can
satisfy
their
requirement
and
offer
diverse
scheduling,
abilities
and,
what's
more,
they
could
use
volcano
to
work
in
a
job
to
unify
a
tensorflow
pattern.
Mpl
preacher.
Eventually
they
decided
to
migrate
the
from
er
to
kurenitis
and
the
volcano.
A
A
For
the
public
adoption
we
do
get
a
lot
of
user
adoption,
especially
for
people
are
running.
The
ai
big
data.
Gene
transcoding
workload
on
kubernetes
here
are
some
parts
of
the
adapters
using
volcano
in
production
environment
for
the
code
diversity
in
recent
one
year
you
can
find
out
that
we
get
good
diversity
in
community
development.