►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
The Evolution and Innovation of CubeFS - Leon Chang, Oppo
Speakers: Leon Chang
CubeFS is a cloud native distributed storage solution for big data processing, machine learning，data sharing and protection, etc. CubeFS is compatible with three access protocols(S3/POSIX/HDFS)， it provides multiple data redundancy strategies ，such as replication and erasure-coding, it aslo support hybrid-cloud acceleration。 In this talk, there will be several parts to have an introduction of Cubefs and deep-dive discussions to talk about the technical details, the recent release, and future plans. CubeFS was accepted as an incubating project by the Cloud Native Computing Foundation in July 2022
A
A
At
the
beginning
of
this
year,
we
released
version
3,
which
is
a
significant
release,
support
usual
code
report
to
reduce
the
cost
in
the
scenarios
as
the
trick
and
hradp
online
for
one
year
in
our
previous
storage
platform,
they
are
honored
that
the
Cuba
has
become
an
incubation
project
future
and
in
August
we
will
release
the
Virgin.
We
already
released
the
Virgin
3.1,
which
is
supposed
to
have
read
the
cloud
acceleration
and
Qs
flow
control,
components.
A
In
terms
of
Open
Source
Community
groups,
We
compare
the
dates
before
and
after
joining
the
sandbox,
the
increasing
numbers
indicates
that
cubas
increase
the
Impractical
applications.
Many
software
Engineers
are
willing
to
join
and
contribute
to
the
product
and
many
end
users
likely
to
make
tests
or
use
in
production
environment.
A
A
A
A
Storing
small
files
centrally
in
storage
nodes
to
stream
moving
the
small
files
to
one
big
file,
which
can
improve
the
sort
of
output
and
performance,
especially
for
FTD
HDD
disk,
need
to
utilize
the
pawn
hole
interface
of
the
Linux
to
synchronize.
The
freeze
disk
space
occupied
by
the
latest
note
files,
which
greatly
simplifies
the
engineering
work
of
dealing
with
small
validations.
A
A
A
In
contrast,
each
message-
node
or
cube
advice
catches,
all
the
film
Biometrics
in
memory
to
avoid
the
expensive
disk
IO
input
output.
Second,
the
override
in
Cuba
files
is
in
place
which
does
not
need
to
update
the
file
message.
In
contrast,
the
all
right,
except
we
really
need
to
work
through
multiple
keys
and
only
up
to
the
date.
Automated
have
been
persisted
and
synchronized.
The
commit
message
can
be
returned
to
client.
A
Football
has
stores
all
file
metrics
in
memory
to
avoid
the
expensive
disk
input
output
during
salary
and
in
the
case
of
small
forward
right
so
Cuba
files.
Clients
are
no
need
to
ask
the
resource
manager
for
new
exams
instead
extends
to
the
rest
request
to
the
this
new
directory,
which
further
reduce
the
network
or
highs.
A
A
We
put
Cube
office
in
three
parts
in
Practical
AI
platform
easily
and
Spark
remote
Shuffle
service
in
AI
platform
as
an
online
storage
or
AI
platform.
Cube
Wi-Fi
shows
Source
serious
opening
points
such
as
massive
small
houses,
storage,
millions
of
directories
and
voltage
access,
emotional
learning
scenarios
of
region
of
a
various
original
original
copyright
model.
The
tutorial
interface
and
training
supposed
to
advertising
games,
recommendations,
civil
image,
processing
and
other
online
business
in
the
Deep
link.
There
are
many
problems
should
be
solved.
A
The
main
problem
is
about
80
advice,
the
message
subsystem
name
node
has
Limits
The
Limited
the
scalabilities,
and
this
skill
of
the
signals
singles
cluster
is
limited.
Second,
is
the
storage
and
the
Computing
are
integrated,
and
the
operator
operatability
is
poor.
Second,
is:
is
pro
cluster
stability
for
its
historic
cost
and
not
friendly
to
small
files?
A
A
This
page
is
in
case
study
of
project
Solutions
because
we
need
to
know
what
it
did
like.
A
data
lake
is
a
centralized
Miss
poetry,
so
that
allows
you
to
store
all
your
structures
and
unstructured
it
in
at
any
skill.
You
can
store
your
data,
as
is
without
having
to
First
transfer
the
date
and
when
different
types
of
analytics
from
dashboards
and
visualizations
to
Big,
digits,
processing,
real-time
analytics
and
machine
learning
to
get
better
decisions
just
about
Italy
can
we
watch
the
requirements
for
two
by
five?
First?
Is
we
need
to
provide?
A
A
A
A
This
is
a
significant
subsystem
of
Cuba
cuba5s.
The
left
figure
shows
the
components
of
the
rule,
individual
coding,
sound
system
and
the
flow
of
date.
The
red
finger
shows
the
distribution
of
data
in
different,
easy
and
easy
modes.
To
be
specific,
we
can
use
real
Solomon
codes
33,
which
contains
three
digit
fragments
and
three
code
fragments
all
fragments
are
stored,
stored
on
different
storage.
A
To
note
that
the
single
easy
we
can
also
use
RS
104,
which
has
air
durabilities
and
lower
cost
and
can
be
used
after
configuration,
we
use
our
is
610
and
rs159
with
actively
under
280
and
3G.
More
fragments
are
usually
stored
on
different
nodes
on
in
different,
easy,
always
a
location.
We
all
we
all
fragment
your
funnies
is
entirely
lost.
We
can
still
recover
the
data
from
the
fragment
to
another
easy.
A
Additionally,
we
Implement
local
reconstruction
code
lrc
with
reference
to
a
paper
individual
coding
in
Windows,
later
storage
lrc
will
ask
to
reconstruct
of
language
fragments
with
lights.
Azure
coding,
fragments,
but
most
important
benefit
to
lrc
is
that
it
reduces
bandwidth
and
iOS
input
output,
inquiries
required
for
data
recovery
so
as
to
maintain
a
significantly
lower
storage
or
high.
A
There
were
coding
on
something
on
how
five
advantages:
large
cluster
capability,
high
durability,
lower
TCO,
reduce
redundancy
from
3
to
1.3
or
less
for
multi-az
deployment
by
multiple
specification,
configurable
initial
coding
model,
of
course,
kubernetes
directly
use
online
EC.
This
avoids
data
migration
and
sales
storage,
space.
A
A
They
made
money
optimizations
to
a
visual
code
coding
subsystem
under
the
condition
of
the
usual
code
code
is
accountable.
Certain
red
failures
are
loud
and
background.
Synchronous
patching
is
performed
for
the
field
data
blocks
through
this
call
through
through
this
kernel
mechanisms,
the
problem
of
the
tail
support
can
be
effectively
resolved
for
small
small
files.
Using
this
code
to
replace
time
codes
introducing
redundant
date
blocks
for
EC
encoding
can
effectively
reduce
the
number
of
input,
approach
and
inputs
read
performance,
that's
a
tiny
file,
storage
of
replica
subsystem.
A
Multi-Copy
implements
is
simple:
data,
durability
is
average
results,
the
condition
is
low
and
the
storage
costs
are
high.
Visual
code
are
more
complex
to
environment,
with
Heritage
durability,
hair
resource
utilization
and
lower
story,
cost
comprehensively
evaluated.
Redundancy
strategies
for
multiple
Dimensions,
such
as
business,
did
skill
access
model
durability
and
costing
requirements
taking
open
mobile
phone
Cloud
available
business
as
an
example
which
suppose
hundreds
of
millions
of
users
to
access,
and
this
volume
is
large
and
the
cost
of
demand
is
strong.
A
At
the
present,
a
lower
cause
error
codeine
engine
have
has
been
fully
used
in
Oppo
next
I
want
to
introduce
the
hybrid
Cloud
acceleration
section,
hybrid
Cloud,
AI,
training
scenarios
that
is
most
of
the
GPU
Computing
comp
power
is
in
self-build,
Cloud
machine
Rule
and
a
small
part
of
the
computing
power
will
use
the
public
Cloud
GPU.
Then
the
self-build
cloud
GPU
resources
can't
miss
the
user.
Certain
computer
power
needs.
A
The
task
can
be
grammatically
scheduled
to
public
Cloud.
To
Advantage
of
this
scheme
is
that
name
of
the
subfield
Cloud
gpus
can
be
controlled,
which
can
only
meet
the
user's
normal
training
needs,
but
also
meets
the
user.
Sudden
increment
mental
needs
to
self-build
clouds
only
need
to
maintain
a
thermal
computing
power
level
which
can
effectively
reduce
the
all
TCO.
However,
this
prevent
brings
new
challenges
to
underlying
storage
in
the
hybrid
Cloud
scenario
will
often
face
challenges
of
this
transmission
and
storage
between
public
cloud
and
private
cloud.
A
First
is
performance
problem
in
storage
during
cross
Regional
due
to
the
network
bandwidth
daily,
nearly
2
microseconds,
the
excess
storage
performance
of
the
public
Cloud
GPU
host
drops
primitively
compared
with
open
private
clouds.
Gpu
host
accessing
storage,
simple,
is
processed
by
single
GPU
card
for
e-port,
photo
nearly
2
to
4
times
different,
the
out
bottlene
likely
to
low
GPU
card
utilization,
which
is
a
great
waste
of
task,
training,
time
and
GPU
resources.
A
Second,
is
the
high
cost
of
data
migration
as
the
initial
stage,
the
task
approach
dispatcher
to
public
Cloud
GPU,
which
mounts
the
public
Cloud
file
storage,
require
recurring
users
to
migrate.
The
cube
effect
stage
to
Google
Cloud
to
the
public
cloud
storage,
which
is
a
high
cost
for
users.
Three
is
the
data
security
issue
of
public
on
public
cloud
or
catch
acceleration
mechanism
is
designed
on
the
client
side
to
achieve
local
caching
to
remote
training
digitals
to
GPU
Computing.
A
You
know
to
reduce
the
current
Cube,
has
methods
latency
to
microsecond
level
and
reduce
the
data
lendency
to
Android
micro
second
level
is
the
details
on
how
the
design
and
the
development
cache
first.
The
big
trip
low
is
deployment
architecture.
The
data
acceleration
mode
is
deployed
in
the
public
Cloud
GPU
host
in
the
demand
side
site
mode,
the
cache
disk
is
mounted
to
task
port
and
the
block
output
is
and
in
the
way
of
host
pass,
to
reduce
the
request.
Uniform
storage
provided
cloud.
A
There
are
two
types
of
requests.
First
is
manage.
Second,
is
date,
so
we
build
the
mechanism
to
catch
both
of
them.
First
is
new
to
adapt
the
flexibility
flexibly
to
different
types
of
final
tasks
due
to
cash
explosion
term
inside
too
long.
So
memory
can't
be
released
in
time
if
it
is
set
to
short
to
catch
heat
which
will
be
affected.
Second
need
to
consider
the
scope
of
calcium,
based
on
the
specified
directory,
build
the
sketch
of
the
entry
and
I
node
so
as
to
manage
the
cash
slope
and
left
cycle
Mouse
flexibly.
A
For
that,
for
this
catch
had
two
key
factors
versus
builds
div
cash
service.
You
need
to
consider
the
resource
limitation.
The
generality
second
need
to
build
index
management
and
data
management.
Therefore,
the
Deep
skill
model
blockage
is
designed
as
a
live
resource
module
which
mainly
the
police,
sensory
functions,
cache,
read,
write,
function,
catch
date,
index
measurement
and
cache
disk
space
management.
A
The
picture
is
about
the
test.
Comparison
result.
It
can
be
seen
from
the
date
in
the
finger
that
up
to
the
public,
Cloud
GPU
starts
the
cube
effect
client
to
accelerate
the
performance
distribution
of
result
18
in
dashboarder
worker
one,
since
he
increased
by
300,
CC
and
114,
and
it's
not
increased
by
130
and
80
respectively
at
work,
13
and
24.,
so
effective
is
very
obvious.
A
A
The
patch
history
has
a
very
important
impact
on
the
overall
overall
performance
also
to
catch
this
cap.
Capacity
of
the
of
a
single
GPU
host
is
limited.
It
is
limited
by
the
number
of
GPU
cards
and
GPU
resources
of
a
single
machine,
and
the
number
of
the
chain
tasks
running
at
the
same
time
is
not
large,
so
to
catch.
Hitter
bridge
is
busy
basically
stable
at
a
high
level.
A
Therefore,
in
our
implementation,
there
is
no
dependency
on
external
module
and
there
are
mainly
two
modifications.
One
is
the
flow
control
module
of
collect
and
the
other
is
global
flow
distribution
management,
module
of
Master
through
periodic
plant
reporting
and
traffic
request,
Master
supports
locations
and
do
more
prior
locations
for
of
buffer
for
clients.
A
A
A
It's
so
Supreme
pumps,
automatic
storage
and,
at
the
same
time,
in
the
multi-user
platform
of
AI
training,
so
amongst
of
thousands
of
people,
may
operate
the
same
battery
teach.
For
example,
the
data
security
and
historical
recovery
have
become
a
demand
point,
which
is
also
so.
This
storage
demand
of
the
open,
Hardware
Key.
A
A
The
color
block
diagram
also
didn't
not
reflects
organ's
ownership,
but
there's
no
need
to
identify
the
version
in
this
node,
and
the
word
is
controlled
by
much.
It
has
several
nice
features.
First
is
create.
Snapshots
in
seconds
is
no
like
snapshot
of
our
memories.
Three
is
no
right.
Templarification
force
is
made,
it
did
with
offspace.
Redundancy
file
is
strong
consistency,
which
one
is
the
index
design
inside
our
match.
A
As
mentioned
earlier,
the
index
will
match
in
is
stored
by
P3.
The
picture
shows
the
measurement
relationship
of
different
version
information
inside
of
the
file.
We
will
be
using
that,
according
to
the
time
order
of
the
workers
and
reading
any
audience
can
get
the
completed.
Storage
share.
The
information
snapshot
is
a
collection
function.
For
example,
q1v
multiple
components
are
involved
to
implement.
A
A
snapshot
operation,
almost
all
key
inter
internally
routines,
need
to
be
updated.
Q2
is
the
central
deletion.
Scenarios
involves
the
visibility,
range
statement
and
operation.
The
flows
and
entries
just
read
the
problem
of
excessive
fragment
tension
of
the
data
caused
by
the
modification
of
data
fragmentation.
A
Okay,
next
page,
we
will
talk
about
the
remote
Shuffle
Solutions.
The
technical
evolution
invicted
Computing
engine
has
always
been
inspirable
from
the
optimization
of
Shuffle.
Whether
it
is
optimization
of
execution
plane
is
about
travel
operations
as
much
as
possible,
all
the
evolution
of
various
Shuffle
mechanisms.
It
is
also
shortened
time
consuming
of
Shuffle
as
much
as
possible.
There
are
two
main
factors
for
Shuffle
to
become
the
key
factors
for
the
efficiency
and
the
stability
stability
of
both
Big
Data
Computing.
A
One
is
fragmentary
and
rights
of
disk
spill
rights
to
disk
multiple
times
and
reduce
only
two
parts
of
the
partition
date,
which
affects
the
efficiency.
Second,
reduce,
read
local,
ditch
on
Max
side,
make
sure
it
requires
map
remote
Network
race,
which
affect
stability
with
the
evolution
of
Shuffle
technology.
The
main
line
is
also
advancing
along.
The
solutions
of
the
above.
A
Two
problems,
for
example,
so
is
the
principle
of
Manchester
is:
is
that
the
math
task
pushes
the
same
partition
due
to
the
remote
the
most
travestyles
and
the
remote
shops
are
more
of
the
data
of
the
same
partition?
Shuffle
uses
a
distributed
file
system
as
a
storage
Base.
Today
we
distributed
storage
technology
is
so
developed.
A
We
don't
need
to
spend
too
much
efforts
of
the
optimizing
storage.
Professional
things
are
hand
over
to
professional
people.
The
main
advantages
of
Cuba
advice
is
first,
reduce
the
complexity
of
sample
system
itself
and
improve
its
stability,
and
certainly
the
distributed
file
system
itself
has
advantages
of
good
stability
load.
Balancing
sorry
sword
is
adapt
to
a
variety
of
the
distributor
system.
Tools,
diversification
and
make
full
use
of
the
advantage
of
different
sets
of
force
is
make
Shuffle
worker
decal
local
storage
capabilities.
A
Let's
introduce
future
flexible
replication
strategy
used
in
shopper,
so
about
figure
is
the
deployment
of
your
couples.
The
client
and
the
substance
are
deployed
independently.
The
volume
finger
shows
the
deployment
of
a
single
copy.
The
client
can
choose
to
mix
deployment
with
data
subsystem
which
will
choose
local
priority
storage
and
will
not
choose
other
nodes.
If
there's
sessions
space
in
replica
options,
we
can
choose
single
replica
two
three
like
at
the
lower
the
number
of
verification,
the
lower
so
or
all
cost.
Let's
focus
on
single
replica
to
introduce
TCO
and
network
traffic.
A
The
lab
page
is
about
the
world
map
of
Cuba
files.
There
were
some
several
important
components
and
the
development
first
is
kubricate,
structured
storage
for
mobile
applications
or
called
device
devices.
Second,
is
a
hdfs
protocol
compatible
direct
to
use
adfi
seamless
to
access
kubernetes
without
any
modification?
A
Three
automaker
name
the
cardboard
middle
students
status
remained
caused
by
the
routines,
be
interrupted
by
something
happened
in
the
distributed
environment
and
there
are
two
main
release
will
release
later.
First
is
the
snapshot
mentioned
before?
Second
is
a
usual
coding
code,
subsystem
reconstruction
reduce
the
count
of
components
depending
on
and
it
will
be
easily
be
deployment
Central.