►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
“KubeGene” a Genome Sequencing Workflow Management Framework - Shenjun Tang, Huawei
In the recent years, exponential growth of genetic data makes it hard for researchers to do analysis on a traditional computers. In this presentation, Tangshengjun will provide an overview of “KubeGene” a bio-genetic management framework built on top of Kubernetes platform for managing TB/PB of data deployed across a cluster with hundreds of nodes. “KubeGene” supports data-processing pipelines for Whole Genome Sequencing (WGS), Whole Exome Sequencing (WES), liquid biopsy, single cell sequencing and other sequencing scenarios with best practices, like Genome Analysis Toolkit (GATK), on the Kubernetes cluster. Furthermore, “KubeGene” can easily deploy a cluster-based data-science platform for data-mining based on the processed data. In conclusion, “KubeGene” provides users with full-stack, easy-to-use, scalable bio-genetic cloud computing solution.
To learn more click here: https://sched.co/FuKj
A
Pave
the
guard
keeper
gene
okay
today
is
the
launching
day
for
the
cuter
genes.
Our
colleague
and
I
would
like
to
share
with
you
about
collision.
I
am
way
past
architects
we
found
if
we
use
container
technologies
to
do
gene
sequencing.
It
is
very
easy
and
very
convenient.
So
we
created
this
crowd
service
for
people,
gene
we're
trying
to
develop
a
ecosystems
for
genome
sequencing
using
the
opportunity.
So
that
is
how
the
name
came
to
the
place.
A
As
you
can
see
here,
this
is
the
agenda
of
my
presentation.
First
of
all
that
you
give
you
background
of
our
industry,
how
we
do
the
gene,
sequencing
introduction
and
the
traditional
solutions
why
we
use
kate
kubernetes
so
and
how
we
use
turbine
artists
should
do
the
sequence
and
why
we
use
turbine
attitude
why
we
use
kubernetes
to
do
the
sequencing,
and
then
we
like
to
talk
about
introduction
of
open
source
first
litigator
introductions.
A
So
after
the
thirteenth
five-year
plan
of
China
and
also
in
the
United
States,
they
trying
to
have
a
government-backed
project
to
promote
precise
medicine
development
and
the
government
is
backing
this
programs
and
also
there's
a
lot
of
non-governmental
capital.
Low
intuitive
project
so
to
cover
the
market.
A
So
the
MTP
is
reaching
several
TB,
and
so
there's
a
lot
of
upstream
requires
requirements
that
cannot
be
met
by
the
tungsten
manufacturers.
The
current
status
default.
The
status
quo
is
off
Jana
Jana
and
the
sequencing
is
on
the
VM
or
HPC
so
which
resulted
in
a
very
big
investment
on
the
infrastructures.
A
A
lot
of
package,
for
example,
the
a
software
requires
the
old
version
of
the
GDK,
so
it
in
fact
influence
each
other
and
also
same
software
cannot
be
run
on
the
same
on
different
PC
at
different
platforms.
So
that
is
the
status
quo,
using
the
HPC
to
do
the
genome
sequence.
So
there's
a
lot
of
imperfection
at
all
these
industries
and
because
our
kings
was
used
to
do
the
containers
technologies.
A
For
example,
some
of
our
clients,
their
city,
all,
are
working
or
working
in
the
United
States
and
the
United
States
headquarters
requires
the
Chinese
team
to
not
use
HPC
but
the
darker
technologies,
and
then
that
is
why
a
lot
of
domestic
office
of
the
foreign
country
is
trying
to
use
darker
technologies
to
do
genome
sequencing
and
now
China
is
also
developing.
This
fear
voir
dire.
They
were
a
village
and
other
companies
are
trying
to
dabble
in
the
darker
technology
to
do
the
gene
sequencing.
A
So
next
let
me
introduce
the
introduction,
in
the
background
of
its
industry,
because
we
are
doing
the
software
and
IT
probably
have
little
idea
about
its
gene
sequencing
and
why
we
use
docker
and
container
in
the
field.
So
as
we
know
that
our
human
body
have
23
pairs
of
chromosomes
and
with
each
chromosomes,
it's
combination
of
x
and
y
Coppertone,
it's
just
like
a
south
hair
of
chromosomes.
If
you
open
it
up,
it
is
not
just
like
a
rope,
and
you
can
say
if
you
have
this
rocks
you
oil,
it's
like
a
rope.
Coils.
A
Dual
works
systems
and
it
was
combination.
These
ropes
with
combined
was
the
combination
of
two
or
four
different
base
pairs,
so
we've
known
as
different
purines,
eight
purity,
pairings,
G
and
C
pairings.
So
we
have
these
four
different
purines
and
that
is
the
sequencing
of
our
genes.
So
when
we
are
doing
the
sequencing
of
the
genes,
what
we
are
actually
doing
is
to
test
the
DNA
of
your
cells.
What
is
the
sequence
of
your
deal
and
a
of
yourselves?
We
have
G
a
and
T
and
the
cake,
and
also
the
GAC
T.
A
What
is
the
sequence
of
the
GAC
T
in
your
DNA?
We
call
it
the
two
helix,
so
basically
they
are
the
same.
There
are
small
differences
and
that
dictates
the
difference
of
your
hair
colors
of
your
pupil
colors.
So
the
purpose
of
the
gene
sequencing
is
to
find
out
the
order
of
the
ATG
securing
and
the
lens
of
the
ATG
C.
So
we
have
base
pair,
you
can
see,
we
have
three
billion
base
pairs
for
each
human
body,
and
you
have
this
a
big
human
base
pairs
and
we
are
using
Sanger
sequencing.
A
A
We
have
a
back
burner:
Human
Genome,
Project
HDPE.
There
are
six
countries
spending
ten
years
spending
the
decades
to
sequence,
the
three
billion
genome
for
the
human
body,
so
this
is
how
they
were
doing
the
work.
So
if
they're
using
Sanger
sequencing,
it
is
impossible
for
general
public
to
do
the
sequencing.
So
then
we
have
a
second
generations
of
citizen
technologies
because
we
have
already
have
for
the
typical
or
standard
human
gene
sequence.
So
the
second
gene
generations
of
city
gene
sequencing-
is
to
adopt
another
way
because
we
have
already
have
a
standard.
A
Then
we
have
NGS
citizen.
First,
the
break
that
broke
into
small
fragments
and
sequence
them
separately,
because
we
have
different
ways
of
secrecy
and
we
have
a
three
billion
sequence.
We
have
sitting
patterns
and
then
we
put
it
into
300
groups
so
using
the
NGS
sequencing
method.
It
is
very
fast
and
there
is
high
efficient.
We
can
sequence
different
300
different
for
a
complicity.
So
after
we
sequence
the
fragments,
then
we
need
to
put
it
into
big
groups.
A
Then
we
need
to
compare
and
contrast
each
fragments,
as
you
can
see
here
on
the
slides,
we
put
it
into
fragments,
we
put
the
isolated
DNA
into
fragments
and
then
we're
using
a
Python
to
do
the
sequencing,
we're
putting
for
gene
to
do
the
secret
Z
and
then,
as
you
can
see
here,
we
will
to
repair
it.
So.
B
A
Each
fragment
we're
trying
to
find
out
where
it
is
with
each
fragments
where
it
is.
What
is
the
position
of
this
fragment
in
the
total,
isolated
DNA,
and
then
we
are
trying
to
repair
overall
fragments
and
make
it
into
a
big
book,
so
it
is
very
fast,
but
we
need
to
do
a
lot
of
computing
during
the
sequence.
So
let
me
give
you
an
example
about
the
workflow
of
the
computing.
Let's
look
at
the
first
row.
A
This
is
the
total
length
of
the
3
billion
PP,
as
you
can
see
here
and
after
the
sequencing
machines
they
separated
into
different
fragments
and
with
a
300
PP,
so
where
it
is
in
the
overall
BP.
We
need
to
compare
and
contrast,
and
we
found
the
position
of
this
small
fragment
in
will
found
that
the
actor
position
after
the
match.
A
We
then
do
the
overlapping,
as
you
can
see,
tgc
on
this
part
and
the
confusion.
This
part
is
the
same
thing.
It
is
overlapping.
So
after
we
do
the
sequencing,
we
know
the
position,
if
you
don't
know
the
positions
of
this
fragment.
That
means
that
there
are
some
fragments
that
was
missed
during
this
process,
so
this
is
actually
the
repairing
process,
so
it
is
very
lengthy.
It
is
three
billions
18
and
with
each
fragments
after
this
abusive
machine
give
out
each
fragments,
then
each
fragment
has
a
300
DP
length.
A
A
So
you
need
to
have
for
30
different
layers
of
chopsticks
overlaps
together
to
calculate
the
length
of
the
overall
genes.
So
it
is
a
home
computing
workflow.
So
let
me
summarize:
first,
we
use
the
sample,
we
calculate
a
sample
mass
and
then
we
are
sequencing
machines
to
put
it
into
fragments
and
then
some
pieces
have
deformations
and
pieces
have
interference.
Then
we're
trying
to
repair
that
and
then
we
are
trying
to
calculate
a
number
of
the
pieces.
That
is
very
huge.
A
Some
other
factors
will
cause
the
deformation,
is
potential
bacterias
and
also
the
pea.
Fragments
of
numbers
is
very
huge.
So,
as
you
can
see,
we,
if
you
think
of
it
as
a
newspaper,
we
put
it
into
shredding
machines
into
different
shreds,
and
then
we
ask
you
to
put
the
straps
back
into
a
complete
newspaper.
So,
as
you
can
see,
the
computing
workflow
is
very
huge
and.
A
Here
is
human
bodies
kind
of
fleet
sequencing,
so
there
are
230
gigabits
of
our
informations
when
is
to
the
sequencing
and
to
computing,
so
that
the
figures
and
numbers
could
only
go
bigger
to
6
600
GP,
so
the
one
person's
whole
genome
sample
data
file
is
about
200
GB
and
then
after
we
do
the
mapping
filter
duplications,
alignments
and
index.
We
use
different
applications
to
do
the
computing
spell.
We
have
the
output
data,
which
is
mountains
of
data
after
this
process,
so
for
traditional
solutions.
A
B
A
B
A
B
A
B
You
can
at
home
to
run
it
successfully
as
well
as
on
the
cloud.
For
example,
like
our
a
cloud,
an
Amazon
Cloud,
there
will
be
all
the
same
and
also
the
low
usage
rate
of
resources,
because
it
can
be
scheduling
more
effectively.
On
one
note,
you
can
mix
your
tasks
to
balanced
scheduling
on
the
currents
knows.
B
That
is
a
advantage
of
using
container
for
gene
sequencing,
and
here
I
would
like
also
to
introduce
Cuba
gene
this
using
cuban
artists
gene
sequencing,
the
project
framework.
So
the
bottom
you
stock
hurts
you
to
the
isolation
which
is
to
solve
the
installation
of
software
and
overlapped
deploy,
and
we
use
Cuban
Alice
to
solve
the
problem
of
scheduling.
A
B
Top
is
qg,
that's
the
management
of
the
process.
You
can
define
different
testing
flow
like
the
tuber
testing
or
the
blood
testing.
As
you
mentioned,
to
find
all
the
flows
and
random
walk
of
flow
to
country
kubernetes.
It
will
be
scheduled
through
the
different
nose
to
run
the
tasks.
As
you
can
see
you,
bananas
subjects
are.
B
We
have
a
lot
of
workflow
management
system
in
this
industry
already,
but
why
we
are
doing
this
new
one,
because
it
has
a
customized
scenario
for
gene
sequencing
for
our
workflow.
It
is
the
definition.
Grandma
is
quite
simple,
but
with
powerful
functions,
it's
the
basic
grandma
definition.
As
you
can
see,
you
can
define
the
steps
how
much
resource
is
moved
and
the
darker
mirror.
You
use
the
resource.
You
use
the
command.
You
would
like
to
execute
also
the
highlighted
in
red.
It
supports
the
direct
shell
method.
B
B
A
A
B
A
A
B
A
B
You
can
when
this
command
notified,
multiple
pi
is
iteration.
So
Cuba
Jean
supports
this
3
grandma.
Basing
on
your
original
scenario,
you
can
find
the
most
suitable
one.
If
you
list
them
all,
you
can
just
copy
paste
if
this
variation,
you
can
also
copy
variation
here,
it's
very
easy
to
transfer
the
original
scenario.
B
And
base
kubernetes,
we
have
this
adjustment.
If
you
use
cuban
edits
to
run
the
task,
it's
scattered
on
the
nose,
but
temp
will
have
a
new
assignment.
You
know
first
delivered
to
the
nose
that
can
contain
to
the
black
nose,
but
we
have
this
kind
of
adjustments
to
prioritize
the
delivery,
so
we
use
one
knows
fully
than
most
blank
ones.
In
this
way,
we
want
to
save
the
resources.
Save
cost
also
is
the
scalpel
or
strategy
also
to
save
and
make
the
use
of
resources
more
effectively.
B
B
A
B
B
B
A
B
B
B
B
B
B
A
B
A
A
And
this
is
cribbage
in
three
basic
concept
tools:
execution
result
in
the
process.
When
it
comes
to
tolls
it
any
sequencing.
Tasks
can
be
support.
Our
own
attempts
to
workflow
it
is
a
combination,
also
each
stickers
and
tool
and
the
dependent
and
dependencies.
For
example,
if
we
went
to
screen
the
tumors
or
there's
a
warrant
in
Ireland,
which
is
the
main
place,
the
name
of
your
samples,
so
in
two
classes,
if
you
set
the
viral,
make
it
a
constant
parameters,
then
there's
an
execution.
So
you
can
execute
your
data.
A
A
So
now
our
task-
our
project-
is
like
this,
so
why
a
cloud
is
based
on
kubernetes
and
about
technologies.
We
are
actually
biggest
contributors.
The
mornings
came
out.
You
saw
that
we
are
doing
the
CF
in
the
CN
CF.
We
are
the
first
international
members
for
CN
CF,
which
means
that
we've
been
a
lot
of
time
in
the
back
in
the
past.
Do
this
we're
trying
to
continue
to
upgrade
our
contributions
in
business
EF
community
and
now
our
open
source
projects
have
a
lot
of
contributors.
Also,
for
example,
huawei.
A
The
cuban
gene
open
source
project
involves
file
a
next
omics
and
Kermit
X
Turing
Power
Technologies
and
a
bio
trainee
calm.
So
these
are
the
founding
members
for
our
cube
gene
open
source
projects
to
do
the
coop
gene.
This
is
the
website
for
cube,
GM
and
the
on
it.
You
can
find
how
to
install
Cugini,
how
to
the
basic
sequencing
process
for
a
gtk,
purines
sequencing
x',
and
it
can
even
also
find
our
product
process
on
this
website,
and
we
have
files
documents
to
support
you.
You
can
find
it
there.
A
A
Alright,
previously,
let
me
repeat
the
presentation
in
the
workflow:
first,
no
there's
no
input
file
and
output
file
destination,
now
researching
different
platforms
for
gene
sequences
and
different
platforms,
45
input
and
output
file
and
according
to
the
output
files,
is
the
same
with
Inca
files
of
next
step,
but
we
found
that
is
not
a
good
solution,
because
that
makes
our
tour
definition
very
complex.
You
have
to
define
their
software,
you
have
to
define
the
input
output
files,
it's
very
complex.
Some
software's
cannot
capable
it's
not
capable
of
doing
that.
A
So
all
in
our
process,
we
are
very
generic.
You
don't
care
about
the
input
output
files
if
you
just
write
a
lot
of
files.
So,
just
like
the
same
with
your
writing
commands,
it
turns
up
input
a
code
that
would
command
you,
don't
have
to
define
any
files
to
include
an
output.
Just
like
writing
codes.
So
for
us
we
don't
defined
files
to
control
the
sequence
we
controlled.
The
dependent
using
the
content
found
the
sequence,
so
it
is
more
generic
and
easier
to
do.
You
can
see
if
this
is
a
dependency
task.
A
If
the
a
depends
on
P,
then
we
use
a
dependent
logic
to
do
that.
To
connect
this,
you
don't
have
to
define
the
input
output
files
because
they
are
already
in
the
catalog.
We
don't
need
to
find
an
input
output
file
and
if
you
need
to
map
the
input
output
file
that
you
can
use
this
dynamic
concurrence
files
to
do
this.
So
there's
no
problem
about
that
and
the
job
name.