►
From YouTube: Introducing Kubeflow (w. Special Guests Tensorflow and Apache Spark) - Trevor Grant, IBM
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Introducing Kubeflow (w. Special Guests Tensorflow and Apache Spark) - Trevor Grant, IBM
Data Science, Machine Learning, and Artificial Intelligence has exploded in popularity in the last five years, but the nagging question remains, “How to put models into production?” Engineers are typically tasked to build one-off systems to serve predictions which must be maintained amid a quickly evolving back-end serving space which has evolved from single-machine, to custom clusters, to “serverless”, to Docker, to Kubernetes. In this talk, we present KubeFlow- an open source project which makes it easy for users to move models from laptop to ML Rig to training cluster to deployment. In this talk we will discuss, “What is KubeFlow?”, “why scalability is so critical for training and model deployment?”, and other topics.
https://sched.co/Nrvp
A
This
is
a
an
introduction
level
talk
if
you
want
like
a
really
really
big,
deep
dive
on
coop
flow
or
something
advanced.
This
isn't
it
so
you
can
leave
also
normally
this
talk.
It's
me
and
a
co-presenter
and
we're
telling
jokes
the
whole
time.
But
jokes,
don't
really
go
over
the
language
barrier?
Well,
so
if
you
were
expecting
a
lot
of
jokes,
this
is
actually
gonna
be
like
real
contents
and
I'm
sorry,
so
you
can
also
leave
now
and
I
won't
be
offended.
A
First
of
all,
things
that
aren't
on
here
and
also
a
little
bit
about
this
presentation.
Normally,
when
I
give
talks,
I
may
be
like
the
day
before.
That's
when
I
really
start
practicing
I
go
through
it,
a
lot
I
correct
all
the
errors
I
find
on
the
slides
for
this
I
submitted
the
slides
a
week
ago.
So
they
were
all
done.
Great
I
came
here
and
I
wasn't
able
to
get
into
my
Google
account,
so
I
couldn't
edit
any
of
the
slides.
A
So
there's
lots
of
errors
and
mistakes
and
I'll
try
to
point
those
out
so
I
apologize
about
that
in
advance.
Also
I
changed
some
content
so
that
slides,
don't
exactly
match
up
with
what
I'm
talking
about,
but
that's,
okay,
too
I
also
talked
really
fast.
I
am
working
very
hard
to
talk
as
slow
as
I
can,
but
if
I
am
talking
too
fast,
please
just
like
raise
your
hands,
and
let
me
know
that
I
need
to
slow
down
and
I
will
that's?
Okay,
we'll
do
questions
at
the
end
as
well.
A
Also,
there's
some
idioms
same
if
I
say
something
and
it
sounds
weird
and
doesn't
make
any
sense,
just
raise
your
hand
and
I'll
stop
and
I'll
explain
what
I
mean
by
that
and
I'll
also
be
watching
my
five
translators
in
the
back,
if
I'm
going
too
fast.
Just
let
me
know
and
I'll
slow
it
down.
So
about
me.
My
name
is
Trevor
grant
I'm
from
Chicago
I
work
at
IBM.
Doing
stuff.
A
I've
worked
a
lot
in
the
Apache
communities.
That's
a
lot
of
my
open-source
history
has
been
coop
flow.
Is
my
first
adventure
into
the
kubernetes
ecosystem
I'm,
putting
together
an
IOT
tract
at
Apache,
con
North
America?
If
y'all
want
to
come
you
you
should
you
can
find
me
pretty
much
all
over
the
internet
with
this
handle
of
rock
and
Trevo
I've
got
a
web
site
and
blogs
and
github
Xand,
you
name
it
rock
and
Trevo.
A
A
It's
been
a
problem
that
people
have
been
talking
about
an
AI
and
ml
space
for
a
while.
Now
that
there's
you
know,
is
there
any
ever
going
to
be
a
tool,
a
single
tool
that
everyone
rallies
on
and
I
think
the
answer
is
no,
and
so
then
the
question
is:
how
do
we
start
dealing
with
this
rampant
proliferation
of
frameworks
because
it
seems
to
be
growing
too
there's
more
and
more
frameworks
popping
up
solve
more
and
more
new
problems.
A
Another
example
is
there's
three
talks
on
just
frameworks
on
kubernetes
going
on
right
now,
I
appreciate
you
sticking
around
and
coming
to
mine,
but
again
underlines
how
much
oh
he's
going.
Who's
gonna
go
see
one
guy,
the
ones
I'm.
Sorry,
it
underlines
how
much
there
is
of
the
problem
of
this
proliferation.
A
A
The
model
or
the
algorithm
the
data
science
makes
is
really
one
small
and
trivial
part
of
creating
production,
alized,
machine
learning
and
artificial
intelligence.
There
are
there's
data
prep,
there
is
machine,
learning,
library
version
control.
There
is
model
version
control,
there
is
validation
of
the
model
and
doing
all
these
things
in
real
time,
and
while
the
data
that
the
data
scientist
model
having
a
good
one,
is
good,
a
very
elegant
model
that
you
can't
production
alized
is
worth
less
than
a
simple
linear
regression
that
you
can
put
in
production.
A
In
my
opinion,
oh
another
little
asterisks
on
this
I
work
at
IBM,
but
all
of
the
things
I
say
are
my
opinions
and
you
not
necessarily
represent
the
opinions
of
IBM,
so
it
bears
that
yeah
going
back
into
just
you
know.
The
model
serving
and
model
training
being
able
to
serve
a
model
and
being
able
to
scale
gracefully
on
different
types
of
hardware
is
an
important
thing
for
your
machine
learning,
your
artificial
intelligence
pipelines.
A
This
joke
still
carries
so
as
you,
your
data
scientists,
you
can
usually
tell
you
get
a
feel
for
how
experienced
they
are
by
how
much
they're
demanding
that
they
need
so
much
me
more
GPUs
cloud
providers,
we're
always
happy
to
sell
you
GPUs
and
time
on
our
GPU
machines.
What
is
kubernetes
as
everyone
at
a
kubernetes
conference.
I,
assume
that
you
know,
and
if
you
don't
it's
magic
and
containers.
A
A
So
you
can
imagine
that-
and
this
is
our
data
scientist
who
thinks
that
they're
very
important
and
that
they're
like
the
critical
piece
of
the
organization
in
2019,
again
Trevor,
not
IBM
in
2019.
Anyone
who
is
self
identifying
as
a
data
scientist
is
not
a
technical
user.
So,
in
addition
to
all
these
problems
that
I
was
just
talking
about,
we
also
need
to
create
something
that
data
scientists
can
use,
which
has
a
nice
user
interface.
It's
easy
to
configure
otherwise
they're,
just
not
going
to
use
it
and
they're
gonna
write
some
glue
code,
so.
A
The
problems
we're
discussing
are
common
the
solutions
to
these
problems,
which
they're
common,
like
across
all
industries,
all
sectors,
lots
of
businesses,
the
solutions
that
people
come
up
with
traditionally
our
ad-hoc
glue
code,
hack
scripts,
and
by
that
those
are
all
idioms,
and
by
that
I
mean
it's
individual
teams,
creating
these
little
one-off
solutions
that
will
solve
this
particular
problem
that
they
have
on
their
team
or
on
their
product.
For
this
one
case
and
the
problem
with
that
is
you
get
lots
of
duplicated
effort
when
you
do
that?
A
You're,
writing
a
you
know,
it's
the
same
thing
over
and
over,
but
you
know
a
lot
of
times.
Maybe
it's
a
copy
and
paste
and
then
just
change
a
couple
things
to
fix
it
for
this
next
problem
and
the
even
bigger
problem
with
that
is
you're
getting
a
lot
of
technical
debt.
So
if
you
have
one
person
who's
in
charge
of
production,
iliza
all
of
these
things.
A
A
It's
a
here
are
some
of
them.
It's
not
all
of
them.
You
can
add
your
own,
but
in
general
the
pieces
of
this
buffet.
You
need
to
create
a
machine-learning
meal
for
lack
of
a
better
word
is
you're.
Gonna
want
to
have
machine
learning,
libraries,
which
are
version
controlled.
You
need
data
prep,
you're
gonna
need
the
models,
data
prep
model
model
management.
So
when
you
have
a
model
and
then
you
have
the
next
version
of
the
model,
maybe
you
need
to
roll
back
and
then
you're
gonna
want
to
have
serving.
A
So
how
does
this
all
fit
in
an
ecosystem?
So
you
might
be
saying
right
now:
hey
I!
Don't
need
to
do
all
that
stuff!
I
just
do
research,
my
management
comes
and
they
say
we
need
to
know
something,
and
then
we
do
research
and
then
we
give
them
the
answer
in
a
report.
That's
great.
You've
got
a
great
job,
keep
it
because
those
jobs
are
going
away
in
favor
of
we
want
answers
in
real
time.
We
want
to
know.
A
We
don't
want
to
have
to
do
a
post-mortem
analysis
on
why
sales
were
down
last
quarter.
We
want
to
know
in
the
middle
of
the
quarter.
Why
say
that
sales
are
down?
Why
they're
down
and
how
to
fix
them?
And
that's
where
the
world
is
going
so,
but
for
everyone
else,
that's
what
cool!
That's
the
problem,
the
coop
closed,
trying
to
solve,
trying
to
help
and
also
to
scale
out
those
resources.
A
The
other
day,
the
data
scientists
on
the
laptop,
a
big
problem
with
laptops
is
that
they're
not
great
for
training,
machine
learning
models
a
lot
of
times.
They
don't
have
a
lot
of
power,
that's
not
what
they
were
built
to
do.
They
were
built
to
say
you
could
code
stuff
in
a
coffee
shop.
Everyone
can
see
you,
you
look
cool,
but
they're
not
great
for
actually
training
models,
so
you
can
scale
out
your
training
resources
you
can
deploy
to
production,
and
so
why
would
you
want
to.
A
A
Go
look
at
the
docs.
It's
not
that
hard!
It's
even
easier
than
this
there's
not
control
scripts
anymore.
There's
another
version
coming
out
next
week:
that's
gonna,
make
it
even
easier,
I'm,
just
gonna
short
it
out
with
that
the
chef's
recommended
pairing,
though,
if
you
use
those
out-of-the-box
scripts,
you're
going
to
get
Jupiter
hub,
you're
gonna
get
tensorflow
serving
Central
Java
hub
PI
torch
ketab,
which
is
hyper,
parameter
tuning,
a
list
of
things
and
pipelines
which
is
Argo
and
a
little
bit
of
magic,
and
you
might
be
saying
to
yourself
what
are
those
pipelines?
A
What
does
that
mean?
There
used
to
be
a
joke
about
this
cat
and
it
made
sense
with
pipelines
but
I've
forgotten
it
so
you're,
probably
thinking
okay
cool.
We
I've
got
some
scripts
that
let
me
deploy
a
lot
of
services
and
components
on
kubernetes
it,
but
it
still
needs
to
be
easy
for
my
non-technical
data.
Scientists
use
because
they're
not
gonna
go
learn.
Kubernetes
they're
gonna
keep
doing
things
their
old
way
and
writing
their
glue
code
and
we're
never
gonna
get
anywhere
with
this,
and
also
like
that's
not
an
oppressive
you.
A
You
wrote
an
install
script
that
just
installs
a
lot
of
things
which
I
guess
with
pipe
lines
is
saying
is
that's
not
what
coupe
flow
is
coupe
flow
is
a
way
for
these
non-technical
data.
Scientists
keep
track
to
set
up
experiments
to
run
these
different
experiments
with
different
hyper
parameters.
Oh
configuring
in
and
out
different
say
input
data
streams
also
to
have
reproducible
results.
Are
there
any
data
scientists
in
here
and,
if
so,
I'm
sorry
I'm
making
fun
of
you
so
bad
I'm?
No
one
wants
to
raise
their
hand.
A
A
So
you
want
to
be
able
to
keep
track
of
those
experiments
and
then
it's
even
worse,
if
somebody
else
wants
to
also
know
like
there
may
be
working
on
something
similar,
ok,
I'm,
just
going
to
zip
up
all
these
notebooks
and
send
them
over
good
luck.
It's
not
a
great
system!
So
you
want
to
have
this
easy-to-use
UI,
you
can
build
experiments,
run
experiments
where
you
can
take
the
same
pipeline
of
data
prep
and
data
validation
and
all
of
those
things
that
are
important
and
then
move
it
from
training
to
production
and
every
nothing
changes.
A
And
that's
a
really
important
thing
too,
because
if
you
train
a
model
and
you
prep
your
data
one
way,
but
then
in
production,
the
data
prep
does
something
different
and
you
wouldn't
think
that
would
happen.
But
changes
in
machine
learning
libraries
will
quietly
introduce
changes
in
the
way
a
data.
Prep
step
is
done
or
the
way
a
model
is
computed
and
you
train
with
SK
learn.
A
This
is
actually
a
screenshot
from
coop
flow
in
the
pipeline
UI,
it's
not
doing
a
great
job
of
showing
how
to
set
it
up.
It's
done
with
the
ammo
files
and
Python.
You
can
do
a
Python
as
well,
if
you
like,
but
it
has
some
important
parts
to
it.
You've
got
your
validating
your
data
and
you
can
again
doing
that
the
same
in
production
where
you've
got
streaming
data
coming
in
and
making
sure
it
looks
the
same
and
is
about
like
and
also
so
in.
A
Not
only
schema
is
the
same,
but
also
that
your
sensors
aren't
broken
so
there's
two
ways
your
data
can
fail.
If
you
will,
you
can
have
a
schema
change
and
maybe
something
that
used
to
be
true.
False
is
now
coming
in
as
a
one
in
a
zero
and
that's
being
treated
as
an
integer
instead
of
a
boolean.
That's
a
problem,
but
you
can
also
have
like
busted
sensors.
So
if
we've
got
a
sensor,
that's
supposed
to
be
between,
you
know
negative
10
and
positive
20
degrees
that
those
are
reasonable
values.
A
If
that
thing
starts
reporting
like
500,
you
know
it
could
be
that
the
things
running
at
500,
but
it
also
could
be
that
your
sensors
broken,
and
so
you
want
to
be
able
to
know
that
fast.
You
don't
want
the
thing
to
start
making
predictions
based
on
the
machines
running
at
500
degrees,
activate
the
fire
suppression
system.
Well
know
you
need
to
fix
the
sensor,
and
you
want
to
know
that
quick.
So
another
big
important
part
about
this
is
treating
errors
in
your
data,
like
their
bugs
in
software.
A
That
kind
of
goes
down
a
rabbit
hole.
It's
just
something
to
think
about
and
keep
in
mind
when
you
are
deploying
machine
learning
jobs
to
production.
So
all
so
you've
got
your
data
transfer
data
transformation.
That's
the
pre-processing,
I've,
already
kind
of
touched
on.
Why
that
doesn't
always
work
out
as
well
in
production
as
it
doesn't,
your
training
or
things
can
change
quietly
and
another
problem
that
you
have.
We
move
from
training
the
production
is
your
data,
scientists
will
say:
oh
we
found
this
really
good
predictor
variable
Ignat.
A
Does
you
know
if
it
counts
for
98%
of
the
variation
in
the
model?
Well,
that's
because
it
gets
calculated
after
like
later
it
is
an
exam
it
isn't
available
at
run
time.
You
want
to
find
that
out
sooner
rather
than
later,
whatever
so
I
think,
a
lot
of
people
were
maybe
here
for
a
spark
and
coop
low
Holden
is
a
PMC
on
Apache
spark
and
really
really
knows
this
topic.
Very
well
I
can
give
you
her
contact.
A
That's
just
pictures
of
stuff.
Pi
spark
is
the
interface
layer.
Basically
here
she's
talking
that
about
the
serialization.
It's
a
there's
a
lot
of
serialization
going
on
and
she
says
you
should
use
arrow.
This
is
another
aside,
don't
ever
trust
vendor
benchmarks.
Thunder
benchmarks
are
usually
garbage
because
the
vendor
makes
the
sets
the
tests
up
to
make
whatever
they
want
to
say.
Look
great.
A
A
A
So
cross
clouds-
this
is
something
I
can
talk
about.
So
one
of
the
great
things
about
kubernetes
is:
we
are
theoretically
moving
toward
a
world
where
you
can
create
these
jobs
and
you
can
deploy
them
on
various
cloud
providers.
Ibm,
Amazon,
Alibaba
Huawei,
maybe
has
a
cloud
you're
evading
rate,
various
cloud
providers.
You
don't
have
to
get
locked
into
one
cloud
provider
in
theory,
the
same
kubernetes
job
should
work
wherever
you
two
play
it.
A
A
That
being
said,
how
do
you
do
that
with
coop
flow?
Well,
the
good
news
is
it's.
If
you
get
coop
flow
up
and
running
on
your
cloud
provider,
which
is
not
exactly
trivial
on
all
kraut
writers
at
the
moment,
I
will
be
the
first
to
admit,
however,
we're
working
on
it.
The
pipeline's
make
it
very
easy
to
transfer
your
training
jobs.
You
create
consistent
results
on
different
clouds.
A
There
it's
a
series
of
steps,
it's
our
go
really
again
with
a
thin
wrapper
over
it,
but
that's
good,
because
you
can
run
an
example
and
then
you
say
well
I
want
to
try
it
like
this,
but
then
your
cloud
provider
is
more
expensive
and
this
one's
cheaper
cool,
I'm,
gonna
ship,
the
job
data
gravity
is
a
new
buzzword
that
you
can
learn
today.
That
is
where
is
your
data
store?
A
The
other
way
you
can
do
this
is,
let's
say,
you've
trained
a
model
on
cloud
provider
one,
but
over
time
they
think.
Oh,
we've
really
really
got
you
locked
in
and
we're
gonna
yeah
we're
gonna
keep
turning
up
those
kubernetes
costs
on
you,
you
take
the
pipeline.
You
redeploy
to
cap
cluster
number
two
or
Krupp
cloud
provider.
Number
two!
That's
what
happens
is
you
are
able
to
basically
tell
cloud
provider
number
one
all
right,
it's
been
fun,
but
we're
leaving
now
and
run
on
a
cheaper
cloud
provider.
A
There
are
some
other
videos,
oh
man,
I'm
guessing
you
guys
can't
get
most
of
these
wings.
Sorry
about
that!
I
didn't
realize
until
the
day.
Just
now,
actually
why
you
shouldn't
use
this.
It's
0
5!
Now
it's
0,
6
lots
of
API
breaking
changes
are
being
introduced
all
the
time
it's
starting
to
settle
out,
but
it's
still
a
thing
I.
This
personally
would
not
say
this.
A
Is
production
grade
software,
yet
I,
don't
think
many
people
would,
but
it's
interesting
and
it's
exciting,
and
it's
something
to
start
thinking
about
for
years,
6
to
12
months,
architectural
plans
like
what
are
we
gonna
move
all
our
data
scientists
onto
compared
to
doing
it
locally?
This
is
a
lot
of
overhead.
You
got
a
little
cooler
now
to
use
cluster.
A
You
got
8
15
different
services,
running
you'd
need,
like
your
laptop's
gonna
start
like
just
buckling
under
the
weight
of
all
the
stuff
you're
running,
just
to
get
up
and
do
simple,
like
machine
learning,
it's
hard
to
do
a
hello
world
example.
There's
also
workarounds
for
that
right
now,
but
another
issue:
all
of
these
components
mean
that
three
different
talks
on
coop
flow
could
give
you
three
totally
different
component
sets
and
three
it
not
even
look
like
it's
the
same
project,
though
it
is,
and
then
also
and
again.
A
This
is
me
and
saying
this,
but
I
so
coop
flow
is
not
in
a
foundation
and
I'm
not
saying
this
is
a
shot
against
coop
flow,
but
you
should
always
beware
of
software.
That's
not
open
source
software!
That's
owned
by
company!
Not
a
foundation
I
think
Whoville
does
have
a
very
strong
community
and
they
seem
to
be
working
really
well
with
the
community
and
I.
Don't
think
that's
gonna
change,
but
it's
just
you
know
anytime
you're,
looking
at
some
open
source
software
and
it's
owned
by
a
company,
not
a
foundation,
be
careful.
A
A
A
A
The
question
was
but
yeah
he's
a
book
to
be
poverty
too
soon
we're
hoping
to
get
the
book
out
before
sometime
near
January,
1st,
so
I
wouldn't
say
soon
soon.
Part
of
the
problem
and
the
reason
for
that
is
hoof
load
just
recently,
refactored
away
from
case
on
it,
which
means
a
lot
of
the
code
changed
and
we're
like.
B
A
Are
a
few
ways
you
have
I'm,
you
can
use
network
file
stores
what
they're
called
persistent
volumes.
S3
versioning
your
data
I,
don't
believe
there
is
anything
I'm
kind
of
looking
for
a
head
nod
or
a
shake,
but
I,
don't
believe,
there's
anything
associated
kupaka
moment
for
versioning
your
data,
correct!
There's,
no
K,
that's
wrong.
So
there
is
something
at
the
moment
so.
B
A
C
A
So
I
think
that
would
be
the
idea
with
the
pipelines
that
you
would.
You
would
basically
just
set
up
your
coupe
flow
cluster
next
to
the
data.
So
if
I've
got
my
data,
let's
say
I
save
two
copies
of
my
data
or
two
slices
of
my
data,
one
on
AWS
and
one
on
IBM.
Just
because
that's
what
I
worked
for
and
I
should
plug
them
at
least
once
so.
A
The
idea
there
would
be
that
you
would
have
you'd
run
the
job
on
AWS
and
you'd
run
around
coop
flow
as
far
as
merging
things
back
together
like
if
you
had
two
slices,
and
you
want
to
merge
it
together-
that
don't
move
again
that
there's
enough
I
don't
have
experience.
I
can
I
can
thought
exercise
how
I
would
go
about
it,
but
I'd
be
literally
making
it
up
on
the
fly
yeah,
but.
C
Because
I
think
that
that
actually
can
mitigate
that
problem
with
data
distribution
and
also
with
eventually
keeping
data
confidential
at
the
yep,
add
whoever
owns
the
data
and
through
some
kind
of
encryption
and
Homo
freak,
encryptions
and
so
on.
Let
you
get
just
the
insights
and
I
know
that
at
least
I
saw
that
new
tensor
flow
has
this
federated
learning
options?
I,
don't
know
for
other
tools
and
I
was
wondering
you
know
how
difficult
would
be
in
q4
to
set
up
these
pipelines
and
to
and
to
define
that
this
federated
learning
mode
of
action.
D
One
of
the
GCP
is
a
major
directions
in
the
future
is
to
work
on
hybrid
cloud
and
like
including
both
cloud
environments
and
there's
a
broad
projects
called
our
anthos,
and
this
was
announced
at
gcpd
next
this
year.
So
you
know
training,
that's
like
on
premise
and
across
multiple
cloud:
that's
all
one
of
the
director
general
tracks
and
for
answers,
so
the
AI
component
for
for
for
your
situation
will
be
also
available,
but
I
cannot
give
a
an
estimate
on
that
day.
At
this
point
and.
A
A
You
also
want
to
be
able
to,
let's
say-
and
maybe
a
more
realistic
use
case
is
to
Train
on
your
private
cloud,
where
the
data
is
and
then
serve
scalably
on
a
public
cloud,
and
I
always
forget
to
mention
that,
but
that
is
literally
the
use
case
of
the
moving
the
pipeline
and
the
cross
cloud
training,
but
I
always
just
come
up
with
arbitrary
clouds
and
that's
wrong.
So
thank
you
for
that
as
well.
E
Okay,
it
sounds
like
a
lot
of
the
driving
force
behind
cube
flow
is
in
order
to
enable
data,
science
and
machine
learning
at
large-scale.
My
question
actually
relates
to
small
scale
and
and
the
applicability
of
cube
flow
there
I'm
working
with
a
research
team
at
a
university
and
right
now,
our
it's
a
machine
learning
project,
but
our
pipeline
is
a
series
of
Jupiter
notebooks
scattered
across
five
github
repositories.
You
have
to
look
in
a
20-7
repository
for
a
20-17
repository
for
some
data.
E
It's
really
really
bad
and
I
see
a
framework
like
this,
which
is
almost
like
a
full
sack
kind
of
data
data
pipeline
machine
learning
framework
comes
with
a
little
bit
of
overhead,
given
that
you
need
to
have
kubernetes
involved.
But
my
question
is
from
your
experience
in
the
project
so
far.
Do
you
think
that
the
job
management
and
the
Model
Management
and
the
versioning
that
you
get
from
cube
flow
is
applicable
to
smaller
scale
projects
or
are
there
simpler
frameworks
that
I
should
be
looking
at
as.
A
A
Do
you
agree,
yeah
that
you
know
it
I
don't
want
to
over
slam
the
whatever
you
need
a
server,
and
that
can
be.
You
know
that
can
be
public
cloud
there
are
you
can
as
a
university
I'm
guessing
you've
got
old,
blades
laying
around
here
there
and
everywhere
you
really
just
need
somewhere
that
can
hold
that
it'll
just
stay
on
and
holding
data
like
if
you've
got
that
you're,
probably
gonna,
be
okay,
yeah,
that's
the
I
think,
but
I
think
I
think.
Yes,
this
is
definitely
something
to
watch
and
keep
an
eye
on.
B
Servers,
you
know
house,
so
a
lot
of
work
is
done
without
a
cow
or
even
on
your
local
computer,
because
we
do
experiments
locally.
So
we
we
use
a
lot
of
Condor
containers.
So
it's
not
so
I
think
in
the
future.
We
really
something
that
can
also
work
the
kubernetes,
but
also
can
integrate
and
co-production
and
and
I
also
use
ml
all
forms
from
bigger
bricks,
and
they
are
machine
and
in
fact
they
have
a
personal
effect
in
with
counter
container
so
for
small
scale.