►
From YouTube: OpenShift Commons ML SIG Meeting March 1 2019 Full MLFlow Introduction and MLFlow Operator demo
Description
OpenShift Commons
ML SIG Meeting
March 1 2019 Full Meeting Recording
MLFlow Introduction and MLFlow Operator demo
Databricks Mani Parkhe
Red Hat Zak Hassan and Diane Mueller
A
All
right,
there'll
be
a
couple
of
small
files
there
when
I
look
at
the
recordings,
so
today
I'm
welcome
everybody
back
to
the
machine
learning
on
open
shift.
Sig
and
my
name
is
Diane
Mueller
I
am
the
co-chair
of
this
mi,
regular
that
cig
meeting
and
today
we
have
a
couple
of
speakers
lined
up
to
talk
about
ml
flow
and
an
ml
flow
operator.
That's
being
worked
on.
If
you
have
a
chance
in
the
chat.
There's
a
link
to
the
meeting
notes
that
you
can
see
on
your
screen
right
now.
A
You
can
add
that
if
you
can
add
your
name
do
the
attendees.
That
would
be
great,
as
I
was
mentioning
before
I
hit
the
record
button
to
Manny
and
Zak.
One
of
my
goals,
for
this
is
to
what
the
next
steps
are
in
terms
of
the
ml
flow
operator.
So,
unlike
many
to
give
and
take
the
screen
over
now
and
maybe
explain
what
ml
flow
is,
give
us
some
context
for
that
and
then
Zack,
who
has
been
working
on
an
ml
flow
operator
and
he's
on
repo.
A
We
get
that
work
moved
into
a
space
in
the
ml
flow
repo
that
other
people
can
collaborate
and
work
on
there
that
the
maintain
needles
are
telling,
if
that's
the
appropriate
thing
to
do
that's
kind
of
what
I'm
trying
to
suss
out
today
too,
and
for
those
of
you
who
don't
know.
Yesterday,
we
launched
something
called
operator
hub,
do
where
there's
some
about
a
dozen
now
one
day
in
operators
that
are
that
run
on
kubernetes
generic
standard
stuff
and
as
well
as
on
an
open
ship.
A
B
All
right,
hello,
everybody!
Good
morning,
my
name
is
Manny
Parque
I
am
an
engineer
at
data
brakes
working
on
a
Mel
flow,
so
I'm
here
to
talk
to
you
all
about
what
is
ml
flow,
the
motivation
behind
building
this
machine
learning
framework,
what
the
competence
of
ml
flow
looks
like
and
where
you're
going
to
take
this
so
I'm
gonna
introduce
all
of
this
in
the
next
15
minutes
or
so
as
a
preamble
to
Zacks
presentation.
So
anybody
I'm
sure
this
team
appreciates
how
complex
ml
flow
development
is
and
also
appreciate.
B
That's
a
slightly
harder
than
traditional
software
development,
as
we've
done
in
the
last
few
decades.
Sort
of
to
tease
this
out
a
little
bit.
I
want
to
sort
of
talk
about
some
differences
between
traditional
software
development
and
machine
learning
development.
So,
let's
start
with
a
problems
in
each.
So
let's
you
take
a
problem
in
traditional
software.
Let's
say
you
building
a
credit
card
transaction
system
or
a
functional
verification
system,
or
any
of
that
you
start
with
a
phone
or
with
a
functional
specification.
B
You
know
exactly
the
terms
and
conditions
and
what
products
you're
trying
to
build.
So
the
goal
is
pretty
clear
what
you
want
to
do
in
machine
learning.
The
goal
is
to
optimize
the
metric,
so
there
is
no
perfect
answer.
You
just
try
to
get
better
and
better.
The
metric
could
be
increasing
accuracy
or
it
could
be
a
vector
of
different
metrics
that
you're
trying
to
optimize
all
right.
The
other
difference
is
quality
in
traditional
software.
B
When
we
go
to
machine
learning,
there's
not
only
all
the
code
and
the
system,
but
also
the
data
that
we
use
for
training
the
models
and
then
how
how
well
the
model
is
T
own
and
how
we
like
regularly
continue
to
update
with
fresh
data,
and
we
have
to
tune
the
models,
use,
different,
algorithms
and
and
so
on
and
so
forth.
So
the
quality
is
sort
of
a
shifting
goal
and
a
moving
target
just
like
the
goal
that
we
talked
about,
and
the
third
thing
I
want
to
talk
about
is
in
traditional
software.
B
You
develop
off
of
a
common
software
stack,
something
that
the
team
is
Flexer
is
has
worked
with
in
the
past
release
different
projects
on
you're,
pretty
well
acquainted
machine
learning.
On
the
other
hand,
you
want
to
constantly
experiment
with
new
libraries
that
constantly
come
keep
coming
out
different
frameworks.
Your
algorithm
is
very
different
types
of
models,
and
not
only
that
or
other
than
experiment.
You
must
know
how
to
production
lies
them.
So
just
having
a
model
that
is
built
with
this
new
framework
is
not
good
enough.
B
B
And
then
you
have
all
of
these
different
frameworks
where
you
can
deploy
that
the
using
all
of
this
can
sort
of
present
all
its
challenges,
and
typically
for
machine
learning
practitioners.
It
is
incentivized
to
use
different,
different
algorithms
and
different
frameworks
in
order
to
best
receive
to
get
the
best
quality
that
we've
been
talking
about
where
your
model.
So,
if
optimizing
metrics
is
the
goal,
then
you
pretty
much
end
up
trying
out
various
different
tools
until
after
that,
oh
just
being
able
to
train
and
create
a
model
is
not
sufficient.
B
We
all
know
that
using
the
right
parameters
for
tuning
is
a
quintessential
to
getting
the
right
model.
It
can
make
all
the
difference
between,
like
you
know,
just
guesswork
versus
using
the
right
set
of
parameters
to
get
like
the
ideal
results
that
you've
been
looking
for
for
development
of
product
and
beyond
that,
as
we
talked
about
deploying
of
these
models
and
managing
the
lifecycle
of
each
of
these
models
becomes
a
challenge
in
itself.
B
So
you
may
have
a
team
working
on
on
a
team
of
data
engineers
working
on
the
ETL
and
logs
part
of
it,
and
a
set
of
data
scientists
working
on
the
training
aspect
and
a
set
of
system
engineers
on
the
deployment
part
of
it.
So
you
have
to
sort
of
work
through
the
lifecycle
of
a
model
how
it
passes
through
these
different
teams
and
governance
of
it.
B
So
this
in
itself,
what
we
have
seen
is
an
extremely
complex
process.
There
is
not
a
single
tool
out
there
that
makes
it
easy
to
use
and
in
fact,
you
have
to
end
up
using
multiple
tools
that
you
and
the
team
are
comfortable
with
right.
So,
as
we
start
looking
at
this
or
when
we
built
mo
flow
as
an
open
platform
to
be
able
to
for
users
to
manage
their
models
and
machine
learning
frameworks,
we
wanted
it
such
that
it
works
with.
All
of
these.
B
Existing
tools
makes
it
easier
to
work
across
the
stack
these
various
tools
and
not
necessarily
be
one
of
these
tools,
but
then
would
work
with
a
lot
of
these
tools
to
solve
the
problems
that
have
not
been
solved
so
going
over
to
stuff.
You
have
to
talk
about
what
is
ml
flow
in
a
quick
three
bullets
line.
So
no
flaw
is
an
open
platform
that
helps
you
manage
a
machine
learning
development
lifecycle
and
it
does
it
in
three
ways:
one:
oh,
we
have
lightweight
ABI
to
be
able
to
work
with
any
ml
or
library.
B
So,
as
you
talked
about
as
like
everybody
has
their
favorite
library
to
use-
or
in
fact
you
want
to
try
out
different
libraries,
they
could
be
in
different
programming
languages.
The
key
aspect
that
we
saw
here
was
data
scientists
and
machine
learning.
Practitioners
don't
want
to
get
locked
in
to
one
particular
library
or
one
particular
language.
B
You
want
to
sort
of
be
able
to
use
any
kind
of
language,
so
that's
where
we
build
ml
flow
as
an
API
first
approach,
so
you
can
talk
to
ml
flow
as
components
using
REST
API
of
Python
Java
and
an
are
they
all
built
up
on
this
basic
REST
API
and
lets
you
interact
with
them
a
flow.
The
second
thing
we
wanted
to
do
is
we
wanted
to
make
it
a
make
reproducibility
of
the
runs
a
primary.
B
So,
for
instance,
you
have
you
typically
train
your
model
on
your
local
machine,
and
then
you
want
to
sort
of
make
sure
that
it
reproduces
the
exact
same
results
when
you're
running
it
on
a
cloud
somewhere
any
kind
of
cloud
platform.
What
makes
it
hard
is
getting
get
that
reproducibility
again,
if
you,
if
you
send
your
code
to
some
other
engineer
on
your
team,
they
should
be
able
to
do
it
the
exact
same
way.
So
having
the
having
your
ml
sessions
run.
B
The
exact
same
way
on
any
cloud
was
one
of
the
other
development
principles
of
he
principles.
Behind
building
Emma
flow
and
then
finally,
simplicity
and
ease
of
wrapping
up
was
the
other
goal
such
that
it
should
be
useful
for
one
engineer
on
an
org,
but
also
easy
to
scale
up
to
a
thousand
people,
or
so
the
scale
and
ease
of
use
was
also
sort
of
thought
in
when
we
sort
of
designed
that,
okay,
so
from
here,
the
ml
flow
supports
like
three
different
components.
B
To
begin
with,
one
is
the
ML
flow
tracking
server
we're
going
to
spend
some
time
talking
about
that.
It's
necessarily
a
centralized
repository
to
store
all
the
critical
information
that
is,
that
is
generated
from
and
required
to
generate.
Your
machine
learning
run
when
this
could
be
taken
with
the
parameters
and
configs
and
and
and
also
the
metrics
that
come
out
right.
So
that's
that
and
then
it
has
mechanisms
to
query
that.
So
it's
like
a
like
a
database
for
all
your
machine
learning
runs
the
second
one.
B
Mlpro
projects
is
a
code
packaging
format
and
that
is
targeted
towards
making
your
runs
reproducible
when
you
run
it
on
any
cloud
platform
or
at
any
point
of
time
later
later
on.
So
this
is
a
way
to
sort
of
store
everything
such
that
you
can
guarantee
that
you
are
reproducing
that
runs
and
anywhere,
and
anybody
else
can
do
it
as
well
and
the
third
one
is
a
anode
models
is
a
generate
of
format.
B
A
packaging
format
for
models
such
that
the
models
that
are
once
written
out
by
ml
flow
can
be
deployed
across
a
variety
of
production
platforms.
May
it
be
excuse,
me
may
be
in
real
time
scoring
format,
or
it
could
be
like
that
scoring
or
or
streaming
platforms.
So
that's
that's
the
three
big
components
and
then
I'm
going
to
spend
a
few
minutes
talking
about
each
one
of
these.
B
So
let's
jump
right
into
the
key
concept
of
tracking
ml
for
tracking.
We
talked
about
tuning
machine
learning
your
algorithm,
so
parameters
which
could
be
like
a
numeric
or
string
parameters,
are
a
key
to
sort
of
making
sure
you
get
what
you
want
out
of
be
our
machine
learning
one
data
scientists
sort
of
try
out
thousands
of
experiments
with
the
same
algorithm
to
sort
of
get
the
right
right.
B
You
get
the
right
results
that
they
are
looking
for.
So
one
of
the
key
concept
is
a
dictionary
of
parameters
that
might
be
associated
with
your
particular
one.
Another
thing
is,
after
your
run
is
done
you
you
generate
numeric
values
by
scoring
you
or
on
a
test
data.
So
what
are
the
metrics
accuracy
error
so
on
so
forth?
That
becomes
the
key
concept
that
the
tracking
server
might
want
to.
The
engineer
might
want
to
keep
track
of
in
this
consolidated
repo
and
then.
B
Finally,
if
you
start
looking
at
your
models,
the
bottle
generated
from
the
attract
training
run
or
could
be
itself
an
artifact
that
you
might
want
to
store
away
to
keep
track
or
governance,
or
even
use
it
port
for
scoring
later
on.
Along
with
that,
you
want
to
keep
track
of
metadata
around
the
or
around
of
what
was
metadata
around
how
the
model
was
called.
What
source
code
was
used?
The
exact
version
maybe
get
our
ID
get
hash.
B
These
are
some
of
the
interesting
things
you
may
want
to
store
around
along
with
your
training
data,
and
then,
finally,
they
could
be
some
texts
that
you
might
want
to
store
some
document
that
might
want
to
capture
the
details
of
your
model,
and
for
that
we
have
this
high
level
tag
or
notes
that
that
you
can
record
for
your
training
run
and
then
at
the
bottom
right.
You
see
a
UI
that
sort
of
shows
how
it
would
show
up
on
an
ml
flow
UI.
B
This
is
the
query
layer
that
I
was
referring
to
where
you
have
an
ability
to
show
all
your
runs
and
then
using
this
sequel
like
syntax
query
that
run
for
metrics
and
parameters
and
sort
of
slice
and
dice
your
data
based
on
specific
metrics,
for
example,
you
can
say
metrics
tour
accuracy
is
greater
than
point
98
or
so
and
so
forth.
Okay,
so
how
do
people
use
this
consolidated
storage?
B
So
your
ml
for
tracking
server
as
I
said,
is
a
database
and
you
could
have
various
people
write
into
this
database
for
each
training
one
either
through
some
hosted
notebooks
or
a
local
app
running
on
your
machine
or
a
cloud
job
right.
So
they're
writing
to
this
centralized
database
through
REST,
API
or
Python
API,
and
and
that's
how
they're,
storing
as
as
we'll
see
how
it
is
done
and
then
you
are,
we
have
built
a
UI
layer
and
then
there
is
an
API
layer
to
be
able
to
query
that
database.
B
So
in
this
picture
we
sort
of
abstracted
away
the
database.
We
started
off
when
we
opened
up
when
we
a
release
demo
flow
using
a
file
back
store,
and
now,
if
you
have
various
different
stores
will
soon
be
releasing
a
seek,
will
be
a
store
to
sort
of
store,
all
the
metadata
around
runs
and-
and
so
you
can
query
that,
using
again
as
a
UI
that
we've
built
or
using
python
api
all
right.
Oh
now.
B
Finally,
how
does
this
look
like
the
lightweight
api
I
was
talking
about?
It
starts
off
with
very
simple.
You
start
a
run
and
then
you,
you
can
record
the
parameters
that
are
super
important
for
the
particular
run,
and
you
can
see
that
highlighted
in
green
here
and
then
you
can
paste
in
your
training
algorithm
and
at
the
end
of
it,
you
are
tell
you,
you
compute
some
metric
based
on
some
test
data
and
you
can
law
that
metric
or
you
can
actually
log
with
the
artifact
or
some
higher
level.
B
You
can
walk
the
model
or
some
high
level
artifact
like,
for
instance,
a
lot
of
symmetric
data
and
that
can
be
logged
as
well,
and
then
you
in
this
particular
example,
the
model
happens
to
be
a
chance
of
work
graph.
So
we
are
logging,
it
like
a
tensor
flow
graphs
and
the
flow
has
these
packages
within
it.
To
understand
that
hey.
This
is
a
tensor
flow
graph
and
we
write
it
as
a
model.
So
any
kind
of
tool
that
understands
the
tensor
flow
graph
can
use
that
for
deployment.
B
So
with
very
simple
API
that
surrounds
your
existing
or
training
algorithm,
you
can.
You
can
now
very
easily
use
a
tracking
server
and
whatever
happens
during
this
run,
gets
logged
to
the
tracking
server
all
right,
so
that
was
a
tracking.
Now,
let's
talk
about
projects
now.
As
a
reminder,
this
is
again
a
mechanism,
a
code,
a
packaging
mechanism
to
make
it
easy
for
you
or
anybody
on
your
team
to
reproduce
your
runs
on
any
cloud
platform
anytime
later.
B
So
what
are
the
things
that
we
need
to
be
able
to
do
that?
So
in
this
packaging
format,
we
obviously
need
to
include
the
code
that
is
needed
for
training.
You
may
also
need
some
configs
and
some
pointer
to
data,
so
this
effectively
constitutes
of
what
would
be
needed
to
be
able
to
reproduce
the
run,
whether
it
is
running
it
locally
on
your
somebody
else's
local
machine
or
running
it
on
some
remote
cloud.
B
But
then
how
exactly
does
this
packaging
format
look?
Like
there's
an
example:
it's
necessarily
a
directory
structure
where
you
have
this
file
called
ml
project
and
if
you
look
at
the
file,
use
an
example
of
a
very
small
version
of
the
file.
It
right
of
the
bat
tells
you
that
it
means
a
Khanna
environment
right.
So
this
is
the
current
environment
that
the
data
scientists
use
to
train
their
model
and
the
content
is
a
Yama
file
that
is
included
within
that
directory
structure.
B
And
then,
if
you
look
at
the
next
thing,
it
talks
about
one
entry
point,
a
main
entry
point
for
being
able
to
run
this
project,
and
it
says
it.
Ok,
you
need
these
parameters,
which
is
a
path
for
the
training
data
and
a
float
parameter
called
lambda,
and
it's
here
it's
defined
as
a
default
value
of
0.1,
oh
and
then
how
does
ml
flow
know
what
to
do
when
it
sees
that
entry
point?
B
It
says:
okay,
it's
runs
of
a
simple
Python
command
called
me
in
dot
pi,
and
it
gives
it
the
training,
data
and
lambda,
and
obviously
this
is
the
code
main
that
PI
and
then
there
could
be
other
dependent
modules.
Included
of
that.
This
is
the
code
that
you
have
written
to
create
your
training
rod.
So
pretty
simply
when
you
write
a
training
algorithm,
you
include
this
within
that
restructure,
create
a
simple
ml
project
file
and
capture
your
condo
environment
and
boom.
B
Anybody
is
able
to
then
reproduce
your
run
using
a
simple
couple
of
mo
slow
commands
that
can
just
take
your
existing
code,
use
their
data,
use
their
parameters
and
we
and
in
create
a
run
or
you
can
reproduce
your
run.
So
how
does
somebody
do
that
through
a
command-line
ml
flow
space,
wrong
space
I'll
get
how
people
or
it
can
be
a
link
to
this
local
directory
structure?
If
you
want
to
do
this,
programmatically
there
is
I'm
also
run
a
Python
command
to
do
that.
B
So
for,
let's
take
an
example
where
excuse
me
wait,
you
have
created
a
model
and
it
could
be
like
a
tensor
flow
graph
model
that
in
your
local
environment,
and
then
you
want
to
write
it
out
in
such
a
format
that
it
can
be.
They
can
be
loaded
on,
let's
say,
Amazon
sage
maker
and
then
also
in
some
kind
of
environment
that
runs
Python.
B
So
this
would
be
a
way
of
saying
that
the
same
model
that
is
used
by
one
team-
for
that
say
bad
scoring,
is
also
used
banner,
the
team
for
some
real-time
scoring
and
it
uses
the
same
model
that
is
being
written
by
a
mouse
law.
So
how
does
this
look
like
again?
This
is
a
packaging
format,
though
the
format
will
be
very
similar
to
what
you've
seen
and
projects.
It
is
animal
model
file,
but
it
has
a
few
things.
B
B
Excuse
me
that
can
run
a
tensor
flow
format,
would
need
to
be
able
to
this
utilize
this
model
and
make
it
executable
and
then
any
environment
that
uses
Python
can
use
this
ml
flow,
Mart
module
column
L
for
a
tensor
flow
to
be
able
to
then
load
this
model
module
and
then
execute
it
like
like
it
would
with
any
Python
function.
Again,
it
all
depends
on
making
sure
that
all
the
dependencies
are
in
play.
B
So
how
can
you
learn
about
a
little
aspect?
So
this
was
a
little
flavor
of
how
a
mil
flow
looks
like
there
is
a
lot
more
in
terms
of
documentation
and
n
training
examples.
If
you
want
to
do
some
type
of
parameter,
tuning
multi-step,
workflows
for
forests
and
bat
scoring,
you
can
start
off
with
installing
ml
flow
through
PI
bi
or
you
can
download
it
through
the
github
repo
and
and
try
it
out.
So
the
point
the
key
point
to
go
to
would
be
ml
flow
org
that
hosts
the
website
and
has
all
the
examples.
B
Okay,
in
the
last
few
minutes,
I
want
to
just
talk
about
the
open
source
community
growth
and
how
ml
flow
that
started
off
about
eight
months
back
has
has
been
well
at
ppreciate,
annex
and
an
embrace
by
community.
So
this
is
a
quick
I
wanted
to
use
for
metrics
to
sort
of
show
that
it
started
months
ago
and
was
released
during
a
sparkle
CI
summit
in
June.
B
B
Look
at
ml
floor
org
and
there's
a
Google
Form
for
this
survey.
We
want
to
know
if
you
have
used
a
more
slow
and
if
you
have
not,
what
sort
of
things
are
you
looking
forward
to
in
that
I
want
to
know
more
about
what
what
what
makes
what
were
the
requirements
that
you're
looking
for
and
what
would
make
it
easy
for
you
to
use
ml
flow.
B
B
Currently
we
are
at
version
0.83,
and
we
want
to
sort
of,
as
we
start
going
to
0.9
and
1.0,
they
sort
of
thinking
of
adding
in
point
nine
we'll
be
adding
a
sequel,
backed
store
for
tracking
server.
This
is
one
of
the
most
frequently
asked
requests,
and
then
we
adding
flu
and
Java
and
Scala
API
a
lot
of
UI
scalability.
B
B
A
lot
of
requests
came
around
that
okay,
this
I
want
to
use
a
model
I'm
going
to
save
it,
and
then
I
want
to
sort
of
have
inject
some
custom
code.
It
could
be
sort
of
a
querying
form
of
feature
store
or
it
could
be,
like
you
know,
transform
suite
features
a
little
bit,
so
the
sort
of
custom,
logging
and
and
custom
code
is
something
that
is
being
worked
on
actively,
instead
of
releasing
at
point
nine
and
one
point
out.
B
B
You
can
already
do
that
using
the
ml
project
and
you
can
have
multiple
entry
points,
but
but
it
would
it
make
sense
to
have
a
UI
to
sort
of
edit,
your
modifier
multi
but
ISTEP
workflow,
and
then
we're
looking
at
adding
some
telemetry
components
or
logging
data
and
metrics
and
taking
you
back
into
some
analytics
tools.
So
these
are
they're
looking
for
feedback
around
that
so
again,
another
plug
for
the
survey
on
ml
flora
org.
B
We
want
to
know
how
you
use
your
mo
floor,
libraries
and
frameworks
and
how
what
would
you
like
for
us
to
add
as
integrations
in
ml
flow?
So
that's
the
survey.
Thank
you
all
for
your
time.
I
want
to
just
sort
of
again
put
out
ml
soda
or
join
us
on
slack
or
there's
an
email
thread.
You
can
join
and
thank
you
for
giving
me
the
time
to
talk
at
the
summit.
If
anybody
wants
to
job
to
come
to
spark
some,
it
here
is
a
code
that
wanted
to
sort
of
share
with
you
all.
B
A
A
C
C
C
What
are
these
things
called
hyper
parameters,
as
many
pointed
out
in
his
presentation,
you
know:
well,
when
we're
creating
machine
learning
models
will
be,
you
know,
whoa.
We
may
want
to
do
some
hyper
parameter
tuning
right,
so
when
we,
when
we're
creating
mo
models,
you'll
be
presented
with
different
ways
on
how
to
define
your
model
architecture
in
the
beginning.
You
don't
know
what
the
you
don't
know.
What
the
optimal
architecture
should
should
look
like
for
your
model.
C
You
would
need
to
do
multiple,
runs
and
test
your
your
model
with
multiple.
You
know
different
different
parameters
to
find
the
optimal
model
architecture,
so
the
so.
The
parameters
that
we
define
the
the
model
architecture
are
called
hyper
parameters
and
the
process
of
searching
for
the
ideal
model
architecture
is
called
hyper
parameter
tuning.
So
let
me
try
this
back
up
to
the
use
case
here
so
applying
this
to
to
our
use
case.
C
Let's
say:
let's
say:
if
we
have
an
unsupervised
machine
learning
problem
our
machine
learning
technique
such
as
means,
so
you
might
ask
what
is
k-means
k-means
is
used
to
partition
data
points
in
K
clusters
and,
let's
suppose
K
is
3.
Let's
say
we
are
three
clusters
and
we
have
ten
data
points.
What
k-means
does
is
it
takes
that
takes
the
features
of
the
ten
data
points
and
assigns
each
point
to
either
a
cluster
one
two
or
three,
and
the
data
points
that
are
similar
will
be
grouped
under
one
cluster.
C
So
that's
more
into
the
use
case.
You
know,
but
you
know,
mo
life
cycle
is
the
complex
thing.
We
wanted
something
off
the
shelf
open
source
that
we
could
pull
in.
We
we're
running
on
open
shift,
which
is
our
distribution
of
kubernetes.
So
we
wanted
something
that
runs
on
open
ship
to
do
things
like
this
training,
the
model
doing
hyper
parameter,
building
versioning
we
found
mo
flow
great
tool
that
has
these
all
these
great
features
and
then
tracking
experiments.
C
C
C
C
So
so
we
have
open
shift
here.
We
have
our
operator
here
mo
flow
tracking
operator,
so
I've
already
deployed
it.
We
have
one
instance
of
the
server
and
we're
gonna.
We
have
mo
flow
here,
that's
already
running
so
that
one
instance
is
there
it's
pointing
to
an
s3
bucket
called
Zack
Hasan
and
that
s3
bucket
is
here
in
Seminole,
okay.
So
what
I'm
gonna
do
is
I'm
gonna
run
this
this
code
here,
which
is
the
standard
example,
that's
on
mo
flow
website,
but
I'll
just
point
you
to
where
the
code
lives.
C
Which
is
here
it's
just
a
standard
example
where
we're
using
a
standard
example
and
it's
using
the
same
MO
flow
api's.
It's
logging,
the
parameters,
it's
logging,
the
metrics
and
then
it's
logging
storing
the
model
into
the
server
and
it's
choosing
different
parameters.
And
then
we
can
compare
these
three
different
runs.
But
let's
without
further
ado,
this
is
our
our
UI
we're
gonna
refresh
just
so,
there's
no
smokes
and
mirrors
we
can
see.
There
is
no
runs
in
there
right
now.
C
So
if
I
go
ahead
and
do
a
run,
so
we
already
did
a
build
of
a
done
that
get
repo.
We
built
that
already
it's
already
here,
we're
going
to
inject
a
secret
that
secret
is
is
here
since
we
don't
want
to
store
our.
We
don't
want
to
ask
users
for
their
AWS
access
key
their
secret.
We
kind
of
want
to
keep
it
in
secrets,
and
then
these
secrets
would
get
injected
into
the
container
okay.
C
C
C
C
C
So,
just
to
wrap
this
up.
What's
next
for
this,
so
the
next
thing
we
did
was
we:
we
created
a
PR
in
the
mo
flow
repo
and,
like
all
things,
open-source,
it
always
starts
with
the
PR,
and
we
have
a
PR
in
ml
flow
and
we're
contributing
this
this
operator
to
the
mo
flow
community
and
collaborating
there,
and
that's
that's
pretty
much
it
so.
A
A
A
I'm
muting
everyone-
you
all
are
now
free
to
unmute
yourself
and
ask
questions.
I
have
one
question
for
Zac
about
your
operator
and
that
is:
are
there
any?
Is
there
anything
any
dependencies
on
the
openshift
distribution
of
kubernetes?
Or
could
this
just
run
nice
nicely
on
any
standard,
kubernetes
distribution.
C
A
So
what
what
are
it
sort
of
the
next
steps
that
you
need?
I
would
love
to
see
this
by
the
way:
I
cuz,
I'm,
totally
biased
and
and
I
put
the
link
in
there
to
contribute
it
to
operator
of
Daioh.
So
if
you
can
take
a
look
at
that
link
and
see
if
we
can
get
it
in
there
anytime
soon,
that
would
be
awesome
from
my
point
of
view
and
it
would
give
some
more
visibility.
Ml
flow
and
folks
at
data
bricks
are
doing
and
I'm
just
wondering,
but
is
this?
C
I'd
like
to
compile
it
it
first
off
it
get,
it
probably
have
more
rigorous
testing
and
you
know,
run
it
on
more
current
kubernetes.
Perhaps
certain
google
cloud
and
other
different
cloud
providers
and
see
how
you
know
have
it
running.
There
probably
have
some
documentation
on
how
to
run
it
on
Google,
Cloud,
IKS,
Azure
kubernetes.
It
should
all
run
all
perfectly
fine
there.
C
It's
just
I
should
I
want
to
try
to
run
it
up.
There
also
I
want
to
get
feedback
from
the
ML
flow
community.
There
are
multiple
users
and
have
them
try
it
out,
maybe
run
it
on
mini
shift
or
mini
cube
and
and
and
get
feedback
there,
and
then
we
can
continuously
improve
it
and
make
it
better.
So
I
think
it's
it's
it's.
They
should
be
able
to
be
used
today.
We
should
be
ready
to
use
today.
Okay,.
A
Well,
that's
good
news
and
I
think
if
we
can
get
it
into
operator
hub
do
and
maybe
money
if
we
can
send
an
email
out
to
your
with
the
videos
from
today
and
the
links
to
the
mailing
list.
We
we
ought
to
be
able
to
get
you
some
more
eyeballs
on
the
codebase,
and
maybe
you
won't
have
to
be
doing
all
of
the
testing
yourself
back,
though
that's
one
of
my
hopes
here.
A
D
A
D
Do
know
that
there
are
a
lot
of
like
you
mentioned.
It
was
great
to
know
that
there
are
new
versions
of
ml
flow.
Coming
with
with
some
of
the
features
that
you
mentioned,
and
the
only
questions
some
of
the
questions
I
had
was,
as
I
was
testing
out
ml
flow.
Let's
say
we
have
like
three
or
four
data
scientists
who
are
working
and
testing
different
kind
of
models.
One
is
running
k-means.
Someone
else
is
running,
you
know
k-means
plus
+
or
some
spectral
clustering.
D
So
when
we
run
the
ml
flow
server,
which
is
a
one
server
which
is
shared
by
multiple
users,
is
there
a
way
to
sort
of
identify
or,
if
we're
all
pushing
it
under
one
experiment?
Is
there
a
way
to
sort
of
analyze
this,
or
do
we
all
just
sort
of
push
it
under
one
particular
experiment?
I
d--
or
give
it
a
name?
B
That's
a
really
good
question
email,
so
I
I
can
answer
it
a
couple
of
ways.
So
an
experiment,
if
you
start
thinking
about
it
is,
is
your
way
of
organizing
different
runs.
So
let's
say
the
all
you
four
or
five
data
scientists
are
working
on
the
same
project
and
you
want
to
sort
of
work
off
of
each
other's
runs.
You
want
to
sort
of
share
your
data
with
each
other
and
you're
related
to
one
exact
problem
that
you're
solving.
Then
it
actually
does
make
sense
to
wrap
them
up
as
one
experiment.
B
Okay
and
then
you
all
are
submitting
your
runs
to
the
exact
same
experiment,
and
then
they
will
be
recorded
as
different
different
runs.
That
are
because,
if
you
look
at
it
the
runs
are
you
you
IDs,
so
you
can
then
have,
and
in
the
in
the
UI
they
will
be
recorded
with
your
specific
user
ID
as
to
who
recorded
it.
So
it's
very
easy
to
sort
of
do
that.
Now.
B
It
might
get
a
little
harder
to
compare
across
those,
but
if
you
really
need
them
all
to
be
together,
Emma
for
supports
that,
if
you
prefer
to
have
them
as
different
experiments,
you
can
just
create
different
experiments
and
and
and
store
that
so,
for
example,
let's
say
all
of
you
working
on
some
eric,
like
tricking
tracking
or
something
like
that,
and
you
want
to
sort
of
have
two
different
experiments
for
two
different
class
of
models.
Right,
let's
say
one
is
a
deep
learning
model.
B
If
you
want
to
go
a
traditional
model,
it's
going
to
the
other
one,
so
it
all
depends
on
how
you
want
to
use
it
ever
flow
supports.
All
of
this,
as
I
said,
it
is
designed
to
scale
across
a
lot
of
users.
In
fact,
yeah
a
lot
of
users
that
use
a
muscle,
open
source.
Have
it
such
that
the
servers
running
off
of
an
ec2
instance,
the
data
stored
on
s3
and
and
a
bunch
of
people.
Dozens
of
people
in
the
organization
are
reporting
the
data
to
the
same
server.
A
I'm
just
looking
to
see
if
there
are
any
other
questions,
we're
almost
to
the
top
of
the
hour,
so
I
always
love
it
when
I
think
I'm,
giving
everybody
15
minutes,
but
you
just
blow
it
out
of
the
water
and
then
use
the
time
wisely.
So
I'm
going
to
thank
everybody
for
their
time
today
and
we'll
there's
a
couple
of
things
that
I'm
looking
for
people
to
add
into
this
the
notes
here.
A
So
if
you
can
link
in
your
slides
or
send
me
a
PDF
version
of
your
slides
I'll,
send
it
out
through
the
mailing
list,
as
well
as
post
it
on
a
blog
on
OpenShift
comm
with
the
video
and,
if
there's
a
topic
that
someone
who's
listening
today
wants
to
talk
about
at
our
next
meeting,
which
is
on
April
5th.
Please
just
reach
out
to
myself
and
again
at
D
Mueller
at
Red,
Hat,
comm
or
via
the
mailing
list,
and
we'll
add
you
to
the
agenda
for
the
next
one.
A
So
of
note,
there's
a
couple
of
things
coming
up:
okay,
they're!
All
just
thank
yous
you're!
All
welcome
I'm,
really
thrilled
to
host
this,
because
I
learned
something
new.
Every
day
with
the
with
the
stuff,
the
guys
are
sharing.
So,
as
I
mentioned
at
the
beginning,
they're
on
March
11th,
there's
going
to
be
an
open
ship,
Commons
gathering,
which
is
a
number
of
the
videos
coming
to
talk
about
their
inference.
A
Work
on
video
gpus
and
deep
learning,
they're
boomers
coming
to
talk
about
their
M
3db
operator
and
there's
going
to
also
be
an
operator's
workshop
hands-on
in
the
afternoon
in
a
second
room,
parallel
track.
So
if
you're
in
the
area
for
that
come
for
that,
as
well
as
I'll,
add
in
the
link
here
or
they,
the
SPARC
summit,
and
thank
you
Manny
for
the
data
bricks
discount
and
if
there's
any
other
events
coming
up.