►
From YouTube: IncEng MLOps Update - 2022-08-29
Description
ML Experiment Tracking: What it is, what we are not doing, what we are doing and current progress
GitLab ML Experiment Tracking SSoT: https://gitlab.com/groups/gitlab-org/-/epics/8560
This update: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/61
All Updates: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/16
A
Hello,
everyone
and
welcome
to
another
update
for
occupation
engineering
envelopes
and
today
we're
going
to
talk
about
machine
learning,
experiment
tracking.
Let
me
first
go
and
explain
exactly
what
is
machine
learning
experience
tracking
before
we
move
on.
So
when
a
data
scientist
creates
a
machine
learning
model,
there
are
many
things
that
can
influence
the
final
performance,
some
of
them,
including
parameters,
the
algorithm
being
used
for
learning.
The
algorithm
itself,
is
a
hyper
parameter
that
we
called,
or
even
the
data
used
and
over
the
progress
of
the
project.
A
It's
common
for
data
scientists
to
lose
track
of
all
the
possible
configurations
that
they
used
for
a
specific
trial
that
they
had
so
they
create
a
model,
but
they
don't
know
how
to
replicate
that
because
they
lost
track
of
what
was
the
specific
configuration
they
had.
So
experiment
tracking
is
all
about
this.
A
It's
about
helping
users
have
open
data
scientists
to
keep
to
log
the
all
the
parameters
used
for
a
specific
trial
or
for
a
specific
candidate
and
an
experiment
is
a
set
of
comparable
candidates
that
they
have
an
experiment
can
be
many
things,
but
in
gitlab
it
could
be
a
merge
request.
It
could
be
a
commit,
a
commit
that
will
trigger
an
experiment
or
just
they
explore
like
a
data
scientist
exploring
on
their
local
environment
as
well,
so
the
biggest
alternative,
the
the
most
common
solution
for
this
right
now
is
ammo
flow.
A
I
have
spoken
about
this
dim
already,
so
it's
a
common
open
source
tool
very,
very
popular,
and
we
want
to
help
users
that
already
use
mflow
to
use
gitlab
as
well.
So
let
me
just
start
by
saying
what
we
are
not
doing.
We
are
not
packaging
emma
flow
inside
gitlab.
A
This
was
discussed
as
a
potential
solution,
but
it
wouldn't
lead
to
the
best
user
experience
for
our
self-managed
customers
and
the
complexity
of
deploying
on
the
way
gitlab
does
would
not
be
worth
it.
So
what
we
are
doing
is
that
we
are
implementing
the
tracking
server
native
to
git
lab.
So
when
you
go
to
your
project
project
page,
you
will
find
an
a
and
a
tab
for
experiments
and
over
there
you
can
track
your
experience,
so
I'm
employing.
Why
is
git
laby?
A
It's
git
lab
expert
tracking,
but
we
are
providing
full
support
for
emflo
rest
api.
So
if
you
use
mflow
client,
you
just
need
to
swap
the
tracking
your
eye
to
gitlab
and
that's
it
that's
all
you
need
to
do
to
use
gitlab.
Instead,
we
on
that
note
progress.
We
are
ready.
We
have
now
nmr
for
the
three
first
endpoints
create
experiment,
get
experiment.
I
gotta
experiment,
my
name,
we're
focusing
on
demo
flow
compatibility
right
now
and
we
automated
the
documentation
for
the
difference.
A
So
here
you
have
the
same
request
being
called
both
on
gitlab
and
mflow
and
the
difference
between
the
results.
So
you
know
what
you
expect
when
you
switch
between
one
and
another
up
next
implement
the
remaining
eight
endpoints
to
make
this
script
work.
This
is
our
mvp
make
this
script
work.
This
is
example
trip
of
mfo
track.
They
use
mfo
client
to
track
an
experiment.
A
There
are
eight
remaining
endpoints
to
do
so
and,
furthermore,
the
front
end
necessary
and
the
documentation
we
are
on
track
for
15.5.
A
If,
okay
in
case
you
have
any
feedback
you
want
to
discuss,
you
want
to
know
where
we
are
going.
The
this
epic
8560
is
our
source
of
truth
and
we
will
post
a
link
on
below
on
this
video.
This
is
where
discussion
will
happen
and
where
we
will
post
all
directions
and
updates
that
was
it
for
today.
Thank
you
very
much.