►
From YouTube: MLOps SEG Update Mar 10th 2022
Description
Let's check what we we have accomplished so far on our four objectives.
- Toggleable Notebook Diffs on Commit page
- Citer
- Planning for Pipeline with Stubbed Jobs
- Hands On Machine Learning Bookclub
Text version: https://gitlab.com/gitlab-org/incubation-engineering/mlops/meta/-/issues/47
A
A
Our
vision
for
this
sag
is
to
make
it
lab
a
tool
that
a
scientist
and
machine
learning
engineers
love
to
use
not
have
not
need,
but
love,
and
within
that
this
vision
we
set
ourselves
to
four
basic
objectives
in
the
beginning,
so
one
deliver
rendered
notebook
divs
to
explore
better
experience
for
pipelines,
data
science,
three
be
an
ambassador
for
data
scientists
and
machine
learning
engineers
and
four
explore
the
idea
of
having
an
analytics
rapport
within
gitlab
itself.
So
we're
talking
about
what
we
did
and
where
we're
going
to
within
these
four
objectives.
A
Now
this
seems
a
little
bit
weird
that's
in
here,
but
I
am
in
the
firm
opinion
that
the
better
we
at
gitlab
understand
machine
learning
and
and
train
our
software
engineers,
the
engineers
on
machine
learning,
the
better
one.
The
more
often
we'll
find
nice
and
new
case
use
cases
for
machine
learning,
name
to
improve
the
our
product
to
the
users,
but
also
it
will
increase
empathy
to
our
towards
our
users,
the
better.
A
We
know
how
to
use
machine
learning
the
better
with
dog
food
machine
learning
within
the
lab,
the
better
we
can
provide
products
for
machine
learning,
engineers
and
data
scientists,
and
the
second
point
over
here
the
mlms
community
is
a
large
slack
based
community
for
demo,
ops
enthusiasts,
I
created
a
channel
over
there,
so
I
can
discuss
directly
with
the
community
about
machine
learning
topics
or
how
to
use
gitlab
within
machine.
How
to
use
skill
gitlab
for
machine
learning
also
to
share
my
updates
and
ask
feedback.
A
The
third
objective
is
jupiter.
Notebooks
render
them.
This
is
what
I've
been
working
the
most
for
for
a
while,
and
we
have
delivered
on
now,
a
detectable
diffs
for
commitment.
Talkable
div
means
users
can
also
swap
between
the
derivative,
the
regular
one
and
the
cleaner
gift
that
we
implemented
on
30.5.
A
This
is
important
because
when
we
create
the
cleaner,
we
might
clean
things
that
are
actually
important
so
having
the
tuggable
improves
user
experience
for
them.
It
just
works
in
the
community.
It
works
on
commit
page,
it's
enabled
for
50
of
the
projects
and
we
are
working
on
polishing
and
adding
this
to
the
mark
page
as
well.
A
A
Fourth,
objective
pipelines
for
this
use
case
and
I'm
going
to
I'm
going
to
release
this
very
often
and
because
it's
important
in
envelopes
pipelines
are
part
of
the
crane
stage.
You
can
think
about
pipe
train
a
model
as
compiling
an
app
so
trainer
model
international
model
is
the
same,
follow
the
same
iteration
process
as
compiling
a
software
or
an
app
or
whatever,
but
it
takes
sometimes
training
a
model
can
take
hour
minutes
days
hours.
A
Sometimes
you
to
parallelize
this
kind
of
thing
so
pipelines
they
enter
the
envelopes
process,
not
in
the
verify,
but
on
the
create
stage,
and
since
it
is
a
great
stage,
it
is
really
really
important
that
we
focus
on
making
iteration
on
pipelines
easier,
so
make
it
faster
to
change
data
to
to
bring
something
new.
So
I'm
going
to
talk
about
two
things
over
here
that
are
interesting,
one
that
we
already
delivered.
We
delivered
a
poc
for
running.
A
They
already
spoke
about
about
this
on
the
last
update,
but
we
delivered
a
poc
for
running
a
pipeline
without
anything
being
on
ticket
rapport.
So
this
completely
bypasses
the
git
commit
get
pushed
please
give
commit.
Please
work
this
time,
get
push
and
run
and
look
for
the
pipeline,
and
everything
like
that.
Now
it
works
on
the
on
the
command
line.
A
It
uploads
a
completely
arbitrary
pipeline.
That
is
not
a
repository
with
arbitrary
code
that
started
a
project
or
repository
runs
the
pipeline
on
the
rippon
gitlab
itself.
It's
not
a
local
runner,
it
uses
gitlab
runners
and
it
makes
a
lot
faster
to
iterate
on
this.
So,
if
you're
recording
a
new
pipeline,
it's
really
really
quick
a
lot
quicker.
You
reduce
a
lot
of
the
of
the
downtime
waiting
for
stuff
to
finish,
but
it's
very
hacky.
I
wouldn't
recommend
to
use
this
in
production.
A
It
is
it's
just
showcased
this
idea
of
showcase
how
it
could
be
the
experience
that
we
could
have
with
gitlab
pipelines
on
this
area,
so
I'm
still
trying
to
find
ways
and
how
we
could
approach
this
for
including
directly
into
product,
but
it
is
possible.
It
is
certainly
possible
and
you
for
more
interest.
If
you
are
interested
about
this
one,
the
last
update
was
all
about
this,
so
yeah
head
over
there.
I
say
I
go
for
10
minutes
almost
over
this
and
now.
A
What
do
I
mean
by
that
suppose
that
you
have
this
pipeline
suppose
that
it
has
five
steps
and
then
suppose
that,
for
example,
b
or
c
are
really
long,
but
t
is
really
fast
and
here
really
fast
and
suppose
that
you
are
working
this
pipeline
and
then
you
have
a
works
b,
you
work
c
works
and
then
it
breaks
on
d
now
regularly.
What
you
would
do
is
fix,
b,
fix
d
and
then
have
to
rerun
everything
again.
So
imagine.
A
This
pipeline
takes
three
hours
to
run,
which
is
not
uncommon
for
machine
learning
use
cases
and
what
you're
just
iterating
over
dne.
What?
If,
instead
of
doing
this
running
everything
again
for
the
new
commit?
What
if
we
could
just
run
d
and
e
from
the
left
using
the
e
stubbing,
the
input
of
d
with
the
last
successful
run
of
c.
A
So
these
are
different
commits
right.
I'm
using
abhc
that
was
run
from
for
a
previous
commit
as
a
stabbing
into
the
this
can
lead
to
problems.
This
can
lead
to
inconsistency,
but
it
can
really
improve
the
time.
The
the
the
the
iteration
time
when
you
are
working
on
dna.
This
wouldn't
be
used
on
a,
for
example,
a
production
pipeline
on
a
pipeline
that
it's
like
on
a
protected
branch
or
something,
but
if
you're
iterating
either
creating
a
new
pipeline
or
you're
working
on
your
code
base
and
then
some
test
fails.
A
For
example,
you
don't
need
to
run
everything
again.
You
can
try
to
just
fix
that
specific
job
and
then
afterwards
run
everything
again
think
about
as
running
tests.
So
when
you,
when
a
test,
fails,
you
don't
keep
running
the
entire
test
that
suit
all
the
time
you
keep
fixing
you
keep
working,
fixing
that
specific
test.
Then
you
run
all
tests
and
then
you
figure
out
okay.
Now
this
is
wrong
and
then
you
go
into
the
next
test.
You
don't
keep
running
your
entire
test
suit.
A
All
the
time
same
thing
applies
here
we
could
run
just
the
failed
pipelines,
but
this
is
a
very
complex
problem.
A
lot
more
complex
than
the
notebook
devs.
So
what
we're
doing
this
is
what
I
mean
instead
of
using,
instead
of
creating
the
whole
pipeline,
we
can
just
connect
them
and
stop
the
input
of
d2
in
this
case,
with
the
output
of
c1
understanding.
What
complications
might
arise
from
that,
of
course,
but
the
potential
is
really
big
here.
A
So
we
are,
this
is
a
very
complicated
problem
and
we
expect
not
to
work
on
this
until
we
finish
notebook
tips,
but
meanwhile
sees
the
difficulties
that
arise
from
here.
We
are
already
we
created
an
epic,
it's
an
issue
over,
but
it's
an
epic
and
we're
already
discussing
how
who
what
approaches
we
can
take.
What
are
the
possible
problems
we
can
face?
A
So
then,
when
we
start
working
down
the
road,
we
already
have
some
better
idea
instead
of
just
facing
the
problem
out
of
the
work,
and
a
good
thing
over
here
is
that
we
can
use
our
own.
We
can
dog-foot
this
with
our
own
pipeline,
so
I
did
some
back
of
the
napkin
computation
and
assuming
that
five,
that
for
in
a
fail
pipeline,
only
five
percent
of
the
jobs
fail.
A
A
That
would
be
really
great.
I
forgot
that,
and
this
is
only
in
the
gitlab.com
repository,
I
imagine
all
the
others
that
we
could
for
our
user
bases
as
well.
So
a
lot
of
potential
right
here-
I'm
really
excited
about
this
one.
If
you
have
feedback
on
this,
if
you
have
used
cases
please
head
over
to
the
issue
android
epic,
let
us
know
that
we
can
include
this
while
we're
planning
our
writing
from
now
and
yeah
so
up
next,
I
will
continue
working
on
the
notebook
dev.