►
From YouTube: CDF - SIG MLOps Meeting 2021-10-07
Description
For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/
A
A
I'm
good
thanks
and
congratulations
to
you.
A
So
just
I'll
bring
you
up
to
speed
a
little
bit
because
it's
been
been
a
while,
since
you've
had
the
time
to
come
and
play
on
this
side,
so
we've
been
kind
of
plugging
on
mostly
with
communications
work
this
year,
so
I've
been
doing
a
bunch
of
conference.
A
Speaking
explaining
the
road
map
and
letting
people
know
what's
going
on,
you
may
have
seen
we
did
a
slot
at
devops
world
and
yesterday
I
was
at
apcsm,
which
is
the
advanced
process,
control
and
smart
manufacturing
conference
in
the
us,
where
there's
a
lot
of
people
working
with
machine
learning
in
in
the
semiconductor
industry.
A
So
that's
been
quite
positive.
We've
had
a
lot
of
interest
in
consuming
the
roadmap,
so
people
are
definitely
keen
to
understand
what
the
challenges
are.
A
So
one
of
the
things
I
need
to
do
over
next
month
really
is
get
some
updates
into
this
year's
document
and
get
some
people
to
review
it
see
if
anyone
has
anything
that
they
do
want
to
contribute,
and
then
you
start
the
work
to
finalize
a
2021
release.
A
Attendance
at
the
meetings
has
been
low
over
the
past
month
or
so,
but
you
know
it
does
tend
to
drop
off
over
the
summer
period,
as
people
are
busy
doing
other
things.
A
But
I
think
we
will
probably
need
to
pick
up
on
some
more
promotional
work
for
next
year
so
that
we
can
encourage
more
contributors.
A
A
Yeah
sure
so
basically,
what's
happening
is
that
the
eu
has
defined
a
series
of
classifications
for
different
types
of
machine
learning,
product.
A
And
certain
products
will
be
completely
prohibited,
so
there
are
a
number
of
concerns
about
the
use
of
targeted
data
to
manipulate
particular
groups.
A
A
A
A
So
that
regulation
basically
brings
in
a
whole
set
of
requirements,
and
let
me
just
flip
across
because
I
think
I've
got
a
list
open
here.
Yes,
just
one
second,
when
I
get
to
the
relevant
section
yeah
so
so
there
in
order
to
com
comply
with
this
regulation,
you're
gonna
have
to
have
a
set
of
mandatory
data
quality
controls,
so
all
the
stuff
that
we've
talked
about
last
year
within
the
roadmap,
about
having
to
integrate
all
of
our
data
management
into
the
continuous
delivery
process.
A
Well,
that
will
be
required
by
law
for
anything
that
is
classified
as
high
risk,
and
you
will
have
to
have
mandatory
end-to-end
documentation
to
support
that.
So
so
you,
you
will
need
the
traceability
that
we've
previously
been
discussing,
but
they're
also
requiring
transparency
of
system
decisions
and
human
oversight
over
the
functioning
of
any
systems.
A
A
So
from
a
from
a
roadmap
perspective,
it
really
escalates
the
need
for
a
whole
bunch
of
the
key
challenges
we
addressed
last
year
to
to
actually
be
worked
on
and
implemented
in
solutions,
because
you
know
there
will
be
a
whole
raft
of
things
by
2023
when,
when
this
is
expected
to
come
into
force,
that
will
will
be
mandated
in
law
and
right
now.
A
What's
not
so
positive
is
that
in
order
to
release
a
product,
that's
classified
as
high
risk
you'll
be
required
to
pass
a
third
party
conformity
assessment
and
every
time
you
change
anything,
you
will
have
to
go
through
a
new
manual
conformity
assessment
process.
Again.
A
A
A
And
there
will
be
other
requirements
so
what
they're
calling
post-market
monitoring,
which
is
basically
surveillance
of
all
active
ai
systems?
A
A
So
again
that
end-to-end
audit
process
that
we
previously
discussed
is
now
going
to
become
law
in
europe.
A
And
if
you
don't
get
it
right,
it's
a
30
million
euro,
fine
or
six
percent
global
revenue,
whichever
is
the
higher,
so
it's
definitely
gonna
have
very
significant
impact
for
anyone
either
based
in
europe
or
with
customers
in
europe,
because
this
legislation
applies
regardless
of
where
you're
operating
the
system.
A
A
The
the
legislation,
as
it's
worded
at
the
moment
requires
people
to
understand
how
the
system
is
making
decisions,
but
due
to
the
nature
of
the
type
of
systems
that
people
are
likely
to
build,
it's
unlikely
for
any
individual
human
to
be
able
to
understand
the
the
complexity
of
the
data
involved
in
making
those
decisions.
So
it's
an
unachievable
goal
in
many
cases.
A
So
this
is
likely
to
really
negatively
impact
the
development
of
machine
learning
in
the
european
union
and
I'm
I'm
not
sure
whether
this
will
make
it
into
the
final
legislation.
It's
it's.
Certainly
a
significant
gap
between
you
know
what's
actually
going
to
be
built
and
what
the
legislation
requires.
B
So
this
legislation
is
proposed
to
be
enacted
in
2023,
but
is
not
yet
I
don't
know
enacted
in
law.
Yet.
A
Yeah,
so
it's
going
through
the
process
of
it.
It
went
through
a
a
review
process
over
the
over
the
past
year
to
formulate
the
the
wording
of
the
legislation.
A
That
wording
has
now
been
published,
and
so
it's
now
entered
into
the
into
the
process
of
being
enacted
in
into
law.
A
Now
there
there
is
lobbying
going
on
at
the
moment
to
to
try
and
improve
the
the
quality
of
the
requirements,
and
I
imagine
we'll
see
some
fairly
significant
changes
before
it's
actually
passed,
but
the
broad
outline
of
what
they
they
think
they
need
has
has
been
published.
B
Right
and
this
idea
of
data
management
as
part
of
an
ml
ops
process,
it's
pretty
interesting.
I
actually
was
I've,
read
a
little
bit
about
some
work
being
done
on
making
data
quality,
systemic
sort
of
like
really
looking
at
that
systemically
within
and
in
mlaps
sort
of
iterative
process,
and
that
that
was
pretty
cool
are
we
are
we
looking
at
that
within
our
roadmap
at
all,.
A
Yeah,
so
that's
one
of
the
things
that
we
we
had
in
the
roadmap
last
year
and
I
think
you're,
probably
one
of
the
relevant
updates
for
this
here-
is
to
really
set
that
in
the
context
of
you
know
now,
these
things
that
we've
been
discussing
are
actually
going
to
become
legally
mandated.
A
So,
for
example,
the
law
requires
that
you
provide
full
access
to
all
of
your
data
to
nominated
surveillance
authorities,
so
it
won't
be
a
case
of
you
need
to
use
this
data
to
defend
yourself
in
court.
If
something
goes
wrong,
it
will
actually
be
a
case
that
the
government
will
regularly
come
in
and
inspect
your
data
and
you'll
have
an
issue
if
it's
not
compliant.
B
Yes,
one
of
the
things
I
was
looking
at
is
less
from
a
regulation
point
of
view,
but
more
of
how
you,
actually,
I
guess,
massage
your
data
and
label
it
really.
If
you
have
conformity
in
how
you're
labeling
the
data
that
you
can
get
better
results
faster
with
your
models
and
maybe
even
with
less
data
points,
so
you
can
have
a
smaller
amount
of
data
to
to
build
your
models
and
refine
them.
B
A
Yeah,
so
I
I
think,
that's
that's
something
that
we
could
probably
expand
on
in
in
the
technical
requirements
section.
A
I
think
the
the
the
big
challenge
that
we've
we
sort
of
identified
last
year,
but
which
has
become
really
obvious
this
year,
is
that
there
really
are
two
completely
different
mindsets
in
the
ml
space.
At
the
moment.
A
So,
whilst
we've
been
approaching
this
from
the
continuous
delivery
mindset,
you're
going
to
find
that
the
majority
of
people
working
in
the
field
and
the
majority
of
products
and
tools
that
have
been
built
today
are
taking
the
the
view
that
machine
learning
is
is
special
and
that
it
needs
dedicated
tools
which
only
do
machine
learning
and
which
are
focused
on
you
know
primarily
manual
data
science
processes
about
operationally.
A
In
terms
of
you
know,
just
making
ad
hoc
changes
to
things
in
production
and
not
having
any
real
track
record
of
what
those
changes
were
or
any
way
to
reverse
them
out
again
versus
the
you
know,
continuous
delivery
model
of
needing
to
have
all
of
those
things
in
control
systems,
with
grease
management
processes
and
governance
and
all
the
rest
of
the
good
stuff
that
makes
this
sustainable.
A
And
I
think
you
know,
we've
probably
got
more
work
to
do
within
the
cdf
as
well,
because
looking
at
the
the
other
projects
that
are
out
there,
there
really
has
been
very
little
consideration
in
the
devops
tooling
space
for
incorporating
ml
ops
as
a
as
a
part
of
that.
A
So
so
I
think
one
of
the
things
we're
going
to
need
to
do
is
to
do
a
bit
more
evangelizing
in
terms
of
convincing
people
that
they
they
need
to
be
thinking
about
the
machine
learning
aspect
of
stuff,
because
you
know
we're
getting
very
near
a
point
where
every
product
that's
being
built
will
have
some
aspect
of
machine
learning
in
it.
And
so,
if
your
tooling
doesn't
support
that,
then
people
won't
use
your
tooling
anymore.
B
B
A
Well,
I
think
it's
really
important
that
that
we
get
more
people
asking
these
sorts
of
questions,
because
right
now
there's
a
lot
of
assumptions
that
are
being
made
and
they're
unspoken
assumptions
and
when
you
start
to
dig
into
them,
you
uncover
all
sorts
of
problems
that
people
haven't
fully
recognized.
Yet.
A
A
You
know
the
the
the
teams
who
were
primarily
involved
in
this
data
science
work
are
mathematicians,
who
are
in
many
cases
straight
out
of
university
and
they've
got
a
really
deep
understanding
of
the
statistics
involved
and
they're
coming
up.
Some
really
interesting
analysis
processes
to
generate
good
decision
making
on
behalf
of
the
system,
but
practically
they've
they've
had
you
know
a
couple
of
weeks,
training
in
python
and
r
and
that's
the
extent
of
their
experience
of
managing
software
assets.
A
So
so
they
really
have
absolutely
no
visibility
of
all
the
challenges
that
happen
when
you
try
and
put
something
into
a
production
environment.
A
So,
there's
a
lot
of
problems
right
now
where
people
are
spending,
you
know,
months
and
months,
building
really
complicated
models
and
then
basically
stumbling
at
the
last
minute
because
they
start
trying
to
put
this
stuff
into
production
and
it's
there
are
so
many
issues
in
in
trying
to
manage
it
effectively
that
you
know
they
end
up
failing.
A
But
I
I
think,
we're
probably
going
to
have
a
a
separate
session
in
the
near
future
to
to
talk
about
some
of
the
activities
that
we
need
to
plan
out
for
the
road
map
in
in
general.
A
B
B
A
No
problem,
no,
I
don't
expect
you
to
to
show
up
to
all
of
them
yeah,
I'm
I'm
happy
to
to
sit
in
the
middle
and
coordinate
the
two
time
zones.
That's
not
a
problem,
but
I
think
you,
the
most
valuable
thing
we
can
probably
do,
is
take
a
step
back
and
spend
a
little
bit
of
time,
thinking
about
the
overall
strategy
and
what
we
need
to
do
over
the
next
year
or
so
to
make
sure
we're
managing
this
asset
appropriately.