►
From YouTube: Weekly Sync 2021-05-25
Description
Meeting Minutes: https://docs.google.com/document/d/16u9Tev3O0CcUDe2nfikHmrO3Xnd4ASJ45myFgQLpvzM/edit#heading=h.pmkhhqcd8hw1
A
B
A
A
B
Okay,
so
I
was
trying
to
tune
the
parameters
and
I
noticed
that
if
you
try
to
change
the
parameters
of
a
model
that
is
saved
into
a
directory,
it
doesn't
really
change
the
hyper
parameters
at
all.
B
See
so
I'm
running
this
model,
this
svc
and
it's
saved
in
psychic
svc
right,
so
I
go
ahead
and
re-initialize
the
model,
and
I,
if
I
choose
the
same
directory
and
declare
an
absurd
parameter,
it
will
just
give
me
the
same
accuracy
as
the
one
before,
because
this
is
actually
the
same
model
that
yeah
here.
A
Yeah,
okay,
so-
and
I
think
this
was
let's
see
so
because
I
think
what
happened
here
yeah
so
basically
we
were
tying
the
model
to
the
directory
and
if
you
have
an
existing
model,
it's
just
going
to
load
that
model.
So
I
think
this
is
part
of
that
project.
So
your
project
to
look
at
the
saving
and
loading
of
the
the
model
config
into
these.
You
know
we
you're
gonna.
A
So
I
think
that
I
think
that,
let's
see,
let's
jump
back
here,
okay,
because
what's
happening
right
now,
is
it
basically
just
says:
there's
a
model
in
that
directory
load,
the
model
yeah.
B
A
A
So
what
we
had
done
was
sorry,
so
what
we
had
done
was
we
had
used
all
of
the
config
parameters
to
decide
what
the
what
the
model
was
that
got
saved
within
that
directory
and
that
ended
up
confusing
people
because
it
was
well
basically.
So
if
you
change
the
config
parameters-
and
you
pointed
at
the
same
directory,
then
then
you'd
end
up
with
you
you'd
end
up
with
the
config
well
you'd
end
up.
A
You
could
end
up
with
multiple
models
in
the
same
directory
and
then
you
it
wasn't
exactly
clear
which
one
you
were
using
and
it
was
just
based
off
of
the
the
config
so
like
the
config
structure,
and
so,
if
you
change
the
config
structure
at
all,
you
might
end
up
with
several
different
models
within
the
same
directory
and
you're,
not
really
exactly
sure
which
one
you're
using.
A
So
we
may
want
to
make
this
a
little
more,
and
this
is
sort
of
you
know,
part
of
the
whole
thing
we
may
want
to
make
this
a
little
more
yeah.
The
question
is,
really,
you
know
figuring
out
this
whole
saving
and
loading
thing,
and
you
know,
because
we
we
should
be
doing
that
on
eight
on
the
a
inter
method.
A
Right
is
where
we
should
be
where
we
should
be
loading
the
models
and
the
question
is
like
okay:
well:
well,
do
you
load
uniquely
based
off
of
the
directory
right
and
and
if
you
override
the,
if
you
override
the,
if
you
set
a
different
config
parameter
for
the
same
directory
right,
do
you
want
it
to
blow
up
and
say
hey,
you
already
had
a
model
saved
in
that
directory.
You
see
what
I'm
saying.
C
A
Because,
because
that's
sort
of
the
situation,
we're
in
right
now
is
if
you
change
that
config
parameter,
but
the
model
still
pointed
at
the
same
directory,
then
then
you
know,
should
we
make
the
person
pointed
at
a
new
directory
to
have
a
new
hyper
perimeter?
A
Yeah
we
could
overwrite
the
directory.
The
problem
is,
you
know,
we're
loading
the
contents
right
from
the
directory,
so
this
is
sort
of
where
it
comes
in,
where
what
we'd
do
is
we'd
save
the
config
we'd
save
the
model
config
you
know
without
the
directory.
I
can't
remember.
I
think
we
had
a
meeting
when
we
talked
about
that
at
one
point,
but
we'd
save
the
model
config
without
the
directory
and
then
we'd
load
it
back
right
and
then
we'd
look
at
the
config
structure
and
we
see
if
it
matches.
A
You
know
the
we'd
look
at
the
config
structure
that
we've
been
instantiated
with
and
we
see
if
it
matches
the
one
that's
on
disk
and
if
it
does,
then
you
know
we
know
we
want
to
do
the
same
thing.
You
know
we
know
we
want
to
use
that
that
checkpoint,
that's
on
disk
or
whatever
the
saved
state
is
otherwise.
You
know
we
might
need
to
you.
We
would
use
a
method
that
basically
says
hey.
You
know
just
load
from
that
saved
state.
A
That's
on
disk
and
ignore
you
know
whatever
might
be
the
the
current
config
state
right.
So
this
would
be
the
case
where
you
basically
instantiate
a
model,
so
there's
two
cases
there's
the
case
where
you
or
I
guess,
there's
three
cases.
Okay,
so
maybe
we
should
write
this
out.
How
should
we
do
this?
A
Let's
open
here,
I'll,
just
open
a
text
document
and
we
can
write
it
out.
This
is
a
good
one
for
the
arch
docs.
C
Arch
seriously,.
A
A
A
A
I
think
we've
changed
things
many
times
and-
and
you
know
it's
time
it's
time
to
make
sure
that
everything
is
correct,
because
we
have
half
half
changed
things
at
times
and
and
haven't
finished,
changing
it
all.
So
that's,
and
that
is
this
project
right,
so,
okay,
so
all
right.
C
A
All
right,
so
what
we
want
to
do
is
we
want
to
get
to
a
place
or
so
okay.
So
this
is
the
question.
Is
we
want
to
formalize
how
models
are
saved
and
loaded
how
they
interact.
A
Is
there
anything
else
we
want
to
do
here?
So,
let's
see,
we
want
to
make
sure
that
we
understand
how
they
interact
with
their
config
structures.
We
want
to
make
sure
that
we
know
which
takes
precedence.
A
All
right,
so,
let's,
let's
enumerate
the
cases,
so
we
have
the
following
cases.
A
A
And
then
we
had
so
okay.
We
then
there's
this
one,
that
you
just
discovered
right,
so
we
have
or
that
you
just
highlighted
so
oops.
A
So
we
have
stayed
in
the
directory,
and
now
we're
saying
you
know,
there's
this
config
parameter
there
that
we're
we're
overwriting.
So
so,
and
then
we
can
say
you
know
like
saved
value
of
c
is
two.
A
B
B
B
Yeah
yeah
yeah
using
the
same
model.
Actually,
in
these
cases
we
tried
to
instantiate
re-instantiate
it
right.
A
So
you,
let's
see,
let
me
just
let
me
just
be
clear
on
what
you
did
here.
So
you
did
model
and
then
and
then
you
said
you
know.
B
Yeah
yeah
right-
and
this
is,
if
you
print
the
model
like
print
statement,
the
model.
B
It
shows
it
changed
there,
but
I
don't
know
it
doesn't
work.
A
Yeah
because
it's
not
reinstantiating
that
model-
and
I
think
the
thing
is
so
that's
a
good-
that's
a
good
one
to
track
so
those
those
config
structures
should
be
frozen.
So
that
I
think,
is
a
separate
issue,
because
I
think
we
shouldn't,
let's
see
yeah
those
properties
within
the
config
structure.
You
know
they
are
yeah.
They're
assumed
they're
assumed
to
be
static.
A
So
let's
see
they
were
originally
static
because
we
were
using
name
tuples
and
then
we
converted
the
data
classes
and
I
don't
think
we
made
them
frozen
and
I
think
that's
probably
what's
going
on
here
so
because
or
else
you
wouldn't
have
been
able
to
change
it
so
yeah,
so
you
basically
wanted
to
just
tweak
the
you
just
wanted
to
tweak
the
model
parameter
and
then
re
redo
an
operation
right
like
a
different
train.
A
B
Yeah
yeah
exactly
I
was
trying
to
see
if
we
are
able
to
down
the
model
by
without
having
to
reinstantiate
it.
A
D
So
I
I
have
a
point
here
like
there
are
two
types
of
configurations
you
might
have
in
a
modern
conflict.
One
is
that
that
would
affect
the
architecture
of
the
model
and
cannot
be
changed
once
the
model
is
made
or
saved
and
others
are
which
are
hyper
parameters
and
should
be
changed.
Should
we
should
be
able
to
change
them
right.
C
C
A
A
Parameters,
yeah
and
okay,
and
so
obviously
so
the
setup
right
now
was
was
basically
you
know,
because
because
a
lot
of
this
has
been
done
from
the
command
line
right
and
so
the
the
python
api
has
has
not
really
been,
has
been
more
of
an
afterthought
a
lot
of
the
times,
and
so
you
know
when
we,
when
we
did
different
models,
we
would
have
different
directories
and
then
you
know
you
tuned
to
you
change
the
hyper
parameters
between
invocations
of
command
line
right
and
so
now,
as
we're
doing
more
python
api
stuff.
A
You
know
we're
finding
we're
finding
things
like
this
right.
So,
let's
see
so
architecture
parameters,
so
architecture
parameters
which
fundamentally
ill
to
the
model
which
might
want
to
change.
Okay,
let's
see,
let's
think
about
this
in
more
depth.
So
so
the
question
is
okay.
So
at
what
point
do
we
call
it
to
what
point
do
we
call
it
a
new
model
right?
A
Do
we
want
to
call
it
a
new
model
when
the
hyper
parameters
change,
because
this
this
so
these
two,
these
two
man
I
wish
you
could
see
the
visual
highlighting,
but
these
two
things
sort
of
to
some
extent
affect
this
case
here
right
where
we're
looking
at
the
the
saved
state
right.
So
if
we
want
to
load
it
and
we
want
to
tweak
the
value
right,
do
we
do
we
do
we?
Are
we,
okay
with
tweaking
the
value
on
load?
A
Are
we
only
okay
with
not
to
like
if
you,
if
you
declare
a
hyper
parameter,
that
is
a
tweakable
hyperparameter?
Are
we
okay
with
tweaking
it?
Well,
it's
loaded
or
like
when
it's
loaded?
Are
we
only
okay
with
tweaking
it
at
runtime
right?
Because
here
we're
okay
with
tweaking
it
at
runtime?
A
You
know,
maybe
it
was
loaded
to
a
value
of
of
10
million,
let's
just
make
it
a
million,
so
maybe
it
was
loaded
to
a
value
of
a
million,
but
now
we
want
to
tweak
it
at
runtime
to
a
value
of
42
right,
but
do
we
do
we
want
to
throw
an
exception?
Saying
hey
this?
Isn't
the
same
model
like
if
you're
trying
to
load
a
model
with
a
hyperparameter
of
you
know
10
million,
but
I'm
trying
to
load
it
from
from
a
saved
state?
A
You
know,
and
I
and
I
want
to
give
you
the
accuracy
then
you
know,
will
I
give
you
the
accuracy
using
the
hyper
parameter
or
the
new
hyper
parameter?
Or
while
I
give
you
the
accuracy
using
the
old
hyper
parameter.
D
Like
if
user
is
giving
some
hyper
overwriting
some
hyper
parameter,
then
it
is
his
own
decision
right.
So
it
should
be
like
you
know
what
the
the
parameter
that
the
user
is
providing
should
be
used
right,
yeah
and
you
know
if
it
is
not
an
architectural
one.
If
it
is
an
architectural
one,
then
we
should
reject
potentially
say
like
that.
It
is
not
possible
yeah.
B
It
would
make
things
a
lot
simpler
if
we
just
completely
override
the
parameters
and
not
keep
the
previous
parameters.
The
hyperparameters,
okay,.
A
A
A
Honestly,
okay,
so
if
the
model
is
so
model's
instantiated
with
the
config,
we
have
stave
state.
We
have
the
a
conflict
with
the
save
state.
Architectural
conflict
raise
and
exception,
hyper
parameter,
conflict
override
hyper
parameter,
let's
see,
and
then
with
the
new
accuracy
stuff.
I
think
that
should
be
fine,
because
we're
not
going
to
like
save
the
old
accuracy
with
the
old
hyper
parameter.
A
A
So
this
also
lets
us
know
that
we
probably
need
some
kind
of
config.
You
know
some
kind
of
of
way
of
of
telling
you
know
telling
the
model
which
which
is
or
well
let's
see.
A
Yeah,
because,
when
we're
thinking
about
auto
tuning
stuff,
this
was
another
thing
that
came
into
play
oops
when
we
were
thinking
about
so
this
has
cross
applicability,
okay,
so
yeah.
So
when
we're
thinking
about
automl
in
our
thoughts
about
automl
dots
about.
A
A
We
knew
that
there
will
be
hype
model
model
properties.
A
A
So
in
these
we
knew
that
we'd
have
to
add,
so
so
for
this.
For
this
to
be
something
that
so
for
something
like
automl.
D
D
A
Works
yeah
immutable
is
a
better
well,
let's
say
because
this
is
this
is
talking
about
the
the
things
we
want
to
tune.
So
this
basically
maps
now
that'll
be
a
good
next
sentence.
So,
within
the
context
of
this.
A
Discussion,
that's
because
that's
more
background,
this
becomes
tunable
and
non-tunable
become
mutable
and.
A
All
right,
so
these.
A
Great,
so
so
we
should
yeah
now
now
we'll
want
to
let's
see
so
what
we'll
want
to
do
is
add
infrastructure
or
add
to
the
config
infrastructure
to
to
to
allow
for,
let's
see,
let's
see
what
what
do
we
have
df
file
base.
A
So
if
we
look
at
the
configure
structure
here,
we
need
to
add,
let's
see,
field
metadata
required
field,
where's
field,
here's
field
so
basically
we'll
want
to
add.
You
know
something,
that's
immutable
or,
let's
see,
should
we
tag
it
as
mutable.
We
should
probably
tag
it
as
mutable
and
leave
it
immutable
by
default.
A
So
basically
you
should
assume
that
if
you
change
values,
you
should
assume
as
as
someone
using
these
values
and-
and
this
is
sort
of
the
way
the
name
tuple
works
so
similar
to
the
way
name
tuple.
So.
A
So
what
is
this
typing
fields
or
properties.
A
D
A
All
right
so
now
now
yeah.
So
any
questions
on
that.
A
Okay,
yeah,
okay,
great
so
yeah.
I
think
I
think
this
is
a
good
way
to
this.
This
seems
to
make
sense
all
right,
so
yeah
we'll
add
this
immutable
field,
and
that
way,
you
know
this
mutable
field
is
also
something
that
we
know
that
it's
a
it's
a
target
for
tuning
when
we
get
to
automl
and
from
the
perspective
of
you,
know
the
stuff
that
has
going
on
with
model
saving
and
loading
right
now
we
know
mutable
is
basically
overwritable.
A
So
within
the
context
of
this
discussion,
let's
see
in
the
following
case,
you
know
c.
So,
let's
see
c
is
immutable.
Property.
A
When
we
load
the
model
at
the
in-memory
value
of
1
million,.
A
C
C
A
A
All
right,
let's
see
anything
else
here,
that
we
need
to
talk
about
so
basically
c.
So
anything,
that's
a
a
a
mutable
property.
Let's
see
so
we'll
have
these
config
structures,
yeah.
B
It
would
also
work
without
reinstantiating
the
model
right.
A
Yes
right,
so
so,
if
it's
two,
so
if
it's,
if
it's
immutable,
you
have
to
reinstantiate
the
model,
if
it's
immutable
or
sorry,
if
it's
immutable,
you
have
to
reinstantiate
the
model.
If
it's
mutable,
then
you
don't
have
to
reinstantiate
the
model
right.
A
Okay,
so
and
let's
just
write
that
because
that's
for
clarity,
I
like
this
is
good.
I
like
these,
this
format
for
the
architecture
stuff.
This
is.
This
has
been
very
helpful
to
guide
the
discussion,
all
right,
so,
let's
see
so
so
changes
to
immutable.
A
A
I
think
that
also
follows
that
so
four
four
fields
so
for
config
fields
marked
as
mutable.
B
A
Okay,
so
let's
see.
B
Okay,
I'm
a
little
confused
here,
so
we're
making
it
so
that
the
hyper
parameters
are
not.
A
The
hyper
parameters
are
changeable
runtimes,
so,
let's,
let's
do
so
if
we
user,
so
this
is
so.
This
is
the
saved
value.
Actually
you
know
this
may
need
to
be.
Let's
see
this
has
to
do
with.
B
B
A
No
worries
so,
let's
see
config
fields.
Where
did
we
say
so?
Where
did
we
say
that
so
above
the
500
grit?
Okay,
so
this
is
the
context.
This
is
the
decision.
Okay,.
A
So
sorry,
as
I
move
things
around
here,
so
try
to
put
things
into
the
right
places
so,
okay,
so
this
is
our
context.
We
want
to
formalize
this
with
the
following
cases.
A
You
know
this
is
what
we're
dealing
with.
We
have
two
cases,
so
we
found
that
there's
two
types
of
model
parameters.
A
And
let's
sort
of
put
this
data
down
in
the
other
section
here.
A
So
our
context,
you
know
we
have
immutable
and
we
have
mutable.
We
talked
about
you
know
when
we
were
talking
about
implementing
ml
these
these
map
to
tunable.
So
two
ball
and
two
one
non-tunable
becomes
immutable
versus
immutable.
A
So
now
we're
saying
basically
that
similar
to
the
way
the
name
tuple
works,
and
so
all
our
config
stuff
came
from
name
tuple.
You
know,
config
properties
will
default
to
immutable.
So
this
way
implementation
classes
can
assume
that
they
don't
need
to
deal
with
state
changes
to
config
properties
unless
they've
explicitly
opted
in.
A
A
And
we
say
you
know
c
field.
A
And
equals
field,
so
this
is
the
field
c
hyper
parameter.
A
Is
that
isn't
that
I
think
that
can
be
one
word
and
so
mutable
equals
true.
A
All
right,
so
yes,
so
now
we
have
the
c
hyper
parameter.
It
is
mutable.
Let's
see,
okay
in
the
following
example:
we
have
a
hyper
parameter
and,
let's
see,
I
think
we
have
this
down
here.
A
And
so
we
want
to,
we
want
to
support
changes
to
it
at.
A
A
Okay,
so
this
sort
of
raises
there's,
there's
a
there's
another
there's,
let's
see.
A
So,
okay,
so
I'm
thinking
about
nested
config,
but
I
don't
know
if
we
need
to
think
about
this
because
I
think
nested
config
ends
up
being
a
separate
object.
So
we
don't
need
to
actually
worry
about
that,
so
our
model
wants
to
support
changes
to
it
at
runtime.
So
if
we
change
c.
A
So
we
instantiate
the
model,
we
change
c,
let's
say
default
or,
let's
see
sorry
c
equals
zero,
now
c
equals
42..
A
So
at
this
point,
okay,
so
so
the
model
needs
to
do
something
at
this
point.
You
know
when
when
it
when
it,
when
it
detects
a
change
to
c
right,
so
the
model
needs
to
to
something
needs
to
happen
right,
so
maybe
it
needs.
It
probably
needs
some
kind
of
property
to
to
handle
that
right,
and
it
probably
needs
to
register
this
stuff
with
the
config
structure.
A
So,
for
example,
in
the
init
method
of
the
model
it
it
will
need
to
tell
the
config
structure
that
when
c
so
we're
going
to
need
to
do
some,
some
some
some
like
setters
and
getters
and
stuff.
So
basically
so
config
right
now
is
this
data
is,
is,
is
a
data
class
and
we
may
just
you
know,
get
rid
of
the
whole
data
class
thing,
because
at
this
point
we're
gonna
be
at.
B
A
Point
of
the
data
classes
to
you
know,
help
us
with
the
type
hints
and
and
all
of
that
so,
but
we
may
just
you
know,
create
the
classes
by
hand
using
that
the
type
built
in
so
let's
see
so
because
we're
going
to
need
to
make
these
properties
right
and
so
we're
going
to
need
to
make
each
each
so
these.
So
when
I
say
property
I
mean
like
the
python
built-in
property,
oops
where's,
no
manual
entry
for
property.
So
let
me
just
bring
this
up
here.
A
So
this
is
the
python
built-in
property
right
and
so
we're
looking
at
setters
and
getters
right,
and
so
what
we'd
want
is
we'd
want
to
implement
a
setter
that
basically
throws
an
exception.
If
it's
immutable
and
we'd
want
to
implement.
You
know
the
getter
will
be
a
pass-through
and
we
could
even
do
this.
We
could
even
do
this
as
we
could
even
do
this
as
would
get
at
her
and
set
at
her.
A
We'll,
probably
just
you
know,
we
want
to
check
that
out,
see
which
is
which
is
better,
because
this
is,
let's
see
yeah
yeah.
This
might
be
more
convenient
type
of
thing
here
we
could
probably
use.
We
could
probably
use
something
like
this
and
then
use
like
we
could
use
a
use
like
a
we
could,
let's
see
so
we
could
use
something
like
this
to
say
as
we
go
through
here.
A
You
know
we
could
basically
look
at
okay,
let's
create
that.
Okay,
so
we
end
up.
We
we're
given
a
class
right
when
we
do
the
the
config
decorator
here.
That's
this,
and
so
it's
given
a
class
with
all
these
properties
and
what
we
can
do
is
we
can
do
something
similar
to
data
class,
we'll
want
to
look
at
data
class
and
see
what
it
does,
because
it
has
that
frozen.
A
A
Okay
notice
that
they're
actually
creating
they're
using
it
they're,
actually
they're,
actually
like
compiling
a
function.
It
looks
like
yeah,
they're,
compiling
and
executing
a
function
and
very
interesting,
so,
okay,
so
but
but
the
end
result
is
basically
that
they
are
let's
see
set
at
her
deleter.
A
String
all
right
so
they're
overriding
the
set
out
here
in
dell
at
her
methods
at
the
class
level,
rather
than
at
like
a
property
level.
So
they're
not
using
you,
know,
property
they're,
they're
doing
it
at
this
that
they
get
at
her
and
set
at
her
and
dell
at
her
level,
so
basically
at
the
class
scope.
A
So
let's
see
okay
anyways
we're
a
little
on
the
weeds
here,
but
all
right
so
in
this
case
right
we'll
change,
we'll
need
to
the
point
of
this
was
the
config
structure
itself,
this,
like
config
class
itself,
when
it's
instantiated
it
will
need
some
sort
of
like
callback
functions
for
the
modification
of
mutable
values.
So
you
know,
for
example,
like
set
c
or
something
like
that
or
maybe
so
we
could
do
it
basically
at
the
method,
scope
or
we
could
do
it
at
the.
A
So
we
could
do
it
at
the
scope
of
the
the
on
a
single
property
value.
So
basically
we
could
have
you
know,
set
c
equals.
You
know
some
function.
A
You
know
c
and
then
say,
for
example,
if
we'd
been
doing
so
self
dot
model,
dot
c
equals
c
right.
So
we
could
do
it
at
this
level
or
we
could
do
it
at
you
know.
Mutable.
A
So
we
have
two
approaches
here
right.
We
could
do
it
on
the
individual
property
level
or
you
could
sort
of
say
that
you
know
the
config
structure.
If
it
has
any
any
fields
that
are
marked
mutable,
then
it
must
define
some
kind
of
callback
function
like
this.
Mutable
callback
function
that
that
that
that
that
allows
for
modification
right
because
or
otherwise
we
won't.
We
won't
know
that
we
need
to
go
pick
up.
The
changes
right
does
that
does
that
make
sense
to
everyone.
A
Yes,
yeah:
does
that
sound
good,
so
the
other
option
is
we
can
basically
just
trigger,
so
we
can
trigger
the
let's
see
so
we'd,
probably
let's
see
yeah
so
we
could
trigger.
We
could
just
trigger
a
function.
Call
let's
see
what
it.
What
are
other
languages?
Do
things
usually
do
like
an
event
like
if
you
looked
at
like
javascript
or
something
like
that
yeah.
I
think
I
think
this
is
probably.
A
This
probably
makes
the
most
sense.
This
second
approach
here
so
okay-
and
I
know
we're
running
long
talking
about
this,
but
this
is
complicated,
so
so
all
right
so
now,
basically,
when
we
set
now
okay,
so
so
this
is
what
what
so
we
require.
A
Function
be
registered
before
modification
of
mutable
values.
A
And
in
this
way
that
in
this
way
we
ensure
that
that
that
the
whoever
is
is
whoever
has
declared
a
value
immutable
has
also
you
know,
gone
and
let's
see
that's
thing,
okay,
so
so
you
could
declare
value
immutable.
But
then,
if
you
don't,
if
somebody
modifies
that
value,
I'm
trying
to
think
about
how
do
we
ensure
that
that
authors
of
models-
always
you
know,
always
always
actually
go
and
and-
and
you
know,
register
the
the
appropriate
callback
right.
A
So,
let's
see
because
you
risk
so
we
risk
so
we
risk
it.
We
risk
a
case
here
where
a
model
author,
you
know,
register
or
doesn't
register
a
mutable
callback
right.
So
if
they
don't
do
this
and
then
somebody
uses
their
model
and
they
and
they
mutate
the
and
they
they
never.
They
write
their
test
cases
and
they
never
mutate.
The
value
they're,
never
gonna
find
out
that
they
forgot
to
register
this
mutable
callback
until
somebody
goes
and
does
it
right.
So
the
question
is.
D
Can't
we
write
a
single
setting
sub
like
some
sort
of
mechanism
in
the
base
class
itself,
which
base.
A
D
If
a
mutable
parameter
is
like
change,
then
this
kind
of
setting
is
done.
A
Yeah
yeah,
so
we
can
do
that.
The
thing
just
becomes
you
know
if,
if
you
have
so,
for
example,
for
a
lot
of
models
where
we're
doing
this
a
pass-through
type
of
thing,
so,
for
example,
in
model
scikits
like
like.
A
So,
for
example,
in
scikit
you
know
we
basically,
let's
see
where
is
it.
C
A
So
here's,
where
we're
instantiating
the
model,
so
we
go
and
we
grab
the
classes
so
so
these
are
the
psychic
classes
right.
So
these
are
these
are
being
pulled
from
from
psychic
and
then
we
set
them.
You
know,
let's
see,
so
what
do
we
do?
We
say
you
know
the:
where
is
it.
A
A
Okay,
so
okay
yeah,
so
here
when
we
do
the
a
enter-
and
this
is
exactly
you
know,
the
this
is
exactly-
we've
got
a
roundabout
way
to
come
to
the
code
that
is
causing
that
problem
that
you're
talking
about
so
so
this
is
the
actual
code
that
causes
that
issue
right.
A
Where
we're
doing,
if
the,
if
the
path
is
a
file,
then
we
load
the
file
right,
and
this
is
where
that
10
000
or
that
10
million
parameter
didn't
get
picked
up
because
it
loaded
it
straight
from
the
file
and
this
and
this
yeah
so-
and
this
is
the
case
where
there
is
nothing
and
so
it
instantiates
it
and
it
passes.
You
know
the
config
dictionary
here
right.
So
the
the
thing
is
that
this
is
specific
to
the
scikit
models
and
it
changes
based
on
every
model
or
every
object.
A
A
Yeah,
and-
and
so
this
this
becomes
a
question
is:
is
you
know
how
do
we?
How
do
we
ensure
that?
How
do
we
ensure
that
the?
A
How
do
we
ensure
that
there
is
that
there
is
a
a
method
to
call
right
so
that
somebody
doesn't
potentially
not
know
that
they
need
to
do
this
if
they're
declaring
something
mutable.
A
Yeah
yeah
yeah
because
it
will
so
in
the
main
the
main
risk
here
is,
is
if,
since
we
have
this
system,
where
we
have
a
variety
of
plug-in
authors
right,
we
have.
We
also
have
a
a
situation
where
you
know
if
we
have
to
go
we're
going
to
have
to
go
and
change
all
the
existing
plug-ins
right,
and
so
this
is
to
to
some
extent
to
make
sure
that
we
don't
forget
right
when
we
go
and
do
modifications
right
to
all
the
code
to
tag
things
as
mutable
and
even
and
immutable
right.
A
A
One
way
that
we
could
do
okay,
so,
as
you
guys
see
what
I'm
saying
right,
how
do
we
ensure
that
this
self.config
underscore
mutable
line
gets
run
yeah
so
one
way
and
and
ideally
we
could
do
it
on
config
class
instantiation,
but
that
doesn't
end
up
that
doesn't
end
up
working,
because
obviously
we
don't
have
the
model.
Yet
we
take
the
config
class
and
we
pass
it
to
the
model.
So
this
may
not
be,
it
may
not
be.
You
know,
may
not
be
like
we
could
raise.
A
We
could
raise
an
exception
on
the
getter
if
it
doesn't
so
so
you
know
we
really.
We
need
to
raise
an
exception.
So
so,
if
we
don't,
if
we
don't
have
this
mutable,
you
know
underscore
mutable,
then
we
won't
be
able
to.
You
know
we
won't
be
able
to
mutate
the
values
right,
and
so
what
we
could
do
is
we
could
just
raise
the
exception
on
git
to
right.
So
if
you,
if
you
tried
to
use
the
config,
you
tried
to
use
any
of
the
values
like
self.config.directory,
for
example,.
A
But
if
you
did
it
here,
then
no
exception
right.
So
basically
we
could
implement
the
getter
and
setter
methods
so
that
they
check
as
early
as
possible.
You
know
to
make
sure
that
that
that
this
mutable
has
been
set
and
that
way
we'll
probably
catch
the
most
cases.
You
know
that
way.
We
probably
don't
end
up
with
anybody
having
not
set
that
mutable
callback.
A
Okay.
So
let's
see
this
is
ending
up
taking
the
entire
meeting,
so
we
may
wanna.
We
wanna
talk
about
this
more
next
week
because
I
think
we
still
have
more
to
cover
here
and
we
have
more
to
document
because
there
was
a
lot
there
that
didn't
get
documented
in
and
why
we're
making
this
decision.
A
So
does
anybody
have
anything
that
they
want
to
add
to
this
right
now
other
than
you
know,
like
anything
that
you
want
to
add
to
this,
because
we
can
write
it
down
and
we
can
talk
about
it
next
week,
but
I
think
that
it
warrants
further
discussion
as
well
in
how
we
want
to
do
the
implementation
on.
A
And
we
registered
with
the
config
class
this
way
we
ensure
that
modification
of
muted
values
is
handled
somewhere.
A
All
right
so
to
that
plugin
authors.
A
So
to
combat
class
authors
from
declaring
properties.
A
All
right,
so
if
a
property
is
set
or
retrieved
or
the
instance
is
exported,
then
we'll
we'll
raise
an
exception,
and
hopefully
that
will
prevent
people
from
forgetting
to
set
the
callback,
because
if
they've
set
the
callback,
you
know
at
least
they
should
see
it.
You
know
they've
set
it
immutable.
A
They
can
write
this
in
their
code
and
say
you
know
config
mutable
is,
is
you
know
basically
this
function,
that
that
does
something,
but
you
know
they'll.
They
should
have
two
places
in
their
code.
A
They
should
have
a
place
in
their
code
where
they
declared
as
mutable,
and
they
should
have
a
place
place
in
their
code
where
they
handled
it
right
and
then,
if
they
can't
figure
out,
why
it's
not
doing
something,
then,
ideally
they
they
should
see
that
right
and
the
problem
will
be
within
their
code
base
and
not
just
the
fact
that
they
haven't
done
something
at.
B
A
So
there
will
be
something
to
change.
Okay,
all
right!
So
let's
come
back
to
this.
Let
me
just
make
sure
that
we
have
the
following
parameter.
A
A
A
A
Great,
I
think
that
does
the
trick
for
us
on
this
so
far
and
then
we'll
revisit
it
next
week
is
so
what
else
should
we
note
that
we
want
to
come
back
to
to
flush
this
out?
What
else
do
we
need
to
say.
A
A
A
A
Okay,
I
cannot
spell
this
yet
document
three
and
then
I'll
push
this.
A
A
B
Yeah,
on
the
other
pr
you
mentioned
that
we
need
more
generic
tests
for
notebooks.
I
didn't
quite
get
generic
in
how
watson.
B
Okay,
and
also,
I
also
push
to
the
vr.
A
A
A
So,
which
was
that
sorry?
Can
you
say
that
again
so
the
pr
for
the
download
button
you
did
that
one
and
let's
see.
B
A
A
B
A
B
What
I
did
is
it
doesn't,
it
just
removes
the
master,
and
since
we
don't
have
it
on
the
release,
dock.
A
A
B
A
Okay,
so
let's
see.
A
B
Yeah
I
tried
it
yeah.
A
A
Oh
yeah,
okay,
so
on
the
local
build
okay,
okay!
So
now
I
am
sorry
sorry!
No!
It's
that
made
sense.
I'm
sorry!
I
wasn't
following
okay
yeah
on
the
local
build.
This
will
not
direct
correctly
and
I
don't
think
we
have
any
way
of
detecting
the
local
build
right
now.
So
I.
B
Tried
it,
it
gave
a
404
hour.
A
Yeah,
okay,
maybe
we
could
add.
I
think
there
is
like
a
github
actions.
Environment
variable
we
could
add
a
check
for
that.
We
could
say
and
not
is
os.
Can
we
import
os
in
this
or
now,
let's
see
here,
let's
see
yeah
actually
we're
in
doxconf.
B
You
cannot
import
inside
the
prologue,
but
you
can
use
the
variables
outside.
A
Well,
actually,
it
doesn't
matter
because
we're
referencing
the
the
full
url
here
anyways.
So
this
we
shouldn't
expect
like
it's
not
like.
If
we
clicked
on
this
button,
it's
gonna
download
from
the
main
site
anyways.
So
we
don't
need
to
really
worry
about
this.
A
A
Yeah
yeah,
okay,
so,
let's
see
I
don't
think
we
need
to
change
log
for
the
download
button
for
the
notebooks.
A
Great
thanks,
so
that's
looking
good,
so
we
merged
this
pr
download.
C
A
Okay,
okay,
so,
and
we
didn't
talk
about
linting
the
commits,
but
we
did
talk
about
stuff-
that's
going
to
be
very
relevant
to
your
project
here,
let's
see
so
on
linting
the
commits.
What
was
it?
What
was
it
that
you
wanted
to
talk
about?
I
think
you
had
a
few
cases
that
you
were
still,
let's
see
what.
B
D
The
other
one
was
like
using
those
searching
for
words
inside
a
file,
and
I
just
wanted
to
know
that
what
would
be
efficient
by
opening
the
file
and
searching
it
well.
A
So
I
think
that
basically
we're
going
to
take
out
should
I
you
know
all
the
plugins
are
going
to
get
moved
into
the
second
party
org,
so
that
one
we
don't
need
to
worry
about
so
much
and
then
there.
A
Are
similar
cases
yeah?
I
noticed
that
I
accidentally
merged
one
the
other
day
that
that
had
a
sort
of
a
shortened
path.
Well,
but
this
is
okay.
I
mean
it's
okay,
if
there
are
there,
if
there
do
exist,
cases
right,
because
we
just
want
to
do
this
going
forward.
A
So
I
think
I
think
we
can
probably
safely
go
with
what
we
have
right
now.
If
this
is,
if
you're
passing
on
60
some
percent
of
them,
then
if
you're
passing
on
67
of
them
and
and
we
can
just
merge
it
right
and
if
we
end
up
with
because
let's
see,
let's
yeah,
if
we.
A
Yeah
yeah
definitely
some
test
cases
because
I
think
yeah.
It
looks
like
that
got
pulled
out
or
we
had
talked
about
it
at
one
point.
A
But
so
so,
let's,
let's
get
the
test
cases
in
and
let's
merge
that
and
let's
see
and
then
I
think
I
think
we'll
go
we'll
we'll
just
merge
it
in
to
master
from
there
and
make
sure
that
you
know
the
test
case
is
running
on
the
delta
and
that
way,
if
we,
if
we
end
up
with
it
with
a
job,
if
if
we
end
up
with
a
kind
of
like
the
the
change
log
test
right,
if
we
end
up
with
a
commit
or
if
we
end
up
with
a
pr
that
violates
the
the
test,
then
we
can
still
merge
the
pr
right
and
the
test
won't
fail
in
master
right
because
it's
just
running
on
the
delta.
A
D
A
Exactly
right
because
we're
only
looking
at
the
delta
and
and
and
we
can
choose
at
the
end
of
the
day
to
merge
or
not
merge,
you
know
if
it
looks
acceptable
or
not.
Let's
make
sure
to
run
on
the
delta
commitments.
A
D
D
D
A
That's
great
great
great!
Thank
you.
I
think
this
will
be
great,
because
this
is
a
common
thing
that
will
help
keep
us
honest
here
and
then
suit
honshu.
So
I
noticed
that
the
ice
cream
sales
is
almost
done.
I
think
the
main
thing
was
okay
six
hours
ago
great,
so
you
got
the
link
perfect.
I
didn't
see
an
update
for
that.
For
some
reason.
I
don't
know
why
it
didn't
tell
me:
okay
great.
This
would
be
great
yeah.
We
want
to
make
sure
we
put
this
in
the
change.
A
Log
right
user
facing
stuff
definitely
needs
to
go
in
there.
Okay,
great,
let's
merge
this
on
in
anything
else.
On
your
end,.
E
A
Okay,
let's
check
that
out.
A
D
Are
you
running
this
command
not
from
the
development
environment
and
the
one
which
is
installed
from
prep
inside
your
directory,
something
like
that.
E
D
D
This
is
not
in
this
is
dev
commands
work
in
dev
mode.
A
Okay,
so
we
know
this,
I
think
that
let's
try
uninstalling.
A
With
dashi
yeah,
this
happens
to
me
every
once
in
a
while
too
yeah.
I
think
you're
right
that
that
it's
it's
the
death
mode
thing
so
and
we'll
just
you
know,
we'll
keep
the
issue
up
to
date
for
anybody
else
in
the
future.
Right.
E
A
So
let's
make
sure
we
have
all
the
commands
and
then
let's
see,
let's
comment
back
to
see
if
this
works
or
not
and
close
it.
If
it
worked
out
if
it
didn't
work
out,
then
let's
keep
updating
the
issue
because
I'm
sure
somebody
else
will
run
into
the
same
thing
eventually.
I
know
I
have
similarly.
D
But
the
thing
was
that
what
sort
of
code
is
legal
and
what
is
not,
because
that
example,
with
time
log
one
I
was
a
bit
confused
like
it,
was
used
in
another.
A
D
A
A
Is
a
automated
check
for
it's
a
I'm
trying
to
make
sure
that
we
we've
got
it,
and
so
obviously
I
had
added
that
and
I
had
forgotten
to
remove
it
actually.
A
Yeah
exactly
right
exactly
that
makes
it
all
the
more
confusing.
So
you
definitely
did
the
right
thing.
That's
something
that
that
I
had
forgotten
to
remove,
and
so
we
need
to
go
to
remove
that,
and
I
don't
think
that
we
have
any
use
for
it
at
this
point
because
we
decided
not
to
do
that
pinning.
So
I'm
going
to
go,
I'm
going
to,
I
think,
let's
add
a
issue
here.
So
let's
remove
is
that
I
think
it
might
already
even
have
an.
A
A
Oh
yeah,
oh
no,
never
mind.
This
is
this
one
died
and
then
we
need
a
new
one
to
track
that.
So
let's
see
service
stuff.
A
So
we
need
to
also
document
the
fact
that
that,
on
the
contributing
guidelines,.
A
A
A
new
set
of
dependencies,
no
dependencies,
yeah,
okay,
so
yeah.
So
this
is
a
this-
is
a
really
old
one.
Okay,
let's
just
move
this.
Let
me
just
pin
this
and
assign
it
to
myself,
so
I
think
I
have
a
few
more
things
that
I
have
not
articulated
about
the
internal
processes
that
I
have
to
unfortunately
make
sure
that
we
we
follow.
Somebody,
my
boss,
told
me
they're
doing
audits.
A
If
I
fail
an
audit,
because
I'm
the
one
that's
responsible
for
the
legal
process
on
this
so
we'll
see,
hopefully
everything
is
up
up
to
up
to
par
here,
anyways,
all
right,
so
anything
else
for
today
or
that
we
need
to
make
sure
that
we
have
on
the
agenda
for
next
week.
A
Okay,
well,
you
got
one
for
tomorrow,
so
that
was
the
problem.
So
okay,
so
let
me
let
me
move
this
to
next
week.
Okay,
save
send
and
then
sudhanshu,
I
need
to
yeah.
Oh
and
I
didn't
get
your
email.
That's
why?
A
Okay,
let
me
grab
your
email,
so
sue,
honshu,
yep,
we're
just
trying
to
make
one-on-ones,
let's
see
so
so
all
of
the
all
the
mentors
are
gonna
have
a
one-on-one
with
which
each
mentor
will
have
a
one-on-one
with
each
student
and
and
that
way
you
know
we
get
we
get.
We
make
sure
everybody
has
touch
base,
let's
see,
and
so-
and
this
is
next
week.
A
E
A
Okay,
so
this
is
not
my
correct
calendar
here,
but
I
believe
I
had
another
30
minute
slot
after
that,
so
we
could
do
because
so
sahil
and
I
will
meet
at
9am
on
wednesday.
Do
you
want
to
meet
it
at?
Can
you
meet
like
24
hours
or
let's
say
that
that
would
be
20
23
hours
from
now
on
on
wednes
so
next
week,
basically
next
week,
what
would
be
our
usual
meeting?
A
A
Oh
yes,
does
that
work
for
you
all
right
great,
so
I
will
send
you
an
email,
invite
right
now
then,
and
we'll
put
that
on
the
calendar,
all
right
great,
so
now
we're
all
scheduled
here
and
so,
and
so
since
it's
next
week,
part
of
the
idea
here
was
that
you
know
if,
if
you
guys
end
up
talking
to
yash
and
saksham
and
himanshu
before,
then
then
you
know,
maybe
because
you're
you're
sort
of
right
now
is
the
time
that
we're
gonna
flush
out
your
proposals
right
and
and
and
as
we
move
into
next
week
with
this
seventh
will
be
the
start
of
the
coding
period.
A
You
know
we'll
try
to
make
sure
you
know
my
meetings
with
you
will
be
towards
towards
the
closer
side
of
that,
so
we're
trying
to
go.
If
you,
if
you
have
any
you
know
pre-work
now
right
things
that
we're
looking
at
with
the
config.
A
So
if
you
want
to
look
more
at
the
config
stuff
in
that
architecture,
document
and
I'll
post
that
and
think
more
about
that,
and
look
more
at
that.
That
would
probably
be
good
for
your
project,
and
then
you
know
we
and
we
can
talk
between
then
as
well,
but
but
that'll
that'll
have
us
help
us
have
them,
have
an
informed
set
discussion
right.
So,
let's
try
to
let's
try
to
come,
come
prepared
with.
You
know,
with
with
things
that
you
you
you
you
want
to
talk
about.
A
You
know
with
within
cloaks
props
proximity
within
starting
your
project,
right,
okay,
great
cool-
and
this
is
another
thing.
So
this
is
you
guys
probably
just
saw
this.
So
what
I
realized
is
just
a
meeting
etiquette
thing.
I've
noticed
that
that
that
that
a
good
way-
and
this
is
like
outlook-
does
this
by
default,
and
I
realized.
Why
is
if
you
send
somebody
else,
a
meeting,
then
you
should
put
there
and
you
should
put
your
name
first
so
that
they
know
who
they
were
they're
meeting
with
right
away.
A
In
case
the
names
end
up
cutting
off.
So
if
because,
if
you
want
to
meet
with
somebody,
then
you
know
you
want
to
make
sure
that
they
know
who
they're
meeting
with
because
they
know
their
name.
So
when
they.
B
A
A
Who
they're
dealing
with
immediately
so
just
a
general
general
thing?
I
realized
that.
Finally,
the
other
day
I
was
like,
why
does
it
always
do
that
when
I
finally
got
it
when
the
names
got
cut
off,
so
something
to
something
to
remember
all
right?
Well,
thank
you
guys
good
talking
to
you
today
and
have
a
great.
A
A
A
Okay
for
notebooks.
A
And
this
has
to
do
so,
I'm
currently
in
progress
refactoring
the
console
test
stuff.
It's
been
been
a
couple
weeks
since
I
actually
got
the
chance
to
touch
it,
but
I
I'm
I'm
pretty
far
along.
I
hope
knock
on
wood,
but
the
way
that
we
have
this
right
now
is
basically
any
so.
This
is
this.
Is
your
friend
this
this
area
right
here?
So
basically,
this
code
here
goes
and
and
and
recursively
loads,
every
single
object
in
dffml.
A
So
within
the
main,
the
main
package
and
this
code
right
here
goes
and
goes
and
basically
looks
at
the
object,
looks
at
the
name
and
checks
if
there's
any
any
doc,
testable
doc
strings
in
the
object.
If
it
is,
it
makes
a
test
case
out
of
it
and
it
makes
a
dock
test
test
case
out
of
it,
which
basically
says
you
know.
Okay,
it's
going
to
create
a
function.
A
Hey
wait
a
minute!
I
thought
I
merged
to
change
this
okay.
This
is
actually
creating
a
tuple
which
is
not
a
test
case.
God
damn
it.
I
swear.
I
merged
the
damn
change
to
this
okay
anyways.
As
soon
as
I
push
the
fix
to
this,
there
will
be.
This
will
create
a
test
case,
not
a
tuple
and
and
any
test
case.
A
You
know
anything,
that's
a
function
that
has
test
underscore
will
be
run
by
unit
test,
because
what
we
do
is
we
create
an
instance
of
the
async
test
case
class
for
this
and
you'll
see
it
right
here.
So
it
does
a
full
path
right.
So,
if
you
wanted
to
run
a
the
dock
test
for
one
of
these,
you
know
you'd
say
cli,
cli
version
version
git
hash,
and
this
looks
like
it's
repeated,
which
is
not
good,
send
test
test
doc
string.
A
So
this
would
be
you
know
the
doc
test
of
it.
I
believe
right,
and
so
if
it
had
a
console
test,
so
if
it
has
any
console
test
directives
in
it,
you
know
we
parse
out
parse
the
docs
string
check
if
any
object
has
console
tests
and
automatically
run
the
console
test.
A
So
what
we'd
be
doing
when
I
say
automatically
run,
you
know
automatically
or
automatic
test
case
creation
for
notebooks
is
basically
just
this
here
is
looking
at.
This
is
like
a
recursive
import.
You
just
wanna
would
wanna
you
know:
recursively,
do
like
an
r
glob
star,
ipython
notebook
and
look
for
any
by
python
notebook
files
and
create
a
test
case
class.
You
know
with
the
test
function
in
the
for
that
class
that
will
test
the
notebook
right
so
that
way,
anytime
somebody
adds
a
notebook
file
to
the
code
base.
A
A
test
case
class
gets
instantiated
dynamically
within
some
python
file,
which
means
that
you
know,
then
the
test
case
will
get
run.
A
Does
that
make
sense
oh
yeah,
and
that
that
way
it
prevents
us
from
adding
notebooks
that
don't
have
tests,
and
then
we
can
sort
of
you
know
if
we
have
a
notebook
that
that
we
want
to
override
and
make
the
testing
you
know
slightly
different,
then
we
can
just
override
you
know
specific
methods
or
something,
and
in
addition
to
that,
the
it
will
allow
us
to
like
for
that
for
the
stuff
that
you
did
where
you
ran
some
cells,
but
not
other
cells.
A
I
was
wondering
there
may
be
a
way
to
sort
of
programmatically
catch,
all
invocations
of
cash
download,
with
like
unit
test
mock
and
and
just
modify.
That
first
call
or
that
that
file
path
parameter.
But
then
you
know
you're
dealing
with
also
your
search
statement
there
and
stuff
like
that.
So
we'll
we'll
have
to
figure
that
out,
but
but
ideally
this
makes
it
so
that
we
don't
have
to
so
that
we
can.
We
can.
A
We
can
have
a
uniform
approach
to
testing
all
the
notebooks
and
and
then
and
not
have
anybody
who
writes
a
notebook
have
to
deal
with
testing
as
well
right.
A
All
right
and-
and
you
know
this
is
something
you
know
that's
kind
of
like
you
know,
let's,
let's
get
to
written,
let's
say:
let's
do,
and
this
is
something
for
your
proposal
specifically,
but
but
let's
make
sure
that
you've
written
another
notebook
and
then
we
can
look
at
the
test
cases
of
each
notebook
and
we
can
say
hey,
you
know
what
would
one
generic
test
case
be
right
and
then
at
that
point
you
know
yeah.
A
You
have
your
generic
test
case
and
and
then
you
know,
as
you
write
the
other
notebooks,
you
modify
the
generic
test
test
case
as
necessary
or
you
don't
write
and
you
don't
have
to
keep
writing
test
cases
right.
You
just
write
notebooks.
C
A
So
yeah,
I
wouldn't
worry
about
it
for
now.
Basically
just
write,
you
know,
if
you
want
to
work
on
the
the
notebook
stuff,
then
then
keep
writing
the
notebooks.
Otherwise.
So
I
need
to
the
the
other
thing.
Is
that
that
cache?
So
the
the
splitting
out
the
confidence
and
the
prediction
keywords,
I
realized
that
that's
going
to
have
probably
large
implications
on
that
refactor,
that's
going
on
with
accuracy
so
sudhanchu.
A
I
know
that
you
are
working
on
on
phase
eight
as
well,
so
I
was
thinking
let's
try
to
get
that
merged,
so
we
don't
end
up
with
a
massive
rebase
conflict
in
there
because
they
think
it's
gonna
be
more
straightforward
to
rebase
the
the
you
know,
any
any
changes
to
the
the
confidence
or
the
confidence
and
prediction
on
top
of
accuracy
changes
than
it
will
be
to
to
do
it
into
the
accuracy
changes
since
that's
so
complicated
they're,
both
very
complicated
large
changes,
the
accuracy
one
just
has
like
hundreds
of
commits,
so
I
think
that
that's
probably
gonna
be
more
more
labor
intensive.
A
A
A
Cool
cool
all
right,
well,
great,
all
right,
thanks
everyone
and
we'll
we'll
talk
on
on
twitter
and
we'll
talk
more
next
week
have
a
good
one.
Oh
and
congratulations,
of
course,
to
all
of
you
good,
very
good,
very
good
work,
and
you
all
did
great
pre-work
and
great
proposals
so
very
happy
to
have
you
on
board.