►
From YouTube: Weekly Sync: 2022-04-15
Description
Meeting Minutes: https://docs.google.com/document/d/1vKYEPtqKiwsFwhVKPmPub5ebMqN9HteBcbdFAuTXalM/edit#heading=h.1f4g9rj08f5e
A
B
B
So
this
is
the
old
version
of
the
docs
that
has
the
image
in
it,
and
so
this
is
a
good
one
to
look
at
the
I,
I
believe,
you're
talking
about
the
contributions
section
here
and
so
the
contributions
section,
your
your
this
is,
you
know
norm,
so
it's
normalized
for
six
being.
Whoever
did
the
most
work
now,
not
not.
Many
people
have
gotten
prs
merged
this
year
because
of
the
whole,
like
things
have
been
rather
disorganized
this
year
over
previous
years.
A
B
A
A
B
A
A
B
B
A
B
Okay,
so
all
right,
yeah,
let's
so,
let's
jump
over
to
so
what
are
you?
What
are
you
thinking
about?
We
can
spend
this
time
since
there's
nobody
else
here
today.
You
know
talking
about
you
know
what
what
what
your
thoughts
are,
at
least
until
anybody
else
arrives,
and
you
know
we
can
just
workshop
together.
So
so
so
I'm
sorry,
how
did
you,
how
do
you
pronounce
your
name
again.
A
B
B
A
B
A
A
Started
like
about
four
months
ago,
I've
been
like
really
into
artificial
intelligence,
machine
learning
and
stuff.
So
the
time
series
thing
it
kind
of
resonated
with
the
type
of
stuff
I
was
already
into
right
so
like
in
the
time
series
thing
I
was
attracted
to
the
I
I
was
researching
it,
so
I
I
was
attracted
to
the
concept
of
the
decomposition
of
a
time
series
graph,
okay,
yeah,
the
there's,
a
trend
and
a
seasonality
and
something
else
on
that
residues.
C
A
B
A
B
B
B
A
B
A
I
was
looking
at
the
iris
data
set
and
one
thing
that
struck
me
as
a
little
odd
was
that
we
were
writing.
Individual
codes
for
specific
data
sets
right
like
for
the
irs
data
set.
We
were
writing
one
and
if
we
were
to
introduce
another
data
set,
we
have
to
write
another.
So
why
can't
we
just
make
that
into
one
module
and
then
just
apply
it
new
datasets.
B
A
B
Is
like
you
know,
here's
you.
We
want
to
take
your
idea
and
we
want
to
shoot
it
to
production
as
fast
as
possible
right
and
so,
as
the
steps
along
you
know,
the
steps
along
and
that's
kind
of
the
core.
That's
part
of
dfml's
core
mission
right
is
to
try
to
figure
out.
How
do
we
take
anybody
who
wants
to
do
something
related
to
machine
learning
from
the
idea
to
boom
into
production
in
existing
application
as
fast
as
possible?
Right
and
we'll?
B
B
B
This
is
basically
the
core,
the
core
core
premise
of
dfmo
decoupled
from
dataset
access,
so
so
first
first
solution.
So
first
attempt
so
write
model
access
data
set
directly
or
actually
this
is
just
you
know,
access
data
set
directly
just
to
confirm
that
you
can
do
it.
You
know
right,
you
can
read
file,
connect
to
database,
etc.
B
B
B
Okay,
yeah,
and
so
this
is
a
generally
applicable
process
right.
So
when
you're
saying
okay,
so
so
we
do
one,
we
do
the
next
one
another
one
come
up
with
an
interface
that
works
for
both.
In
this
case
we
came
up
with
sources
and
that's
the
class
right,
okay
and
then
fourth,
you
know
expose
first
and
second
via
interface,
created
in
third,
okay,
so.
B
B
B
A
B
B
B
B
A
B
Yeah,
okay,
so
so
you
can
think
of
this
as
like
building
layers
onto
the
software
stack
right,
you're,
building,
layers
of
abstraction
right,
and
so
what
we
did
first
is
we
first
built
this
layer
of
abstraction
around
a
generic
data
set
itself
right,
so
maybe
a
csv
file
or
my
sql
database
right,
and
so
the
first
thing
we
did
was
we
accessed
the
csv
file
directly.
We
accessed
this
mysql
database
directly
and
then
we
came
up
with
this
sources,
abstraction
or
source
class
right
right.
B
That
allows
us
that
that
we
then
wrap
and
let
me
go
ahead
and
open
it
up-
that
we
then
wrap
the
implementations
that
we
came
up
with
and
we
expose
them
through
those.
You
know
we
we
implement
the
class,
basically
right,
and
so
then
we
move
on
to
the
next
problem
right
and
then
the
next
problem
is
so.
This
would
be.
You
know
this
is
the
general
pattern
and
then
pattern.
A
B
B
C
B
Sahil,
let
me
let
you
know
what
we're
doing
right
now
and
then
we'll
we'll
get
to
your
agenda
items.
So
just
so,
you
know
where
we're
at
so
you
said
your
nickname
is
tintin
all
right.
So
tintin
is
working
on
he's
working
on
a
proposal
for
the
time
series
project
and
so
he's
interested
in
in
the
decomposition
of
time,
series
graphs,
including
trends,
seasonality
and
he
wants
to
add
some
more
data
sets
and
he
was
looking
at
the
irs
data
set
and
said.
B
You
know
it's
odd,
that
we're
writing
code
for
each
data
set.
Why
not
something
write
something
generic
that
allows
us
for
accessing
any
data
set,
so
we're
taking
this
opportunity
to
understand.
You
know
how
we
add
a
light
layer
to
the
software
stack,
the
layer
of
abstraction
how
we
came
about
you
know
with
the
sources
abstraction
in
general
and
then
how
we
can
extend
that
by
we're
gonna.
Look,
we're
gonna,
extend
that
general
pattern
write
it
we're
just
gonna
wrap
a
existing
data
set
real
quick.
B
C
B
B
Some
people
don't
have
the
option
to
do.
Google
docs,
I
know
great
great
great
firewall,
china
and
stuff
yeah
I'm
trying
to.
Basically,
if,
if
I
see
if
I
see
it
I'll
provide
feedback,
I
haven't
you
know
so.
Basically,
people
who
email
my
work,
email
have
gotten
responses,
because
that's
what
I've
been
monitoring
lately
and
so.
C
B
B
Things
that
you
might
need
to
do
is
pre-work
things
that
you
might
need
to
do
things
that
you
might
need
to
explore
to
make
sure
that
your
proposal,
like
things
that
might
be
helpful,
pre-work,
which
are
related
to
your
proposal,
which
you
could
do
as
prs
and
things
that
you
know
you
need
to
look
more
into
probably
because
your
proposal
may
not
align
exactly
with
what
it
within
the
code
base.
Any
other
comments.
A
C
A
C
So
as
as
someone
reviewing
this,
I
would
say
that
if
it
is
to
the
point
and
not
very
verbose
instead,
it
just
delivers
it
as
in
a
small
form
factor
not
just
for
short,
but
in
a
reasonable
amount
of
detail
that
what
that
would
do
writing
long
paragraphs
is
just
you
know.
We
have
to
dig
through
it
and
read
through
it.
That
is
something
something
that
I
would
say
is
not
not
possible
for
everyone
in
the
day
to
day,
because,
like
I
am
or
working
somewhere,
john
is
also.
C
A
C
B
C
B
C
B
C
B
Let's
go
ahead
here,
so
we're
gonna
have
okay,
so
typical
feedback.
What
did
we
talk
about?
We
talked
about
you
know
areas
you
need
to
understand
more.
You
need
to
read
more
existing
docs
to
understand
how
things
currently
work
at
places,
issues
or
potential
prs
where
that
might
be
helpful.
What
else
do
we
say.
A
The
full
request
I
sent
about
the
file
structuring
and
you
know
you
pointed
out
that
it
wasn't
actually
needed,
so
I
am
which
one
the
relative
import
one.
Yes
that
one!
Yes,
yes,
now
like,
I
get
that
that
was
a
rookie
mistake
sort
of
thing,
but
I
sort
of
incorporated
that
as
a
closed
pr
in
my
proposal
would
that
be.
C
C
Yes,
that
kind
of
explains
it
and
it
is
fine,
but
I
wouldn't
say
that
so
what
we,
what
would
count
as
a
contribution
and
what
would
not?
So,
in
my
view-
maybe
maybe
john,
can
add
to
this.
In
my
view,
what
would
have
come
to
the
contribution
is
something
that
has
been
merged
or
that
has
been
blocked
due
to
something
else.
C
C
B
B
C
B
C
That
is,
that
is
pretty
much
it.
I
guess
you
know
like
so
for
every
types
of
data
there
would
be
some
atomic
type
like
for
our
csv.
A
row
is
atomic
type
for
sql.
There
is
a
record
which
is
called
the
tuple
is
something
like
that
and
in
the
fml
we
have
a
record,
which
is
a
dictionary,
and
maybe
it's
on.
B
B
Google
docs,
which
is
that
we
have
this
source
abstraction
right,
and
so
the
record
is
the
abstraction
around
you
know,
whatever
one
of
those
rows
are
so
like
a
row
in
a
csv
file
or
a
row
in
a
database,
or
you
know
an
entry
in
a
json
list
right
so
that
that
is
a
record
so
that-
and
so
we
talked
about
you
know,
how
did
we
get
to
the
source
abstraction?
Well,
we
applied
the
same
pattern
to
get
to
the
record
abstraction
for
each.
You
know
piece
of
data
within
a
source.
B
Right,
so
does
that
make
sense
a
little
bit:
okay,
okay,
yeah!
You
can
just
think
of
it,
because
we
have
this
generic
thing
right
where
we
want
to
be
able
to
access.
We
want
our
models
to
be
separate
from
our
sources
so
that
we
can
use
any
model
with
any
source
right.
Then
we
also
need
an
abstraction,
for
you
know
each
entry
within
that
source
right
or
within
any
source-
and
that's
just
a
record-
is
what
we're
calling
that
right,
yeah
so
and
then
okay,
so
anything
anything
else.
B
So
we
talked
about
okay,
so
mentor
login.
So
I'll
get
on
that
one.
Let's
see
mentor
login!
Okay,
sahil!
Could
you
could
you
send
me
a
email
to
my
to
my
work
email
just
because
the
other
day
I
got
off
a
call
with
somebody,
and
I
I
did
the
exact
opposite
of
what
we
had
just
talked
about
for
like
15
minutes.
So
if
we
don't
write
things
down,
I'm
liable
to
do
the
wrong
thing.
C
B
It's
it's
yeah,
I
just
actually.
This
is
oh,
let's
finish
this,
so
it's
this
john.s
anderson
at
intel.com.
So
thank
you
yeah,
if
you
just
just
put
it
in
the
subject
that
way
because,
like
you,
said
proprietary,
we're
on
the
recording,
I
don't
want
to
switch
over
to
my
calendar
or
email
right
now.
Yeah,
okay,
great,
thank
you!
Okay.
So
this
is,
I
think
we
captured
in
you,
know
some
stuff
here
and
then
let's
say:
okay.
We
also
want
to
grab
hashem's
name.
B
So
hashem
is
also
mentoring
this
year
and
I
think
we
we
may
have
some
other
mentors
involved,
but
I
don't
we
I
haven't
reached
out
to
that.
As
many
people
I
usually
reach
out
to
many
people,
but
I
haven't
had
a
chance
to
reach
out
to
many
people
this
year,
previous
students
and
such
and
I
know
some
of
our
previous
students
were
actually
trying
to
start
their
own
org,
but
I
haven't
synced
with
them
on
how
that
went
for
them
in
a
while.
So
it
it's
sort
of
it's
right
now.
B
B
B
It's
still
sort
of
in
progress,
I
would
say
we're
not
that's
not
fixed
right
now,
so
let
me
just
transform
and
transform.
We
need
to
crop
okay,
never
mind.
This
is
I'm
just
gonna.
Leave
that
one
okay,
so
gsuc22,
I
add
what
did
we
do
here?
So
we're
adding?
You
know
some
clarification
around
proposals
so.
B
We.
So
when
you,
when
we
change
the
branch
name
yeah
it
it,
it
was
an
opportunity
for
us
to
go
through
and
clean
up
these
files
as
well.
Now
we
didn't
totally
get
all
of
that,
but
yeah
that
the
file
cleanup
proved
to
be
a
lot
trickier
than
than
originally
expected.
So
we'll
probably
yeah
we'll
have
to
we'll
have
to
that.
That's
gonna
need
to
be
dealt
with
remorse.
B
Oh
yeah
for
oh
fork,
I
see
which
media
fork
so
so
there
is
so
okay,
so
we'll
we'll
only
spend
a
couple
more
seconds
on
this
and
then
I
want
to
go
write
the
data
set,
but
we
are
trying
to
get
the
project
transition
to
an
external
government
governance
structure.
That's
non-intel,
long-term,
to
enable
people
to
become
maintainers
of
the
project.
Because,
right
now
the
permission
settings
are
locked
down.
A
B
Intel
itself,
so
I'm
in
the
process
of
talking
to
legal
about
that
and
they've
okayed
it
conditionally
on
government's
documentation
from
the
buildtree
org
right
now
and
so
we'll
see
we'll
see
how
that
evolves.
It
stalled
out-
and
that
was
part
of
our
previous
mentors,
we're
running
that
some
of
our
previous
mentors
yash
and
saksham
and
some
others,
but
but
I
haven't
heard
from
them
in
a
while.
I
think
they
got
just
same
same
as
all
of
us.
You
know
your
day.
Job
is,
can
be
a
lot
of
work
right.
B
B
A
B
A
good
question:
that's
a
very
good
question,
so
so
you
know
so
so
I
work
for
intel
and
this
project
was
originally
developed
at
intel
and
then
until
you
know,
we
worked
it
through
until
every
company
has
a
you
know,
different
processes
around
how
they
deal
with
open
source
intel
is
pretty
good
about
it,
and
so
we
were
able
to
open
source
the
project,
and
then
you
know
we
can.
We
can
do
gsoc
and
things
which
is
cool.
A
B
And
so
the
python
software
foundation
is,
you
know
a
within
gsoc.
There
are
top
level
orgs
and
suborgs,
and
so
we're
suborg
of
the
python
software
foundation,
which
means
that
the
python
software
foundation
manages
talking
to
google
and
all
the
complexities
involved
there
to
actually
sign
up
as
an
org
with
google
and
then.
B
We
have
a
simplified
process
which
we
can
follow
for
our
suborgs
right
and
we're
on
a
much
smaller
scale
right
than
the
python
software
foundation,
because
they're
a
conglomeration
of
other
things,
and
so
so
yeah,
that's
the
general
gist
of
it
and
good
good
friend
of
mine
and
co-worker
terry.
She
runs
the
python
software
foundation,
gsoc
involvement,
and
so
she
runs.
B
Called
cbe
bin
tool,
which
is
a
really
cool
project,
and
I
recommend
you
guys
like
that.
You
check
it
out.
If
you
haven't
checked
it
out,
it's
not
not
machine
learning,
focus
security,
focus,
really
cool
stuff,
though
really
important,
stuff
and
yeah.
That's
it.
That's
a
another
good
good
project,
so
yeah,
so
she
she
she
got
us
involved
as
well.
Okay,
so
does
that
answer
your
question.
B
A
B
A
B
B
So
for
step,
one,
we
implemented
this
iris
training
right,
and
so
this
is
the
meat
of
it
right
here
this
we
basically
we
download
the
data
set
and
then
the
data
set
itself
is
really
just
a
csv
file.
So
then
we
just
you
know,
yield
the
appropriate
source
and
you'll
see
heavy
heavy
heavy
usage
of
context,
management
and
async
in
dffml.
B
A
B
Yeah
and
and
the
reason
for
that
is
context,
managers
allow
us
reliable
cleanup
of
any
resources
used
and
asynchronous
methods.
Allow
us
you
know,
network
access
in
a
concurrent
way,
because
a
lot
of
things
are,
I
o
bound
right
and
so
most
most
of
most
things
are.
I
o
bound
machine
learning
models
and
stuff
are
cpu
bound.
We
can
then
schedule
those
out
to
threads,
so
this
is,
you
can
think
of
this
as
as
attempt
one
right.
So
we
wrote
this
thing,
that's
like
okay!
B
Well,
how
would
we
implement?
You
know
a
simple,
abstract
or
a
simple
source?
Well,
we
would
download
the
download
the
data
set
like
because
this
sources,
abstraction
is
around
generic
sources
in
general,
and
then
the
data
set
source
abstraction,
which
is
this
next
layer
which
we're
adding,
and
maybe
I
should.
We
should
probably
say
that
so
pattern
applied
to
specific
data
sets,
develop
the
data
source,
abstraction,
okay.
B
B
B
Okay,
so
in
this
example
here
we
are
in
this
example.
Here
we
say
all
right
like
this
is
the
abstraction
itself
that
we
ended
up,
adding
as
this
layer
right
and
here's
the
training
data
right.
So
instead
of
the
iris
data
set,
that's
a
csv
file,
it's
this
we're
downloading
it
from
some
server
and
when
we
download
it.
This
is
what
we
do
right.
B
There's
not
a
lot
of
network
access
in
this
library
in
in
the
base
library
itself,
but
when
we
do
do
it,
we
have
this
helper
function,
cache
download
and
the
reason
why
we
always
go
or
you
don't
have
to
go
through
cache
download
it's
just
typically,
if
you
download
something
you
don't
want
to
re-download
it
right.
No.
B
A
B
When
you
think
about
when
you
hear
about
like
the
l2
or
the
l1
or
the
l3
cache
right
on
a
cpu,
that
is
just
a
place
where
we
can
store
stuff,
that's
usually
in
main
memory,
but
we
don't
have
to
go
out
to
main
memory,
because
that
takes
lots
of
cycles
right.
So
cache
is
just
you
know
more
of
a
word.
It's
the
word
itself
is
used
just
for
the
for.
A
B
B
Is
that
we
have
for
security
reasons,
these
the
synchronous
download
function
which
this
cache
download
is
backed
by
and
then
cache
downloaded
itself?
Have
a
protocol
allow
list
which
basically,
the
the
purpose
of
this
is
to
stop
people
from
accidentally
passing
it
http
links
which
are
insecure
right.
So
we
do
hash
validation
as
well,
which
is
another
thing.
B
Exactly
and
so
basically,
if
the
hash
doesn't
match,
we
either
re-download
or
throw
an
error,
and
if
somebody
tries
to
you
know,
send
you
know,
give
us
a
link.
That's
in
you
know
an
insecure
link,
we're
going
to
throw
an
error
right,
and
so
here
we're
explicitly
allowing
the
use
of
tls
encrypted
http
access
right
by
by
adding
this
to
the
allow
list
so
and
this
kind
of
gets
in.
B
So
basically,
what
we've
done
here
is
we,
let's
see
so
write
another
one
and
then
I'll
just
put
this
example
right,
and
so
that's
the
example
and
then
you
know
come
up
with
the
interface
that
works
for
both
and
here's
the
same
link.
Basically
because
you
know
this
is
that
that's
the
point
of
this
is
that
we,
the
second
one,
was
ended
up
being
the
the
example
for
the
abstraction
itself.
B
So,
okay,
so
then
your
question
is
why
don't
we
have
something
that
that
you
know
we'll
go
back
to
your
question,
which
was
you
know,
can't
we
just
have
something
that
works
for
everything,
and
you
know
the
answer.
Is
we've
now
built
this?
We've
we've
built
this.
We
did
our
first
attempt.
We
did
our
second
attempt.
B
We
wrote
our
abstraction
layer
right
so
now
we
can
start
basically
saying
well
what
are
all
the
things
that
we
might
want
to
access
right
and
if
the
the
pattern
we're
going
to
follow
the
same
pattern,
basically
for
the
third,
this
third
level
right.
So
the
first
level
was
access
data
access,
a
data
source
right
and
then
the
second
step
was
access.
Specific
data
sets
right,
which
might
be
backed
by
a
specific
data
source
right,
which
is
like
a
csv
file
or
whatever.
Now
the
third
step
is
well,
we
need
to
figure
out.
B
How
do
we
access
an
arbitrary
data
set
right?
So
we
need
to
probably
do
this
for
a
lot
of
different
data
sets
right,
because
we
only
did
two
if
we're
going
to
make
something
that
accesses
an
arbitrary
data
set.
We
need
to
have
a
really
good
understanding
of
all
the
types
of
ways
that
that
might
be
accessed
and
the
only
way
to
get
that
is
to
implement,
implement,
implement
right.
So
part
of
this
project
is
to
go
through
and
implement
right.
So
so
we
had
a.
B
B
Yes,
and
actually
here
here,
we'll
do
one
right
here,
which
is
the:
why
don't
we
do
mnist,
because
that's
a
that's
an
easy
one,
and
we
already
have
an
example
for
that
and
we
don't
have
a
data
set
source
for
that,
so
we
can
basically
go
ahead
and
we're
gonna
get
kicked
off
here
so
we'll
see
if
we
can
do
it
in
like
seven
minutes.
So
this
is
the
this
is
the
deal.
Here's
all
our
shop384
sums,
here's
our
link
to
the
to
the
data.
B
B
You
know
I
need,
I
think
I
might.
I
think
my
calendar
might
have
gotten
cleared,
but
if
we
get
kicked,
let
me
actually
either
either
it
got
cleared
or
I'm
missing
the
meeting
right
now.
I
think
it's
fine,
I
think
we're
fine.
I,
the
past
few
weeks,
I've
accidentally
run
over
another
meeting.
Doing
this.
C
B
That
sounds
good,
so,
let's,
let's
yeah,
let's
let's
plan
on
that,
then
so
I
think
I
think
that
that's
that's.
I
think
that's
doable
documents,
python,
okay!
So
let's
take
a
look
at
what
we
did
here.
So
I'm
gonna
remove
this
file.
I
don't
think
we
need
this.
I
think
that
just
generated
this
one
off
all
right
so
so,
basically,
I
just
copied
the
base
into
the
new
file,
which
is
the
mnist
right,
because
the
base
has
this
example
here.
So
actually,
actually
I'm
not
gonna
do
that.
B
I'm
gonna
copy
the
iris,
because
this
is
one
that
has
a
this,
has
the
whole
url
and
everything
okay.
So
let's
do
mnist
training,
yeah
and
then
the
question
will
be
you
know
how
do
we
then
extend
this
to
be
arbitrary
right,
and
so
you
might
end
up
like
eventually
we'll
end
up
with
something
once
we
implement
a
few
of
these,
that's
really
obvious
that
that
you
know
we
can
basically
just
say
you
know,
source.
B
B
B
So?
Okay,
so
we're
grabbing
actually
we're
grabbing
from
multiple
sources.
That's
right!
Okay,
sourcing,
which
is
data
flow?
Oh
okay,
that's
right!
So
it
was
tricky
one
okay,
so
we're
basically
actually
gonna
create.
So
what
we
see
in
the
usage
here
is
that
we're
going
to
oh,
this
would
be
fun
okay,
so
we're
actually
going
to
do
this
example
as
a
data
set
source.
B
B
Google
has
limited
the
meetings
to
an
hour,
so
we'll
have
to
start
a
new
meeting
link.
I
don't
think
this
will
take.
You
know
more
than
like
15
minutes
here,
but
it's
definitely
gonna
take
more
than
three
minutes
so
so
we'll
we'll.
I
think
we
can
run
into
that,
and
then
I
can
confirm
that
I'm
not
in
another
meeting
right
now.
So,
let's
see
so.
B
Basically,
if
you
look
at
this
tutorial
here,
what
we're
doing
is
we're
showing
how
to
how
to
train
a,
I
think,
a
tensorflow
model
yeah,
let's
see
so
yeah,
okay,
so
yeah
we're
going
to
show
how
to
train
the
dnn,
classifier
and
so
effectively.
What
we're
going
to
do
here
is
we're
going
to
end
up
doing
all
of
these
pre-steps
and
we
can
actually
just
actually.
We
can
just
take
this
whole
thing
and
we
can
just
do
all
of
that.
So
our
first
step
is
download
all
the
data
right.
B
Actually,
no
paste
read
docs
examples,
mnist
all
right.
So
our
first
step
is:
we
want
to
download
all
the
data
right,
so
we're
gonna
go
through
and
we're
gonna
cache
download
the
train,
idx,
3
and
train
labels,
because
these
are
these
are
the
things
that
we
care
about,
because
we're
implementing
the
train
data
set
right
now.
B
Right
so
this
this
data
set
itself
download
features
is
made
up
of
you
know
the
the
it's
made
up
of
four
files.
It's
made
up
of
one
for
the
training
features,
one
for
the
training
labels,
the
things
that
we're
going
to
predict
and
then
you
know
two
more
files
for
the
test
label
features
and
labels,
but
we're
not
concerned
with
those
right
now
because
we're
just
doing
the
training
right.
B
B
Okay-
and
that
was
the
images,
and
then
this
is
the
labels.
Okay
and
then
this
is
also
http
links.
So
we'll
you
know
down
we'll
make
sure
that
we're
we're
adding
that
until
the
allow
list
right
so
and
then
we
can
say
you
know,
training
original
features
and
labels
right
so
we'll
download
these
files
to
the
cache
and
so
we're
going
to
download
the
features
and
then
we're
going
to
download
the
labels
and
we're
going
to
download
them
to
this
cache
directory
here,
which
is
in
you
know,
home
cache.
B
Dfml
data
sets
mnist
and
yeah
we're
going
to
validate
the
shas
based
on.
You
know
that
the
values
that
we
had
given
in
our
tutorial
already
right-
and
we
could
calculate
these.
If
you
look
at
the
cache
download
function,
there's
an
example:
it
just
runs
it
through
shot,
384
sum
and
calculates.
The
sha
so
then
looks
like.
We
also
depend
on
model
tensorflow
and
operations
image
here.
So
we're
this.
This,
which
means
this
kind
of,
goes
to
your
question
about.
Why?
B
Don't
we
use
pandas
every
time
we
have
a
new,
distinct
set
of
dependencies?
We
should
have
a
new,
a
new
plug-in
and
that's
because
so
basically
think
a
distinct
set
of
dependencies
would
be
like.
I
have
models
implemented
using
tensorflow
right
now.
I'm
sure
you're
aware
when
you
download
some
of
these
machine
learning
libraries
they
take
a
long
time
to
download
right
and
they.
B
B
B
B
So
in
this
case,
this
is
going
to
require
a
df
model,
tensorflow
and
dfml
operations
image.
So
we're
just
going
to
do
a
quick
check
here
that
we
have
those.
So
we're
gonna
run
the
version
command
and
see.
Okay,
it's
it's
gonna!
It's
gonna!
Look
at
this
file
all
right.
Okay,
so
we
have
a
bunch
of
stuff
in
here.
B
B
If
you
don't
have
this
stuff
installed
and
and
tensorflow
is
not
one
of
them,
but
you
will
need
the
image
operations
right
because
this
to
to
do
this
so
so
to
to
to
use
this
training
data
set,
because
the
the
because
we
have
to
normalize
the
data
to
do
anything
here,
let's
see,
should
we
do
that,
I
think
maybe
we
should
not
do
that
yeah.
I
think
we
should
leave
the
normalization
out
of
this,
so
we're
actually
gonna
we're
actually
gonna
leave
the
normalization
out
of
this
and
basically
say
hey.
You
know.
B
Yeah
exactly
we're
just
going
to
provide
the
raw
data.
We
don't
need
to
do
any
normalization
built
in,
but
the
but
the
cool
thing
is
that
you
could
right
like
what
we're
seeing
here
is
that
we
could,
you
know,
have
mnis.training.normalized
which,
whenever
you
import,
that
kind
of
like
you're
saying
you
know
why
can't
we
just
have
one
for
everything
like
this
one.
You
could
have
one
specifically
that
already
does
that
normalization
built
in
right.
Oh.
A
Yes,
speaking
about
speaking
on
this,
like
while
I
was
writing
my
proposal
on
one
of
the
aspects
of
the
project
was
writing
various
operations
right,
normalization
and
nan
removal
and
stuff.
So
I
was
wondering
like
I.
I
want
to
implement
a
few
of
them
myself,
but
you
guys
would
already
have
frameworks
for
several
of
these
operations
right
and
so
like.
How
do
I
tell
which,
which
ones
need
to
be
which.
B
So
I
would
say:
git
grep
is
your
friend
and
also
a
big,
mainly
because
the
docs
are
not
entirely
up
to
date
and
then
also
the
plugins
page
and
this
and
we
have
a
current.
We
currently
have
a
bit
of
an
issue
with
the
way
the
docs
are
splitting
out,
but
this
is
your
sort
of
your
your
your
main
list
of
what
exists
where
this
plugins
page,
because
this
searches
across
like
we
said
each
time
you
get
a
new,
distinct
set
of
dependencies.
You
implement
a
new
plugin.
B
So
this
all
of
this,
which
is
split
out
right
now
for
some
reason
we
don't
know
why
we
haven't
gotten
to
that
in
a
while.
That
is
just
the
main
package
right.
So
if
you
were
to
like
get
that,
that's
basically
all
of
the
anything
you
see
inside
anything,
you
see.
Okay,
that's
that's
a
bunch
of
pyc
files.
B
C
B
B
Everything
that's
implemented,
then
you
would
go
to
the
plugins
link
and
now
you
can
see
across
all
the
different
plugins
right.
So
everything
like
you
know
in
if
you
were
to
look
in
you,
know
like
this-
is
the
top
level
repo
right
and
so
that
the
api
reference
is
dffml
and
then
there's
also
like
model
and
operations
right
and
within
those
we
have.
This
is
dfml
operation,
binsec
dfml,
operation
image,
which
is
the
one
that
was
referenced
in
this
tutorial
that
we
were
just
looking
at
wherever
that
was
oh
yeah
here
right.
B
So
this
is
this
slash
operation
image
is
this:
this
package
on
pi,
pi
and,
and
so
the
plugins
page
allows
you
to
search
across
those
and
if
the
goal
is
eventually
will
support
third
party
plugins.
So
basically,
you
know
these
are
all
the
plugins
that
we
as
a
community
have
developed,
but
we
also
want
to
make
it
very
easy
to
just
say:
hey,
you
know,
here's
my
rant
like
we
were
talking
about.
You
know
hey.
B
B
Without
writing
any
code
right.
All
it
is,
is
a
command
line
flag.
So
that's
that's
the
goal.
We
aren't
quite
there
yet
right
now,
it's
all
just
within
the
repo
but
but
yeah.
So
okay.
So
where
were
we
here?
So
we're
gonna
do
oops,
okay,
so
basically
and
then
looking
at
our
this
was
you
know
the
this
is
so
the
the
the
python
there's
a
python
api,
an
http
api
and
the
command
line,
interface
and
they're
all
sort
of
mirror
each
other.
B
B
B
B
And
so
this
is
the
images
here,
and
so
we
we
don't
want
to
pre-process
them.
So
we're
basically
just
going
to
say-
and
you
can
see-
they're
they're
being
pre-processed
using
this
data
flow
and
so
we're
basically
just
going
to
cut
that
out,
because
we
don't
want
to
do
any
pre-processing
and
we
know-
and-
and
we
can
see
within
that
pre-processing-
it
was
actually
accessing
this
idx
3
source
right.
So
we
can
instantiate
an
idx
3
source
and
an
id
an
idx
3
source
for
the
the
the
the
the
train
images.
B
B
And
we
need
to
go.
I
think
I
think
this
works
out
of
the
box
like
that,
but
we're
gonna
go
double
check
that
because
so
yeah,
so
we
need
to
pass
probably
file
name,
oh
yeah,
so
it
looks
like
idx
3.
So
this
instant.
This
says
what
class
to
instantiate
and
then
this
says
property
file
name
equals
this
file.
So
that
means
we
need
to
say.
File
name
equals
this
file.
B
Eventually
we
we're
hoping
to
have
a
converter
that
actually
just
spits
out
python
code
given
command
line
arguments
and
vice
versa,
feature
equals
image,
and
this
says
basically
hey
you're
loading.
All
the
data
from
this
file
in
this
format
like
when
I
yield
a
record
for
each
entry.
What
should
I
call
the
data
that
I
loaded
for
the
from
file
and
we
should
call
the
data
image
so
file
name
on
this.
One
is
images
path
and
then
feature
is
label.
B
So,
basically,
you
know
create
this
sources:
objects,
which
is
really.
You
know
two
sources
in
a
trench
coat
and
each
time
you
read
a
record
from
one
read
a
record
from
the
other
one
and
call
the
data
that
you
read
from
the
idx
3
source
image
and
call
the
data
that
you
read
from
the
idx
one
source
label-
and
this
is
this.
This
should
be
it
so
and
then
we
will
we'll
yield
it
yeah,
okay,
cool
so
and
then
from
here
we
basically
just
say
you
know,
base
so
csvx.
B
B
Contributing
and
then
testing
and
then
here's
how
to
run
a
specific
test
and
here's
we'll
just
do
this
shorthand
here.
So
basically
oops,
oh
and
I
figured
out
this
pretty,
which
is
cool
recently,
which
I
really
like.
So
if
you
do
pdb,
if
you
do
python
and
then
pb,
then,
if
there's
an
exception
that
you
didn't
handle
already
like
you're
developing
some
stuff,
it'll
drop
you
right
to
a
python,
debugger
shell
as
soon
as
you're,
like
as
like
yeah
it'll,
it'll
drop
you
right
to
the
debugger
shell,
which
is
pretty
sweet.
B
B
B
B
B
Tests-
oh,
we
don't
want
discover,
so
we're
going
to
run
this
just
this
test
and
what
it's
doing
is
there
is
a
it's
running.
It's
running
these
console
examples.
When
you
put
restricture
text
in
a
markdown
file,
we
have
some
extra
stuff
that
we've
added
on
to
say
when
you
put
test
it's
going
to
run
that,
and
so
what
we're
missing
here
is
from
the
top
level
we
every
plugin
that
we
talked
about,
needs
to
be
registered
and
so
we're
gonna
go
register
this
reinstall
the
package
and
re-run
mist
and
then.
B
B
B
B
This
would
be
so
so,
basically,
there's
some
some
some
about
this
is
the
implementation
of
this
effectively
there
there's
another
layer
of
abstraction
between
the
data
set
source
and
it's
this
context,
manager
wrap
source
and
it
has
on
line
108
see
over
here
line
108.
This
is
the
the
error
that
we're
getting
function
did
not
yield
source.
B
So
this
is
just
basically,
there
was
input,
validation
being
done
to
ensure
that
we're
yielding
a
valid
source
from
the
data
set
source
right,
because
we
have
this
data
set
source
decorator
that
we
can
use
here,
which
is
this
at
data
set
source,
and
this
function
is
calling
another
function
which
is
in
this
wrapper.py
at
line
108,
which
is
saying
whatever
is
yielded,
let's
make
sure
that
it's
a
valid
source
and
then
it
throws
an
exception
if
it's
not
a
subclass
of
source.
Well,
sources
itself
is
not
a
subclass
of
source.
B
B
B
B
B
B
B
B
C
B
B
B
A
B
B
B
B
We
ran
the
command
the
same
command
we
were
running
before,
but
we
exported
this
environment
variable,
and
this
is
how
you
do
a
temporary
environment.
Variable
you
just
prefix,
whatever
command
with
it
and
then
when
it
ran.
You
know
this
sub
command.
Here
it
picked
up
on
the
fact
that
hey
you
know.
I
need
to
be
enabling
the
debug
logging,
so
here's
the
nice
download
progress
code
that
we
have
and
oh
so
here
you
see
it
instantiate
the
source.
So
this
is
the
source.
B
B
That's
why
because
they're
csv
files,
so
we
can
tell
you
right
now.
I
can
tell
you
right
now
what's
wrong.
So
basically,
these
are
idx
3
format,
files
which
are
gzik
gzip
compressed
and
our
file
source
abstraction
will
automatically
decompress
files
for
us,
but
it
works
based
on
the
file
extension.
So
if
you,
if
you
get
the
file
extension
wrong,
then
it's
not
gonna
properly
decompress
it.
So
we
basically.
A
B
B
How
many
gigabytes
that
is,
like
that's,
a
lot
of
gigabytes,
probably
terabytes.
So
if
we
fix
our
file
extensions,
we
will
end
up
with
sanity.
So
let's
see
so,
okay
and
here
it
is
dot
gz.
So
if
we
fix
these-
and
we
named
these
csvs.gz
right-
you
saw
where
I
just
did
that
here
and
here
then
we
should
end
up
with.
You
know
something:
that's
not
completely.
Nuts.
B
Let's
see
what
happens
it
works.
So
this
is
the
correct
output.
The
correct
output
is
a
giant
giant
giant
all
you
know
those
are
what
what
are
they
yeah?
This
is
I'm
trying
to
control
see
this,
but
you
know
it's
got
the
got
the
better
of
me
here,
but
those
are,
I
think,
what
28
by
28
images
I
think
right.
Does
anybody
remember
yeah?
So
this
is
basically
yeah.
So
these
are
the
flattened,
arrays
versions
of
the
28.
A
B
So
it's
dumping
the
whole
thing
to
standard
out,
but
it
works.
So
we
were
able
to
successfully
okay,
I'm
gonna,
I'm
gonna
kill
it,
so
we
were
able
to
successfully
dump
out
that
and
let's
see
so
if
we
look
at
it
here
because
this
is,
you
know
that
just
just
to
save
ourselves
some
trouble
here,
we
can
dump
it
to
a
file.
So
now
we
can
actually
just
so.
This
is
we're
writing.
B
B
B
B
B
And
then
we
give,
I
always
give
my
quick
look
over
here.
Just
make
sure
everything's
correct,
oh,
and
we
got
a
format
with
black.
I
think
everything's
correctly
formatted,
let's
see
yeah
that
looks
like
correct
formatting
to
me
all
right,
we'll
take
a
chance
here
and
see
the
the
ci
will
kick
us
out
if
we
did
something
wrong.
A
C
B
B
B
B
With
this
github
cli
boom
pull
request
all
right.
So
now
we
can
wait
for
the
ci
to
fail,
because
we
have
lots
of
failing
ci
but
yeah
we
need.
I
need
to
I'm
I'm
planning
on
spending
some
time
fixing
this
stuff.
I
I
think
I
told
sahil,
but
I
I
should
have
some
more
scope.
This
is
basically
has
this
project
has
been
something
that
I
do
outside
of
my
day.
B
Job-
and
I
know
sahil-
does
this
outside
of
his
day
job
as
well,
and
so
you
know
we
we
spend
as
much
time
as
we
can
working
on
it,
but
you
know
sometimes
things
fall
by
the
wayside.
So
currently
the
ci
jobs
are
failing,
and
I
know
I
know
you've
been
working
on
that.
So
thank
you
for
working
on
that
and
fixing
the
ones
you
have
all
right.
So
any
final
things
for
this
meeting
today.
B
Great
and
then
you
and
I
can
jump
on
that
call.
Okay,
all
right!
Let
me
go
here
and
I
will
add
this
as
the
mnist
example
for
how
to
add
a
data
set
source,
and
that
should
give
you
a
pretty
good
idea
and
for
pre-work
for
the
proposal.
I
would
recommend
that
you
maybe
go
through
and
you
know
add
something
you
know
similarly
to
what
we
just
did
and
you
could
follow
the.
A
B
A
B
You
can
work
on
whatever
you
want
before
g-suck
right,
so
keep
in
mind,
though
everything's
open
source
right,
other
people
are
gonna,
see
your
stuff,
we're
gonna
know
the
the
beautiful
part
about
open
source
is
nobody
can
say
that
they
did
your
work
when
you
did
it,
because
it's
right
out
there
and
we
can
tell
that
you
did
it
first.
So
if
anybody
tries
anything
like
that,
we'll
just
say
well,
then
we're
not
even
going
to
consider
your
proposal.
B
A
B
Yeah,
so
basically,
so
what
we
found
here,
no,
that
was
a
very
good
thing.
I'm
glad
you
I'm
glad
you
said
that
so
so
what
we
found.
What
did
what
did
we
find
here?
Let's
pull
up
the
now.
We
have
two
sources
that
we
can
easily
look
at
side
by
side
and
data
set
data
set
and
iris
okay.
So
let's
look
at
iris
and
let's
look
at
our
new
one.
B
Right,
okay,
so
let's
go
to
the
bottom
right
so
so
we
we
started
with
this.
We
copy
pasted
it
modified
it
a
bit
and
got
to
here
right.
So
so
now
we
can
look
at
and
see
well,
okay!
Well,
what
you
know?
How
far
are
we
from
our
hard-coded
two
examples
here
to
the
point
where
we
would
have
something
generic
that
works
for
everything
right
and
I
think,
looking
at
this,
we
can
see
that.
Well,
what's
our
general
pattern
right?
It's!
B
Source
that
you
need
right,
and
then
you
know
in
this
one,
we
happened
to
do
some
pre-processing
on
this
one.
We
didn't
need
to
do
any
pre-processing
right
so,
but
the
generic
sense
here
is
well.
You
know
what
what
is.
What
is
the
file
you
want
to
download
and
then
what
is
the
internal
source?
You
want
to
do
right
now
now
the
the
goal
of
this
project,
so
so
this
is
just
you
know,
regular
old
python
code,
which
is
great
right
now.
B
B
B
I
believe
that
allows
you
to
define
any
class
as
a
data
flow,
and
so
once
that
pull
request
gets
merged.
So
basically
you
can
say
this
method.
Does
this
within
the
data
flow?
So
a
data
flow
is
a
set
of
functions
that
get
run
on
like
on
demand
right
and
on.
Demand
in
this
case
is
whenever
a
method
is
called
right.
We
talked
about
update
and
records
and
record.
So
in
this
case
you
would
run
you
know
a
specific
operation
for
a.
A
B
B
B
For
you,
so
basically
what
you
would
do
is
you
would
say
this.
So,
for
example,
this
is
multiply
right.
So
so
this
is
a
data
flow.
These
are
inputs
here
right,
so
you
could
define
you
define
your
data
flow
and
then
you'd
say
when
I
call
the
records
method
which
lists
all
records.
I
want
you
to
send
the
input
well,
records
doesn't
take.
B
We
would
say
on
records,
call
cash
download,
and
then
you
know
the
and
then
you
know,
run
and
and
and
yield
each
record
in
the
csv
or
yield
the
csv
source
like
it's
it.
It's
all
arbitrary
right
like
it's,
it
can
be
whatever
you
want
right
and
so
right
now
it
and
and
effectively
what
it
does
is
it
allows
you
this
mechanism
to
make
everything
configurable
right.
So
we
would
take
the
code
that
we
have
here.
B
B
So
if
we
wanted
to
create
an
arbitrary
source
that
works
for
any
data
set,
we
would
say
we
would
add
cash
downloaded
as
an
operation
and
then
we
would
add
something
that
figures
out
what
the
file
extension
is
and
instantiates
the
correct
source
based
on
the
file
extension
right.
So
we
would
have
something
that
downloads
every
time
it
sees
a
url
it
downloads.
B
B
Okay,
so
I
think
we're
over
on
time.
I
need
to
drop
here
and
double
check
that
I
can
go
into
nine
o'clock
and
I
think
I
can.
I
think
that
meeting
got.
I
think
I
had
no
eight
o'clock
and
I
think
I,
my
nine
o'clock,
got
canceled,
but
let
me
double
check
and
then
let
me
hop
on
the
meeting
with
you
sahel.
Okay,.