►
From YouTube: Keynote: Building Great Web Experiences with AMP and TensorFlow.js - Jeffrey Jose & Sandeep Gupta
Description
Keynote: Building Great Web Experiences with AMP and TensorFlow.js - Jeffrey Jose, Product Manager, AMP, Google & Sandeep Gupta, Product Manager, TensorFlow, Google
Prioritizing user-experience isn't just for end user delight but it can also make great business sense. In this keynote, we will discuss two open source solutions from Google -- AMP and TensorFlow.js -- that helps you build great web experiences to innovate and stand out on the internet. In this talk, you will learn more about the upcoming Page experience update on Google Search, where user-experience as measured by Core Web Vitals are becoming a ranking factor. You will also learn about how machine learning and AI is transforming web experiences and how TensorFlow.js makes it easy for you to develop and use machine learning.
A
Hi,
my
name
is
jeffrey:
I'm
a
product
manager
at
google
working
on
amp
and
I'm
joined
by
sandeep
gupta,
a
product
manager
on
tensorflow.js
at
google.
Our
keynote
is
divided
into
two
sections.
First,
we'll
take
a
look
at
how
the
page
experience
and
amp
works
together
and
then
sandeep
will
walk
us
through
using
tensorflow.js
to
harness
the
power
of
machine
learning
to
build
novel
experiences
for
the
web.
A
A
The
page
experience
signal
breaks
apart
into
the
four
aforementioned
pillars
as
follows:
for
loading
we
have
largest
contentful
paint
and
first
input
delay.
The
largest
contentful
paint
lcp
for
short,
is
a
metric
that
reports
the
render
time
of
the
largest
image
or
text
block
visible
within
the
viewport
relative
to
when
the
page
first
started.
A
Loading
first
input
delay
fid
for
short
measures,
the
time
from
when
the
user
first
interacts
with
the
page,
that
is,
when
they
click
on
a
link,
tap
on
a
button
or
use
a
custom,
javascript
power
control
to
the
time
when
the
browser
is
able
to
process
the
event
handlers
in
response
to
that
interaction
for
the
user,
announce
pillar,
we
have
cumulative
layout
shift
cls
for
short
cls
measures.
The
sum
total
of
all
the
individual
layout
shift
scores
for
every
unexpected
layout
shift
that
occurs
on
a
page.
A
Such
interstitials
are
often
used
to
build
to
trick
users
to
doing
something
that
they
do
not
want
to
preventing
them
from
them.
From
reading
or
interacting
with
the
page
they
landed
on
from
google
search.
There
are
a
lot
of
great
uses
of
interstitials,
such
as
the
ones
required
by
the
law
gdpr,
for
instance,
or
an
interstitial
that
provides
updated
business
hours
during
the
coronavirus
pandemic.
A
A
Signal
covers
the
accessibility
pillar,
which
measures
how
effective
the
pages
are
on
small
screens,
often
used
by
mobile
phones,
the
first
three
metrics
largest
contentful
paint.
First
input,
delay
and
cumulative
layout
shift
are
the
core
web
vitals
a
set
of
metrics
that
apply
to
all
web
pages
and
should
be
measured
by
all
site
owners,
and
that
will
be
surfaced
across
all
the
google
tools.
Each
of
the
core
web
vitals
represents
a
distinct
facet
of
user
experience
and
is
measurable
in
the
field
and
reflects
a
real-world
experience
of
a
critical
user-centric
outcome.
A
A
A
Let's
take
an
example
of
lcp
lcp
is
where
we
measure
the
loading
performance
of
the
most
meaningful
content
of
the
page
values
less
than
two
and
a
half
seconds
means
that
the
page
is
delivering
a
good
page
experience.
Anything
greater
than
four
seconds
means
that
the
page
is
actually
performing
very,
very
poorly.
A
A
A
Amp
puts
developers
on
a
path
to
success,
not
just
by
creating
great
pages,
but
also
helping.
You
maintain
that
great
page
experience
over
a
longer
period
of
time.
This
means
the
amp
runtime
provides
developers
with
the
constraints
needed
to
create
great
performing
pages,
and
the
use
of
amp
cache
further
improves
this,
because
this
allows
you
to
have
the
pages
experienced
by
your
users
near
instantly.
A
However,
like
many
other
frameworks
are
amp
can't
implement
all
the
web
development
best
practices
into
the
runtime.
This
is
why,
in
the
run-up
to
the
page
experience
ranking
launch,
we've
been
encouraging
developers
to
take
a
look
at
your
amp
pages.
To
make
sure
your
pages
are
performing
very,
very
well
on
your
domain.
A
So
the
first
step
in
starting
to
implement
great
page
experience
for
your
site
starts
at
the
amp
core.
Luckily,
the
amp
core,
which
the
amp
project
builds
is
comprised
of
extensions
and
the
runtime
have
robust
set
of
constraints
implemented
to
make
sure
that
the
amp
is
performing
very
very
well.
Over
the
past
several
months,
we've
invested
heavily
to
making
sure
the
core
is
super,
duper
lightweight
and
extremely
fast.
If
you're
using
the
amp
framework
you
are
taking
advantage
of
this
automatically.
A
The
next
building
block
is
how
the
page
is
served
code.
Web
vitals,
as
I
mentioned,
this-
is
determined
for
the
real
world
outcomes.
That
means
when
the
real
users
are
interacting
with
the
page.
That's
the
signal
that
is
actually
powering.
The
page
experience
signal
in
case
of
amp.
This
is
this:
could
pages
could
be
served
either
from
the
publisher's
domain
or
from
amp
cache
and
depending
on
how
the
users
encounter
the
content?
A
A
You
can
learn
more
about
amp
optimizers,
on
the
link
that
you
see
on
the
screen
right
now
and
that's
how
amp
project
thinks
about
the
page
experience.
We
want
to
make
sure
that
amp
allows
you
to
create
great
page
experience
at
the
same
time
also
continue
to
improve
the
page
experience
without
you
having
to
do
a
lot
of
work.
B
B
B
Due
to
publicly
available
large
data
sets
more
powerful
computing
at
our
fingertips
and
research
in
new
methods.
We
are
seeing
improvements
in
all
kinds
of
products
and
services
powered
by
machine
learning,
and
you
see
some
examples
of
that
here
on
this
slide,
such
as
being
used
to
power
platforms
for
education
in
classrooms,
helping
detect
diseases
or
predict
natural
disasters
and
giving
people
new
ways
of
communicating
and
interacting
and
we're
beginning
to
see
how
machine
learning
can
help
improve
web
experiences
as
well,
whether
it
is
to
interactively
experience.
B
B
B
Another
example
is
from
include
health,
which
is
a
musculoskeletal
care
delivery.
Tech
company,
they're
using
body,
pose
estimation
models
to
deliver,
guided
physiotherapy
at
scale
with
many
people
unable
to
leave
their
homes
or
travel
these
days.
This
technology
allows
for
a
remote
diagnosis
and
treatment
from
the
comfort
of
their
own
home,
using
just
a
web
browser
and
a
standard
webcam
that
almost
everyone
has
easy.
Access
to.
B
B
This
pose
estimator
tool
was
created
by
the
partner
innovation
team
at
google
and
it
allows
you
to
draw
any
svg
character.
You
like
and
then
use
your
body
to
control
it
in
real
time,
giving
animators
a
motion
capture
solution
to
drive,
2d
character,
animation
that
anyone
can
use
with
just
a
webcam,
and
yes,
all
of
this
is
running
entirely
in
a
web
browser.
B
B
B
B
B
Lastly,
it
is
gpu
accelerated,
so
you
get
great
performance
with
tensorflow.js.
You
can
write
your
application
code
once
and
use
it
anywhere
and
running
ml
in
the
web
has
some
unique
advantages
over
running
it.
In
native
apps,
for
example,
there
is
zero
install
needed.
You
can
reach
an
audience
of
billions
of
users
instantly
simply
by
sharing
a
url
with
no
complex
environment
setup
for
your
users.
B
B
B
So
let's
say
you
want
to
build
an
app
to
help
find
the
answer
to
a
question
in
any
piece
of
text
that
you
present
to
it
here.
We
have
a
chrome
extension
that
does
this
on
the
text
on
any
web
page
just
type,
your
question
and
the
model
provides
an
answer
and
scrolls
to
the
part
of
the
page
that
most
likely
answers.
The
question.
B
B
B
B
Finally,
we
can
call
our
model
dot,
find
answers,
method
we
pass
to
this
function.
The
question
we
want
to
answer
along
with
our
search
text,
and
when
this
completes
it
will
return
an
answers,
object
from
which
we
can
get
the
most
likely
answer
from
the
given
passage
of
text.
In
this
example,
the
model
predicted
cats.
As
the
answer
to
our
question,
which
we
proposed,
which
is
correct,
given
the
text
we
had
to
search
and
that's
all
there
is
to
it
pretty
simple,
so
give
it
a
try.
B
The
exact
same
workflow
works
for
using
any
of
our
pre-trained
models.
Here
are
some
of
our
most
popular
models.
Coco
ssd
is
an
object
detection
model
which
can
identify
90
different
object
classes
and
show
you
the
bounding
boxes
in
the
images
where
the
detected
objects
are
media
pipe
face.
Mesh
is
a
high
resolution,
face
tracking
model
which
can
recognize
468
key
points
on
the
face.
It's
only
3
megabytes
in
size
and
it
provides
real-time
face
analysis
for
detecting
facial
gestures,
lips
and
eye
movements.
This
has
many
applications
in
retail,
entertainment
and
accessibility.
B
For
human
pose
estimation,
we
have
a
powerful
and
easy
to
use
post
detection
api,
which
supports
three
different
models
for
a
variety
of
performance
and
accuracy
needs.
We
recently
added
two
powerful
new
models
in
this
api
movenet,
which
is
an
ultra
fast
and
accurate
model
that
tracks
17
key
points.
Optimized
for
diverse
poses
and
fast
actions,
and
the
second
media
pipes
blaze
pose
which
gives
33
key
points,
and
this
extra
granularity
may
allow
better
tracking
for
certain
applications.
B
B
Another
way
to
train
a
model
easily
in
the
browser
interactively
is
by
using
teachable
machine.
This
lets
you
train
some
common
ml
models
directly
in
the
browser
and
then
exporting
the
trained
model
for
your
use.
I
highly
recommend
playing
around
with
this
tool,
so
this
was
a
quick
introduction
on
how
you
can
use
machine
learning
to
give
your
web
app
superpowers
for
more
resources
and
getting
started
material
visit.
Our
website
shown
here
and
join
the
community
discussion
group
on
discuss.tensorflow.org.