►
Description
Presented by Jaume Sánchez Elias, UX/UI Engineer, DeepMind
This session will offer a look back at the history of the web and VR and a look into the future of how we interact with the web. Jaume will cover technology, browsers, APIs, and programming for an XR-enabled web.
For more from GitHub Universe 2020, visit https://githubuniverse.com
As always, feel free to leave us a comment below and don't forget to subscribe: http://bit.ly/subgithub
Thanks!
Connect with us.
Facebook: http://fb.com/github
Twitter: http://twitter.com/github
LinkedIn: http://linkedin.com/company/github
About GitHub
GitHub is the best place to share code with friends, co-workers, classmates, and complete strangers. Millions of people use GitHub to build amazing things together. For more info, go to http://github.com
A
Hi
welcome
to
this
talk
about
webex,
r
and
y
means
for
the
future
of
the
web.
My
name
is
jama
sanchez.
I
work
as
a
ux
ui
engineer,
doing
visualization
at
deepmind.
I've
been
coding
on
the
web
for
many
years,
using
webgl,
webvr
and
now
webxr,
both
professionally
and
as
a
hobby.
Doing
creative
coding
I'd
like
to
thank
github
universe
for
the
opportunity
to
talk
about
webxr,
its
state
and
its
exciting
future.
A
So
what
is
webxr
simply
put
is
a
javascript
api
to
use
xr
devices
cool,
but
what
does
that
mean?
It's
a
way
for
web
pages
to
access
an
xr,
device's
inputs
and
outputs.
Xr
is
the
term
that
covers
vr
virtual
reality
ar
reality.
Mr
mixed
reality
with
the
x
being
the
thing,
the
unknown,
the
variable
in
the
equation.
It's
also
usually
referred
as
extended
hardware
that
enables
xr
applications
is
now
probably
available
to
consumers,
offering
an
immersive
computing
platform
with
new
opportunities
and
challenges.
A
So
as
many
talks
about
the
future,
there
has
to
be
a
recap
of
history
to
understand
how
we
got
here
and
how
it
shapes
the
way
it
progresses,
but
I'll
be
very
quick,
because
everything
that
led
us
here
can
be
forgotten
and
started
new.
Basically,
it
started
with
adding
browser
support
to
the
oculus
dk1,
which
was
a
rotational
controllerless
hmd,
which
means
it
wouldn't
track
your
position
or
half
ways
to
interact.
A
As
new
headsets
came
the
cardboard
gear,
vr
vive
go,
it
became
obvious
that
yvr
wasn't
well
conditioned
to
evolve
with
quickly
changing
hardware
with
new
patterns
and
developers,
expectations,
so
webvr
1.1,
shipped
in
some
browsers,
got
deprecated
and
removed
in
some
engines.
It
was
time
to
do
something
better.
With
the
lessons
learned,
this
new
thing
is
webex.
It's
a
new
api
that
fully
replaces
a
webvr
came
to
me
after
listening
to
developer,
think
of
the
confusing
or
efficient
parts
the
pain
points,
while
developing
iterating
on
emerging
best
practices
and
patterns.
A
So
it's
now
more
flexible
and
future
proof.
It
maps
better.
2X
our
hardware,
the
new
patterns
to
users
and
developers,
expectations
like
low
latency
and
high
refresh
rate
it
provides
improved
support
for
input
and
output,
now
know
more
about
the
devices
which
greatly
simplifies
rendering
to
them
enables
a
unified
input
model
with
standardized
layouts
for
buttons
triggers
touchpads
and,
for
instance,
we
can
now
get
3d
models
of
the
controllers
from
a
content
delivery
network
no
need
to
host
them
or
keep
up
with
them.
A
He
also
provides
performance
improvements,
mostly
supported
by
the
back-ends
like
faster
and
better
rendering
or
video
decoding,
and
enables
a
shared
code
path
for
multiple
outputs.
The
code
for
inline
magic
window
ar
vr
app
doesn't
require
much
branching
now,
which
greatly
simplifies
development
websites
are
under
the
immersive
working
group,
which
is
a
w3c
group.
This
is
important
because
it
means
there
is
a
roadmap,
so
the
implementation
won't
be
changing
under
your
feet.
A
A
So
there's
a
mutually
beneficial
feedback
loop
between
both
apis,
but
it's
not
like
opengl
and
the
api
is
different,
but
most
back-end
implementations
follow
the
open,
xr
spell
being
a
working
group
with
a
spec
means
that
new
features
can
be
supported
fairly
quickly
because
there's
no
need
to
figure
out
anything
or
reverse
engineer:
it
improves
on
the
early
adopter
nature
of
webvr
and
it's
backed
by
industry.
We
have
google
mozilla,
oculus,
microsoft,
unity
and
many
others
behind
all
the
activity
of
this
group
happens
in
github,
so
you
can
go
and
check
it
and
contribute.
A
So
what
is
webxr
for
users?
It
means
that
you
can
enjoy
vr
and
ar
content
the
same
way
you
consume
any
other
content.
Can
you
imagine
a
web
without
embedded
video
players?
There
was
a
time
when
that
was
also
experimental,
so
practically
speaking,
it
means
that
a
web
page
can
offer
the
option
to
go
into
vr
or
ar
track
controllers
if
they
are
required,
with
the
security
of
the
browser.
Sandbox,
no
download
and
installation
of
apps,
with
even
permissions
just
hop
on
virtual
environments
without
leaving
the
browser
pages
can
go
from
flat
to
surrounding
worlds.
A
The
libraries
and
frameworks
you're
familiar
with
the
same
simple
and
affordable
cross-platform
development
environment.
You
will
need
webgl,
javascript
and
some
dom
to
begin
with,
but
all
the
web
tech
can
be
used.
Webassembly
webrtc,
for
instance,
in
the
image
makepad
uses
rust
to
create
a
collaborative
shader
editor,
which
you
can
also
check
on
github.
A
So
what
does
webxr
provide
so
through
the
api?
The
page
can
connect
to
an
xr
device
which
could
be
something
like
a
dream:
headset
or
nokus
go
if
the
users
are
austis
and
then
they
just
capabilities
with
an
xr
session
started
on
that
device.
It
can
keep
querying
the
pose
and
send
images
to
the
device
output.
A
A
The
gamepad's
module
allows
interacting
with
the
buttons
triggers
thumbsticks
and
touchpads
of
an
xxr
input
device.
The
ar
module
works
the
same
as
the
immersive
vr
content
by
guarantees
that
the
user's
environment
will
be
visible
and
aligned
with
the
rendered
content.
This
may
be
achieved
with
see-through
displays,
like
the
horror
lens
or
the
magiclip,
or
video
pass-through
systems
like
air,
core
or
art,
which
is
basically
your
phone.
A
The
heat
test
module,
provides
information
about
the
real
world
and
a
method
for
developers
to
heat
test
against.
It
becomes
very
important
for
really
immersive
ar
applications,
so
things
can
be
grounded
on
the
world.
The
dom
overlays
api
provides
a
way
to
show
dom
content
such
as
text
and
input
elements
for
an
immersive
arc
session.
This
content
is
displayed
as
a
transparent
background
layer
super
imposed
on
the
real
world
scene
and
the
rendered
webgl
content.
It's
very
useful
to
quickly
build
interactive
air
apps.
A
A
A
So
how
does
one
get
to
build
the
immersive
web?
You
can,
of
course,
just
use
webgl
and
javascript
and
grind
through,
and
but
fortunately
there
are
several
javascript
frameworks
that
support
many
web
xr
features
and
have
a
very
active
community
of
developers
and
adopters
in
alphabetical
order.
There's
a-frame
a
web
framework
for
building
3d
ar
and
vr
experiences
declaratively
with
html
and
an
entity
component
system,
babylon
js
with
loads
of
tools
and
a
great
ide
play
canvas
is
another
great
webgl
engine
with
ide
and
webextra
support
and
3gs.
A
Very
widely
used
has
plenty
of
documentation
and
examples.
It
also
has
a
very
active
community
to
bridge
with
other
frameworks
like
3d
react
or
spell
3..
The
four
of
them
are
on
github.
You
can
check
their
progress
and
contribute
any
time
if
not
a
web
dev.
There
are
still
options
to
build
content
for
the
web
and
use
the
power
of
webxr.
Like
some
options
are
facebook
photos
and
videos
with
react,
360
which
allows
to
create
exciting
360
and
their
experiences
using
react.
A
A
Sumerian
is
a
new
platform
for
developers
to
build
and
host
vr
ar
and
3d
apps
quickly
and
with
minimal
coding
in
unity,
one
of
the
most
well-known
cross-platform
game
engines,
there's
an
explorer
by
mozilla
reality
and
other
exporters
are
being
developed
and
finally
verge.
3D
is
a
toolkit
for
blender,
3ds,
max
or
maya
to
create
immersive
web-based
experiences.
A
But
if
you
go
the
way
of
building
yourself,
you
have
to
choose
a
framework
and,
depending
on
your
framework
of
choice,
you
get
support
for
different
key
features
for
xr.
This
is
a
general
thing
with
vr
it's
not
related
directly
to
xr,
but
when
looking
at
a
framework
or
engine,
it's
important
to
look
for
things
like
locomotion.
A
A
A
So
you
need
something
that
can
load
the
appropriate
model
and
do
some
bone
animation
to
distort
it
also
related
input
models,
as
mentioned
there's
a
repository
of
controller
models
that
can
be
used
via
a
content
delivery
network.
So
you
can
draw
them
accurately.
Instead
of
drawing
a
generic
controller
or
a
box
improving
the
immersion
and
finally
things
like
text,
rendering
would
be
ideal
to
have
something
like
css
and
dom
the
layers
module.
A
Basically
for
vr,
you
have
android
with
cardboard
daydream
through
chrome.
You've
got
some
samsung
phones
with
the
android
samsung
internet
browser
and
android
firefox
with
the
polyfill
polyfill
is
a
javascript
implementation
of
the
webxr
device
api,
as
well
as
the
webxr
gamepad
module.
It
allows
developers
to
write
against
the
latest
specification,
providing
support
when
random,
browsers
that
implement
the
webvr,
1.1
spec
or
a
mobile
devices
with
no
webvr
or
webxr
support.
That's
all
for
ios.
You
can
use
the
mozilla
webxr
viewer,
which
is
basically
a
webxr
content.
A
A
So
what's
this
being
used
for
what
is
being
built,
many
things
are
being
built
right
now,
there's
just
a
few
examples:
lots
of
games
and
entertainment.
Some
examples
on
the
bottom
left.
There's
moon
rider
an
open
source
game
similar
to
beat
saver
created
by
super
medium.
The
maintainers
of
a-frame,
a
troll
tower
by
daniel
esteban
construct.
Arcade
has
a
whole
set
of
cabinets
to
play
user
created,
xr
games,
there's
also
space
for
artistic
experiences
that
combine
images:
procedural
generation,
video
streaming
web
audio.
There's
a
very
active,
creative
coding
scene
on
the
top
left.
A
We
have
an
example
by
within
who
have
been
building
immersive
experiences
for
years.
What
you
don't
know
is
an
experience
on
the
creative
process
of
mafia
gear.
Also,
advertising
and
campaigns
for
brands
on
the
bottom
right.
Lucian
has
an
ar
ad
for
a
fictional
sneakers
campaign
and,
of
course,
education,
virtual
visits
to
museums
where
you
can
interact
with
everything,
interactive
collaborative
lessons
with
explorable
diagrams
and
explanations
on
the
top
right
frame
that
lets
you
easily
meet
and
create
with
others
in
private
3d
spaces
is
now
focusing
on
virtual
classrooms,
more
options
for
commerce.
A
You
can
see
objects
close
up
before
buying
them
without
visiting
the
store.
You
can
see
customized
versions
of
any
thing
you
want
to
purchase
place
it
in
the
wall,
place.
Furniture
in
your
house
and
see
if
the
colors
match
play.
Canvas
has
examples
for
ar
furniture.
Shopify
supports
the
model.
Viewer
sketchfab
is
widely
used.
A
More
cases
indicates
real
estate
companies
and
house
hunters
can
use
and
benefit
from
virtual
tours
to
houses
for
productivity,
all
the
collaboration,
measurements,
finding
directions,
great
presentations,
all
kinds
of
useful
tools.
There's
nifty
reality
a
virtual
reality
space
on
the
web
that
lets
you
run
multiple
3d
applications
at
the
same
time
or
solder
by
google
that
uses
webxr
to
help
visualize
distance
social,
distancing
guidelines
in
your
environment
and,
of
course,
research
with
things
like
protein
and
molecule
viewers,
multi-dimensional
data
exploration,
visualizations,
ai
agents,
training
for
examples
at
visualization.
A
But
it's
also
about
building
the
metaverse.
The
web
enables
connectivity
and
sharing,
and
so
you
can
think
as
a
potential
explosion
of
content
like
geo
cities
and
flash.
We
have
examples
like
tracosis,
360,
volumetric,
video
by
xr,
social
and
x3
engine
lots
of
things
happening
in
hubs.
Fabian
beneteau
shows
live
coding
in
a
virtual
room
with
multiple
viewers
as
his
streams,
while
coding
a
light
switch
in
a
vr
environment
affecting
a
real
world
internet
of
things,
light
bulb
all
happening
vr
and
over
the
internet.
A
We've
also
seen
multiple
actors
performing
live
from
different
cities
in
future
stages
of
mozilla
hubs
or
we've
seen
valencia
james
dancing
over
live
stream,
volumetric
video
into
hubs,
so
things
are
coming
soon.
New
features
that
are
here
but
behind
the
flag,
which
means
they
will
be
soon
available
for
everyone.
A
An
input
model
module
exposes
the
poses
of
hand,
skeleton
joints
it
can
be
used
to
render
a
hand
model
in
vr
scenarios
as
well
as
perform.
Gesture
detection
with
the
hands
does
not
provide
access
to
a
full
hand
mesh.
That's
where
the
framework
is
becoming
boring.
As
we
said
earlier
in
the
top
image,
you
can
see
how
3.js
supports
hand,
tracking
and
rendering
out
of
the
box
and
on
the
bottom
image
sign
language
detection
by
handy
js
by
steve
smith.
A
Also,
there
is
sbi
that
brings
composition
layers
which
are
useful
for
displaying
information
text,
video
or
textures
that
are
intended
to
be
focal
objects
in
your
scene,
improv's
performance
and
battery
life.
With
enhanced
visual
fidelity,
you
can
render
images
stereo
cue
maps
as
backdrops
and
more
on
the
right.
You
can
see
a
preview
of
how
super
medium
a
comic
book
in
magazine
either
4xr
uses
layers
to
greatly
improve
fidelity
and
performance.
A
In
progress
also
are
some
really
nice
features.
Depth.
Sensing
exposes
information
about
the
distance
from
the
user's
device
to
the
real
world
geometry
in
the
user's
environment.
You
can
see
it
in
the
first
image
by
maxim's
natives
for
play.
Canvas
the
digital
rabbits
are
partly
occluded
by
the
physical
table.
This
is
huge
for
immersive
ar,
since
it
gives
better
heat
testing
and
object.
Occlusion.
A
Also
image
tracking,
recognizes
user,
specified
images
in
the
environment
and
provides
tracking
information
in
the
center
image.
Also
by
maxims,
you
can
see
a
cube
and
a
digital
book
cover
that
are
attached
and
tracked
with
a
real
book
cover
a
lighting
estimation
that
provides
information
for
the
renderer
to
improve
shading
shadows
reflections,
so
content
appears
more
natural
and
integrated.
A
It's
very
important
because
it
will
make
much
easier
and
seamless
the
integration
of
content
with
xr
and
many
more
new
features
that
are
coming
soon,
not
directly
related
to
web
server,
but
like
in
improving
the
ecosystem
like
web
gpu,
which
is
designed
from
the
ground
up
to
efficiently
map
to
modern
native
gpu
apis
web
gpu
is
not
related
to
what
gl
and
it's
meant
to
be
its
successor
in
the
future.
It
will
be
used
to
provide
the
rendered
content
to
actual
devices
progressive
web
apps
pws
for
short,
are
web
applications
that
have
been
designed.
A
So
they
are
capable,
reliable
and
installable.
These
three
pillars
transform
them
into
an
experience
that
feels
like
a
platform.
Specific
application.
Bring
this
to
vr
and
you'll
be
able
to
start
web
apps
from
within
the
shell
like
if
they
were
native
and
more
adoption
in
the
future.
There's
a
bunch
of
new
hmds
for
both
vr
and
ar
coming
up
new
users,
new
accessories
new
features
in
the
image.
You
can
see
a
promotional
image
of
facebook's
infinite
office,
a
set
of
tools
to
improve
the
virtual
office
experience.
A
A
Maybe
there
can
be
a
web
of
such
simple
and
tangible
things
as
3d
objects,
which
can
be
seen
in
vr
or
placed
in
ar
with
some
hot
areas
or
script
buttons
which
link
to
other
similar
objects.
Sketchfab
and
facebook
photos
have
already
showed
us
that
it's
a
viable
format.
We
may
see
a
return
of
web
rings
where
people
place
their
content
online
and
link
it
in
a
gibson-esque
way.
An
object
can
be
a
small
cube,
but
it
can
also
be
a
whole
room.
All
share
that
interconnected
via
the
web.
A
A
Can
we
look
forward
to
a
full,
valid
markup,
supported
by
the
browser,
to
build
extra
experiences
and
at
the
same
time
we
need
a
way
to
consume
content
that
is
not
ready
for
vr
or
that
doesn't
need
the
third
dimension
in
general,
but
it
could
benefit
from
it.
Wikipedia
is
a
to
the
scrolling
experience
by
illustrations
and
graphs
can
be
3d
and
interactive
like
read
along
an
article
as
objects
and
projections
to
spy.
A
We
could
find
a
way
to
build
even
smaller
components
than
pwas
or
turn
pws
into
widgets
that
we
can
place
around
in
our
virtual
space
nodes
calendars,
videos,
social
feeds
we
could
consume
a
vast
array
of
data
with
small
adapters
for
all
our
web
content
could
be.
The
return
of
microservices
could
be
the
return
of
active
channels
or
yahoo
pipes.
A
Aardvark,
for
instance,
is
an
augmented
reality
layer
that
incorporates
multiple
applications
or
gadgets
on
top
of
existing
vr
worlds
and,
finally,
stepping
out
of
the
page
back
into
the
browser.
Can
a
browser
be
an
xr
experience
itself?
Can
it
transcend
the
2d
projected
window
and
be
a
shell
on
its
own?
How
would
that
work
right
now?
We
know
in
vr
our
whole
view.
Our
whole
world
has
been
replaced
rendered
by
the
experience
and
we
have
to
go
back
to
the
shell
to
be
sure,
we're
safe.
A
We've
developed
all
these
ui
notions
like
the
patch
log
for
secure
connection
that
live
in
the
browser
chrome
in
the
area
outside
of
the
page,
whether
they
leave
when
in
vr
when
the
page,
the
chrome,
the
browser
are
not
there
is
there
physical
manifestation
in
the
vr
world.
Is
the
companion
hovering
near
you
that
you
can
talk
to
like
an
assistant?
Will
xr
be
interacted
with
mostly
via
voice?
A
A
A
Can
we
afford
to
make
experiences
that
discourage
visually
impaired
users
or
make
a
control
or
locomotion
schema
that
can't
be
used
by
physically
impaired
users
or
something
as
simple
as
making
experiences
that
don't
account
for
left-handed
users
or
children,
as
we
mentioned
before,
with
the
risk
of
traversing
their
content?
How
do
we
tackle
security?