►
From YouTube: Saving the web to IPFS while you browse with Webrecorder tools - @ikreymer - Browsers and the Web
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Yeah
so
I'll
talk
about
saving
the
web
to
ipfs
with
web
recorder
tools,
and
I'm
also
going
to
try
to
do
a
live
demo,
so
it
should
be
interesting,
yeah,
so
about
weber
quarter.
So
web
recorder
is
an
open
source
project.
Some
of
our
objectives
are
building
quality,
open
source
web
archiving
tools,
also
making
web
archives
more
accessible,
centralized
technologies,
obviously
a
big
one,
striving
for
highest
fidelity,
archiving
and
replay,
which
I'll
demonstrate
later,
I'm
really
empowering
anyone
to
create,
share
and
use
web
archives.
A
However,
they
would
like
to
and
also
create
portable
web
archiving
format,
and
really
our
motto
is
web
archiving
for
all
and
yeah.
So
as
it
turns
out,
ipfs
can
help
with
many
of
these
goals,
at
least
these
three
and
yeah,
and
I
I'm
the
I
guess
I
should
have
introduced
myself-
I'm
I'm
the
lead,
developer
and
creator
of
the
web
recorder
project
and
yeah
just
to
cover.
There
are
many
different
use
cases
supporting
archives
and
libraries
and
digital
preservation.
A
Work
is
a
major
major
use
case
and
we
work
with
a
lot
of
libraries,
including
the
national
library
of
iceland,
who
is
using
some
of
our
open
source
tools,
a
brief
shout
out
to
them
also
supporting
community
and
personal
archiving,
including
allowing
individuals
to
archive
their
own
content,
allowing
communities
to
archive
shared
content
online.
That's
that's
important
to
them.
A
Data
rescue
and
archiving
at
risk
content,
so
anything
that
might
be
threatened
by
government
censorship,
war
or
just
blink
rat
for
a
variety
of
reasons,
also
supporting
verifiable
web
archives
used
as
evidence.
It's
also
becoming
a
more
more
important
use
case
and
last,
not
least,
of
course,
is
the
kind
of
bridging
between
the
current
web
or
whatever.
You
want
to
call
web
2
and
the
next
generation
of
the
web,
whether
it's
dweb
web
3,
p2p
web
yeah,
and
so
each
of
these
use
cases
could
of
course
be
their
own
presentation
but
yeah.
A
A
Also,
this
is
sort
of
the
manual
archiving
workflow,
which
I'll
demo
there's
also
an
automated
workflow,
where
we
place
archive
web
page
with
a
crawler
that
can
run
in
docker
as
well
as
kubernetes
and
there's
the
api
and
a
gui
being
developed,
and
it's
been
deployed
anywhere
from
the
cli
version
can
be
deployed
anywhere
from
a
raspberry
pi,
in
fact,
to
a
larger
cloud
deployment.
So
I
won't.
A
A
So
let's
try
maybe
something
more
interesting
like
the
ips
thing,
hashtag
and
there's
some
tweets
in
here.
Let's
say
I
want
to
archive
this
twitter
feed.
I
could
click
on
each
tweet.
We
also
have
that
that's
going
to
be
a
little
bit
tedious.
So
we
have
this
thing
called
autopilot
which
what
that
will
do
is
it
will
automate
that
for
me,
so
it
will
start
clicking
on
each
tweet
and
scrolling
down
and
also
scrolling
through
the
images
and
as
it's
doing
that,
that's
all
being
archived.
A
A
A
A
Hopefully,
it'll
loads,
so
I
guess
I
already
have
it
doing
a
lot
here.
So
I
just
loaded
first
and
then
I
can
turn
that
archiving
back
on
so
another
thing
to
show,
while
it's
loading
that
is
while
so-
and
this
is
now
the
replay
so
now
I'm
in
the
now
I'm
viewing
the
archive-
and
this
is
exactly
the
the
view
that
you
can
see
that
it's
logged
in
as
myself.
So
this
is
a
view
of
of
twitter.com
that
is
unique
to
me.
A
A
A
And
then
I'll
just
have
it
locally
as
a
file,
and
now
that
I
click
on
the
sharing
link.
So
I
have
I
make
available
three
different
options,
because
that's
sort
of
was
necessary
to
test
in
different
environments,
and
then
I
can
just
because
I'm
in
brave,
if
I
paste
the
obviously
it's
in
the
same
instance,
and
now
it's
loading
the
same
archive
from
from
ipfs.
And
so
I
should
be
able
to
click
on
on
here.
A
Some
of
these
tweets,
so
it's
a
by
high
fidelity.
The
idea
is
that
we're
archiving
not
just
not
just
the
a
static
version
of
a
site
but
the
actual
sort
of
the
interactive
web
application,
including
all
the
javascript,
that's
involved,
and
so
this
is.
This
is
one
of
the
approaches,
and
this
requires
installing
the
a
browser,
extension
and
archiving
exactly
what's
loaded
in
your
browser.
A
A
So
let's
say
I
want
to
archive
dietrich's
tweet.
Actually
we
also
have
a
way
of
archiving
just
the
tweet
version
itself,
and
so
this
is
a
kind
of
a
a
special.
So
embed
is
a
standard
for
embedding
that
allows
for
essentially
loading
an
embed
like
this
and
twitter
happens
to
support
it
and
so
and
then
I
can
also.
A
Download-
and
we
have
this-
this
url,
which
I
can
load
again
in
in
actually
that
was
in
chrome,
so,
let's
switch
to
to
brave
so
that
the
the
gateway
link
loads
natively,
and
now
we
have
it
loading
in
brave.
Let's
also
try.
Agar
gore
has
two
browsers:
aren't
enough:
let's
try
three
and
actually
I'm
gonna
copy,
the
direct
pfs
link
because
and.
A
And
so
that,
and
now
we
have
it
loading
in
aggregator
as
well,
so
we
could
probably
also
there,
of
course,
for
for
other
browsers,
there's
still
the
the
the
direct
gateway
link
which,
let's
see,
if
that
actually
loads.
Oh
no,
I
guess
I
copied
the
about
a
second
yeah,
so
if
I
just
click
on
that,
that
will
probably
take
longer
so
some
of
the
other
things
we
can
do.
There's
a
also
archive
youtube
videos
in
this
way,
oh
actually
for
youtube.
A
A
A
You
can
load
it
in
in
in
brave
and
then
we'll
have
a
a
version
of
this
okay.
So
so
that's
part
of
the
yeah,
oh
probably,
first
of
time,
I'll
move
on
back
to
the
slides
and
so
yeah.
If
you
need
to
archive
a
page
to
ipfs
in
a
pinch,
you
could
try
express
darker
web
page.
It
generally
should
work
and
as
long
as
it's
not
something
that's,
that's
private
should
work
for
public
data.
A
It's
also
processing
things
through
cloudflare,
because
that's
the
only
way
that
we
can
access
things
in
another
website
due
to
course,
restrictions
and
yeah,
and
so
what
works
really
well
brave
access
and
creating
web
web
archives
on
the
desktop
works
really
well,
as
I've
just
shown.
Also,
access
on
in
aggregate
browser,
thanks
to
move,
works
really
well
on
desktop
and
there's
also
a
mobile
version
of
of
agora
that
can
load
ipfs
links
directly
as
well
and,
of
course,
using
a
web.
A
So
in
the
in
the
case
of
that
geocities
page,
that
was,
I
think,
around
100k
or
so
so
that's
one
kind
of
lower
bound
and
then
part
of
an
archiving
effort
to
archive
ukrainian
websites
from
a
large
crawl,
not
using
the
browser
extension,
but
using
the
crawling
system
which
produces
data
in
the
same
format.
We
actually
have
a
an
archive,
that's
about
one
terabyte
and
yeah,
and
so
we
can
actually
browse
that.
A
So
if
I
just
load
that
here,
I
had
a
preloaded,
it's
a
it's
a
also
in
the
same
format.
I
believe
it's
it's
a
page
in
four
languages
that
has
a
whole
bunch
of
youtube
videos,
there's
a
whole
media
section
that
that
is
just
multiple
hour.
Long
youtube,
videos
and
the
size
is
one-
is
just
over
a
terabyte
yeah,
and
so
the
idea
is
that
this
system
should
support
anything
in
that
range
and,
ideally,
ideally
larger
as
well.
A
So
chrome
is
trying
to
load
for
the
gateway.
That's
going
to
be
slow,
so
we'll
just
move
on
and
yeah,
so
a
little
bit
about
the
waxy
format.
So
it's
a
kind
of
a
standalone
format.
It
packages
some
of
the
existing
formats
that
we
use
in
the
web
archiving
world,
including
work
which
lacks
indexes.
So
that's
sort
of
the
main
limitation
of
work.
A
A
It
can
include
direct
signatures
of
all
the
data
here,
so
it
actually
yeah
everything
is,
is
hashed
and
then
signed
at
the
end
in
the
manifest
or
in
the
digest
file,
and
it's
basically
just
a
zip
file
so
which
allows
for
random
access.
So-
and
this
is
the
structure
of
the
zip
file
and
there's
a
link
to
the
specs
yeah,
and
so
the
idea
is
that
this,
this
file
format
can
be
loaded
from
anywhere
from
ipfs
from
local
file
system,
http
and
yeah.
A
And
that-
and
this
is
what,
when
I
express
a
web
page
and
in
a
comment
page
we're
putting
data
on
ipfs.
This
is
what
it's
putting.
Essentially.
Is
this
a
sid
with
this
structure
and
so
yeah?
How
have
we
tried?
Putting
data
on
ipfs
so
by
the
browser
extension
with
jsipfs
in
the
browser.
So
if
you're
using
chrome,
that's
what
it
will
use,
if
it's
using
brave,
then
it
will
use
the
native
google
ipfs
in
brave,
and
I
think
we'll
probably
do
something
similar
with
aggregor.
A
Then
there's
also
an
electron
version
of
archive
web
page,
which
runs
jsipfs
in
node
and
then
and
then
finally,
with
express
archive
web
page
just
uploading
to
web3
storage
through
regular
http
api.
And
if
I
were
to
rank
these
approaches.
This
is
kind
of
how
how
it's
been
the
js
ipfs
in
the
browser.
I
extension
is
not
really
reliable,
for
I
can
just
generate
a
url
and
give
to
users
and
expect
that
to
work,
which
sort
of
makes
sense
the
way
that
the
native
support
and
brave
works
much
better.
A
But
there's
still
some
hacks
involved.
The
electron
app
with
jspfs
mostly
works,
but
there's
some
yeah.
It's
also
not
not
quite
as
reliable
as
it
is
in
brave
and
then
of
course,
uploading
to
web3
storage
has
been
super
reliable
and
easy
to
use.
So
so
that
gets
five
stars
yeah
and
some
of
the
yeah.
Some
just
more
details
about
the
js
ipfs
and
the
browser
also
attempted
custom
preloading,
because
don't
necessarily
want
to
upload
everything
at
once.
It's
running
in
a
service
worker.
A
There
are
some
limitations,
can't
use
webrtc
but
yeah.
So
there's
there
have
been
a
few
challenges
there,
the
embedded
mode
in
brave
since
there's
no
writable
gateway.
Yet
I
basically
have
to
port
scan
for
a
api
port
that
brave
runs
on.
It
runs
on
one
of
five
ports,
depending
on,
if
you're,
using
brave
release,
version
or
or
dev
build
and
also
have
to
override
cores,
because
the
gateway
is
not
designed
to
be
used
in
the
browser.
So
basically
it's
sort
of
a
huge
hack.
A
The
electron
api
had
to
implement
a
custom
pc
system,
because
the
recording
happens
in
the
browser
and
then
send
it
to
the
electron
node
process
and
also
kind
of
also
very
very
custom
implementation
there
and
then
for
website
storage,
just
creating
the
car
files
and
using
the
rest
api,
so
yeah,
so,
instead
of
all
kind
of
very
different
ways
of
trying
to
do
the
same
thing
and
yeah.
So
now
the
challenge
is
for
reading
from
pfs.
A
A
A
You
might
want
to
just
look
at
a
few
pages
and
then,
if
you
actually
want
to
make
a
copy,
then
that's
when
you
pull
the
whole
thing.
So
the
the
random
access
read
from
a
very
large
data
set
is
is
very
important
here.
Yeah
and
ip
fester
really
be
as
infrastructure
users
of
archives
shouldn't
have
to
know
or
think
about
using
ipfs.
A
I
think-
and
I
think
ideally
just
present
these-
the
the
the
options
on
the
left
in
the
left
column
is
what
I
want
to
present
to
users
and
not
tell
them
about
what's
happening
on
the
right
unless
they're
developers
and
yeah
and
yeah.
So
I
think
that
these
are
some
of
the
key
goals
to
to
be
working
towards
just
briefly
about
future
work,
how
to
build
large
archive
collections
trying
to
standardize
this
data
layout.
A
A
A
The
data
is
actually
stored
in
the
zip
as
uncompressed
data
to
make
it
portable
so
that
users
can
download
a
single
file,
but
if
they're
putting
in
if,
if
putting
that
data
on
ipfs-
and
you
just
aren't
downloading
a
file-
maybe
we
just
unzip
it
first
and
maybe
we
create
a
car
of
this
directory
structure
and
maybe
that
will
be
a
standard
and
more
work
ahead.
So
standardization
authenticatable
web
archives.
So
that's
that's
a
really
big
one.
A
So
since
anyone
can
create
an
archive
and
put
on
pfs,
that's
a
I
demoed
or
downloaded.
How
do
you
trust
this
web
archive
either
as
the
waxy
file
or
as
a
cid
or
or
in
in
any
format?
And
we
have
several
approaches
for
in
the
cloud
but
still
trying
to
figure
out
how
to
do
that
with
archives
created
in
the
browser
interrupt
between
the
cloud
and
browser-based
archiving
tools
so
making
it
easier
to
kind
of?
A
And
for
that
you
could
see
old
web
today,
which
is
a
another
project.
I
worked
on
that
runs
old
browsers
in
in
emulators,
so
that's
way
further
out.
But
since
there's
a
lot
of
talk
about
web
webassembly,
I
thought
I'd
just
put
that
in
there
as
well
yeah
and
thank
you
and
find
out
more
about
webreporter
site
and,
yes,
we're
hiring
if
anyone
wants
to
help.
So
thank
you.
C
D
Regarding
your
cloud
thing,
do
you
have
any
sort
of
like
deny
lists
or
any
sort
of
moderation
features
because
it
seems
like
this
could
be
a
way
for
people
to
just
like
circumvent
blocks
and
other
things
or
start
using
your
infrastructure
to
crawl
things
which
they
maybe
shouldn't
so
like?
Is
there
any
moderation
in
place
or
plans
in
the
future,
or
is
there
even
like
a
worry
of
dmc
takedowns.
A
Possibly
so
it's
still
in
very
early
stages
of
development,
so
these
tools
are
a
little
bit
further
along
right,
now,
kind
of
focusing
on
supporting
archives
and
and
what
institutions
that
are
kind
of
would
run
crawls
in
a
controlled
way
yeah.
If,
if
it's
ever
offered
as
a
public
service,
then
we
definitely
need
to
think
about
that.
Of
course,
it's
all
open
stores,
so
someone
else
could
could
run
it
on
their
own
and
then
they
would
have
to
worry
about
that.
A
B
A
A
So
the
the
the
replay
yeah
so
the
the
replay
here
is
and
again
this
is
all
something
that
that
will
probably
change
yeah.
So
the
right
now
the
replay
is
bundled
with
with
the
archives
themselves,
there's
an
argument
for
possibly
also
separating
back,
because
you
could
actually
point
this.
Since
it's
just
a
service
worker,
that's
loaded,
you
could
load
the
replay
and
then
pass
the
parameter
that
loads
loads,
the
actual
archive
from
a
different
sid,
so
yeah,
so
that
that
that
there's
a
few
things
to
figure
out
there.
A
E
A
Yeah,
so
so
there's
a
difference.
So
what
I
showed
now
with
the
browser
extension,
that's
basically
just
that
that's
all
manual,
so
nothing
is
being
automated
there.
It
was.
It
was
crawling,
so
it
wasn't
crawling.
It
was
archiving
what
I
was
loading
in
the
browser
there's
a
separate
tool
that
does
crawling,
which
is
also
open
source.
Yes
yeah,
I
didn't
quite
have
time
to
demo
it,
but
I
I
can.
I
can
talk
more
about
it,
but
it
produces.
A
Currently,
no
because
the
the
format
includes
timestamps,
and
so
the
way
it's
stored
is
is,
but
if
there
is,
I
mean
ideally,
this
would
be
yeah,
so
currently
the
the
format
itself
would,
because
it
includes
timestamps
another
metadata.
It
would
not.
It
would
be
it'd,
be
a
different
waxy
file
every
time
the
interesting
part
of
it
would
be
if,
if
especially
for
large
files,
if
perhaps
there's
some
sort
of
custom
chunking.