►
From YouTube: sett: data encryption and transfer made easy(ier) by Jaroslaw Surkont, Christian Ribeaud - 🦀 Zürisee
Description
End-to-end encryption provides an increased security layer when sharing data over the network by ensuring that only the intended recipient(s) of a message can read it.
Protecting data privacy is especially important when personal human data is involved, such as genomic sequences. To facilitate secure data transfers between hospitals and academia, we developed `sett`: a tool for data compression, encryption, and transfer. Its primary goals are to ensure a smooth user experience, speed, and data transfer auditability. `sett` is a cross-platform tool (Linux, Windows, MacOS) with both CLI and GUI interfaces.
The initial version of `sett` was implemented in Python. However, due to its many limitations, we decided to port the application to Rust. In this talk, we will present how we gradually transit from Python to Rust (without breaking production) with the ultimate goal of shipping a pure Rust implementation to the end users.
Interactive Slides: https://biomedit.gitlab.io/presentations/sett-zurich-2023/
Slides: https://github.com/rust-zurichsee/meetups/tree/master/2023-04-19_sett
Watch more of our meetup: https://www.youtube.com/playlist?list=PL85XCvVPmGQipj690WrVgsnU4K4x7qFGy
Join us next time: https://www.meetup.com/de-DE/rust-zurich/
Would you like to chat or give a talk? Join us in our Matrix room:
https://matrix.to/#/#rust-zuerisee:matrix.coredump.ch
Chapters:
00:00 Intro
01:51 Big Picture
08:00 Python implementation 🐍
15:00 Rust rewrite 🦀
19:15 SequoiaPGP
21:46 Python bindings with PyO3 & Maturin
23:30 High level code
28:25 Code Maintenance, CI, & Releases
33:29 Stay up-to-date with renovate bot
36:08 Performance improvements
42:57 Questions
https://youtu.be/XKODaZgKcnE
A
A
A
Then
we
are
talking
about
maintenance
or
how
he
is
maintenance
implemented
in
our
project.
Then,
let's
talk
about
YouTube
plans
of
of
the
tools,
and
maybe
we
can
have
interest
some
interesting
discussion
because
we
still
have
open
question
there
and
at
the
final
step
we
will
just
summarize
what
we
have
seen.
A
A
A
Okay,
what
are
set
characteristic
so
set
is
a
end-to-end
encryption
tool
and
data
setting
tool.
This
means,
if
you
want
to
send
data
to
someone,
you
will
use
your
private
key
to
sign
the
data
and
you
will
use
the
public
key
of
the
recipient
to
encrypt
the
data.
So
by
doing
so,
we
are
really
making
sure
that
only
the
right
people
do
access
the
data.
A
A
A
We
are
offering
a
clear
and
a
GUI
graphic
user
interface
and
basically,
you
can
consider
set
as
a
facility
tool
so,
but
you
could
of
course,
of
course,
only
use
Clear
commands
like
gpg
to
do
the
job
for
I'm
talking
about
the
encryption
and
for
the
transfer
you
could
use
SFTP,
which
is
which
could
be
found
on
pretty
any
platform.
So,
but
our
job
here
is
in
the
set
is
to
facilitate
all
all
the
steps
to
hide
on
the
complexity
of
all
the
steps.
A
I
will
show
you
an
image
afterwards.
So
now,
if
I
I
said
we
are
sending
one
one
file.
If
now
we
have
a
look
at
this
one
file.
The
file
looks
as
following.
So
this
is
a
ZIP
file.
We
decided
to
go
for
zip,
because
tip
file
has
a
is
handling.
File
is
handling
compression
five
by
five
and
is
is
easier
to
implement
things
like
streaming.
A
We
have
the
file
is
composed
of
a
metadata
Json.
Then
we
have
the
metadocation
signature,
so
this
mean
it's
a
detached
signature,
so
this
file
is
signed,
but
it's
still
readable
by
humans.
This
is
needed.
We
need
some
the
file
in
the
readable
form
because
we
are
making
some
checks
at
different
points
during
the
workflow.
A
As
you
see
in
the
hand,
we
have
the
checksum
Sha
having
a
deeper
look
at
that
one
I
think
it's
coming
here
exactly
so.
If
now
I
open
the
metadata
Json
the
human
readable
file,
you
will
see
it
contains
some
metadata.
Like
the
sender,
fingerprints,
the
recipient
fingerprints
check
some
of
the
data
file
and
things
like
transfer,
ID
and
version
no
transfer,
ID
and
purpose.
They
are
specific
to
biomedility
infrastructure,
but
you
could
use
set.
We
wanted
to
support
this
use
case.
A
You
could
use
set
as
private
person
so
outside
of
the
biomed,
IIT
context
and
I
already
did
it
to
to
to
send
encrypted
and
signed
data
to
someone
having
a
look
at
the
checksum
shop
256
file,
it's
just
a
file
which
lists
the
content
of
the
data
file
with
checksum.
So
you
see
here
we
are,
we
are
making
sure
that
the
file
which
the
right
person
and
we
are
making
sure
that
the
file
have
not
been
corrupted
or
are
in
are
correct.
A
A
Okay,
so
the
first
tool
has
been
developed
in
Python
I
mean
I
was
not
part
of
this
decision,
but
probably
as
a
developer,
we
decided
to
go
for
part
for
python,
had
had
a
best
command
in
Python
and
I
can
say
I'm,
it's
not
a
bad
decision,
because
python
is
a
quite
easy
language
to
learn
and
you
have
as
well
GUI
support
graphic
user
interface
support,
so
it
was
pretty
easy
to
develop
set.
So
here
you
see
a
screenshot
of
of
the
GUI
component
of
set
it's
composed
of
five
tabs.
A
For
so
first
you
will
as
a
sender
you
will
encrypt
the
data,
then
you
will
transfer
them
and
on
the
other
side
you
will
like
to
decrypt
the
data.
So
you
use
the
tab
in
the
middle
decrypt
and
we
have
another
tab
for
Key
Management.
So
for
downloads,
the
key
or
uploading,
the
keys
to
the
key
server
and,
of
course,
settings
for
the
app.
A
Set
support
all
the
main
operating
system,
including
Center
7,
and
this
is
really
a
cumbersome
Centos
Center
7
is
a
old
operating
system.
Will
reach
end
of
life
by
middle
of
next
year,
but
I
mean
this
is
not
our
decision.
We
have
to
support
the
the
the
operating
system,
whereas
a
project
are
deployed,
I
mean
or
hopefully
I
mean
I
have
been
told
that
we
they
plan
to
move
away
from
Center
server
by
end
of
the
year,
because
we
have
to
support
Centos
7.
We
have
to
use.
A
A
A
A
A
So
we
find
out
that
our
main
problem
is
a
glue
PG.
These
are
most
problematic
dependency.
We
should
really
try
to
get
rid
of
it.
Oneid
I
I
came
with,
but
no
one
was
really
happy
about.
It
was
rewrite
everything
from
scratch:
JavaScript
and
having
actually
set
running
in
the
browser,
because
nowadays
everything
is
a
browser
and
the
browser
is
a
new
OS,
so
no
more
normal
requirement
or
product.
Is
it
like
a
python
or
open
pgp?
A
However,
there
is
even
open,
pgp
JavaScript
implementation,
but
I
mean
I
mean
we
didn't
try,
but
we
think
we
have
to
think
about
it.
I
mean
having
a
browser
you,
the
app
running
in
the
browser
you
could
maybe
face.
Other
challenges
like
I
mean
typically
the
data
which
travel
over
the
wire
are
pretty
big
huge,
so
one
gigabyte,
one
terabyte
and
I'm,
not
sure
the
browser
will
be
able
to
upload
this
kind
of
file
in
a
very,
very
lab,
reliable
way.
A
Then,
finally,
we
decided
to
go
with
rust,
so
we
decided
to
proceed
in
a
surgical
replacement
of
specific
workflow.
So
you
know
I
explained
to
you
that
we
have
encryption
transfer
in
the
description
and
thanks
to
the
python
bindings
you
have
in
Rust,
you
get
actually
replace
only
part
of
the
code
and
not
everything
in
once.
A
This
gives
you
this
give
you
she
gives
you
some
time
to
learn
rust
and
not
surprising
in
a
better
way,
so
use
the
end
user.
Why
we
came
to
this
decision.
I
would
say
it's
a
it's
a
couple
of
hazards,
a
couple
of
a
couple
of
factors,
but
a
bit
a
bit
luck,
I,
would
say.
First
of
all,
we
used
to
have
Gary
in
our
team,
and
Gary
is
a
rust
fanatic
and,
and
so
one
day
we
spent
one
one
full
day
with
him,
and
he
tried
to
to
show
us.
A
Our
great
rest
is
and
I
will
have
to
say
we
get
pretty
motivated
and
we
wanted
more.
So
this
was
the
first
Factor.
The
second
Factor
was
we
want.
We
wanted
to
get
rid
of
gpg
and
we
had
been
observing
Sequoia
pgp
for
a
long
time,
but
we're
not
releasing
the
version
1.0.0
and
we
were
working.
We
were
waiting
for
this
version
and
by
2020
they
finally
released
version
1.0.0.
So
for
us
it
was
really
okay.
Let's
do
it
now.
B
Okay,
so,
as
Christian
explained,
we
decided
to
go
the
rust
way
and
yeah,
probably
you
I
mean
for
sure
already
noticed
that
a
lot
of
software
is
being
Rewritten
now
in
Rust
for
all
different
purposes
and
yeah.
It's
not
only
a
common
pattern
in
software
development.
It
turns
out
that
biology
does
the
same.
That
crap
is
a
perfect
life
form,
so
everything
should
should
destined
to
be
become
a
crop
so
but
other
than
crops
being
cool.
B
There
are
other
reasons
why,
for
us
at
least
for
us,
it
was
a
good
idea
to
rewrite
it
in
Rust,
so
I
mean,
as
Christian
mentioned,
one
of
the
biggest
problems
we're
facing
was
supporting
different
operating
systems,
basically
porting
the
application
and
distributing
it,
and
we
thrust
this.
This
is
a
greatly
simplified.
You
can
compile
a
entire
application,
basically
to
a
single
file,
executable
statically
linked,
so
you
have
no
problems
with
other
dependencies
that
you
also
need.
B
Don't
need
to
bother
users
to
to
install
those
dependencies
support,
different
versions
which
we
noticed,
that
was
a
terrible
maintenance
cost
in
terms
of
Securities
rust
is
secure
by
Design,
which
is
partially
important
for
set
itself,
but
even
more
importantly,
for
the
libraries
that
we
are
using,
that
they
are
doing
the
heavy
lifting
so
and
encryption
should
be
secure
right.
So
it's
great
that
a
library
that
we
are
using
for
it.
It's
simply
about
this
in
Rust
correctness,
so
capturing
bugs
earlier
and
what
I
mean
by
that
is
like
in
Python.
B
Also
saves
a
lot
of
time
in
the
long
run
and
as
Christian
mentioned,
the
two
big
enablers
were
the
libraries
that
they
are
now
available
in
the
rust
ecosystem.
So
the
main
the
most
important
one
is
Sequoia
pgp,
which
I
will
shortly
mention
in
one
of
the
later
slides
and
pilot
3,
which
allows
for
an
easy
integration
of
rust
into
Python
and
yeah
actually
in
set.
B
We
use
a
lot
of
other
libraries,
so
yeah,
it's
great,
that
the
Russ
ecosystem
supports
all
of
it
or
actually
all
of
the
two
that
we
need,
but
those
two
I
would
say
they
are
the
the
biggest
enablers
for
us,
but
at
the
beginning
at
least
it's
not
all
great.
So
there
were
some
challenges,
so
that
was
the
first
serious
rust
project
for
I.
Think
all
of
us,
so
Gary
helped
ourselves
but
left,
let
us
to
to
try
developing
ourselves.
B
So
what
we
learned
is
that
you
need
to
be
motivated
and
spend
some
time
on
it
and
it's
it's
very
useful
to
have
an
expert
if
you
are
stuck
so
the
compiler
only
give
you
can
you
give
you
so
much
of
an
error,
output
and
so
the
second
point
is
mostly
important
in
transition
period.
So,
as
crucial
mentioned,
we,
the
plan
is
to
replace
the
python
app
piece
by
piece,
which
means
that
in
that
traditional
period,
same
functionality
needs
to
be
first
re-implemented
in
Rust
and
still
being
maintained
in
Python,
so
yeah.
B
So
in
this.
In
this
time
you
need
to
support
both,
but
once
you
get
rid
of
the
Python
code,
then
this
this
double
maintenance
cost
disappears
so
yeah
as
I
already
hinted,
that's
pretty
much
the
core
of
offset
so
the
Sequoia
pgp
library
and
it's
yeah.
So
now
it's
no
longer
that
new
but
see
it's
relatively
newcomer
to
the
open
pgp
world
and
we
treat
it
as
a
complete
replacement
of
gnu
PG.
B
We
yeah,
which
has
many
problems,
but
for
us
the
biggest
one
was
the
distribution
and
platform
support
yeah.
So
the
1.0-
oh
sorry,
1.0
release
was
already
like
more
than
two
years
ago.
So
far
we
noticed
it's
extremely
stable
and
it
comes
especially
in
contrast
to
new
PG
as
a
library,
first
approach,
meaning
that
the
developers
develop
mainly
the
library
and
the
surrounding
tools,
the
and
and
also
other
tools
and
in
those
tools
they
use
this
line
Library
as
anybody
else
would
use.
B
So
we
pretty
much
have
the
same
experience
with
the
library
as
they
do
and
yeah
the
experience
is
very
good
so
far
and
Community
is
very
active
and
supportive.
We
even
notice
that
once
we
are
discussing
next
feature
that
we
want
to
implement
on
our
gitlab
issue
tracker
and
then
one
of
the
Sequoia
developers
just
showed
up
suggesting
us
a
different
approach,
because
they
were
actually
creating
a
new
sub
project
that
yeah
that
that
really
actually
simplified
the
feature
that
you
wanted
to
develop.
So
that
was
actually
great.
B
I
know,
I,
don't
know
why
like
how
he
came
up,
how
he
found
our
issue,
but
yeah
I
appreciate
that
he
did
and
segue
it's
not
just
one
project,
his
entire
ecosystem.
So
you
will
find
tools
like
SQ,
which
is
a
command
line
interface
to
Sequoia
that
can
do
similar
things
like
the
gpg
command
line
interface.
Can
you
also
have
the
verifying
key
server,
also
written
by
the
same
team,
which
holds
the
public
server,
this
Keys
open
pgb.org,
but
actually
we
do
use
for
biomatit.
B
So
so,
given
your
Ras
project
with
with
a
bit
of
Pi
or
three
code,
you
can
build
it
into
python
packages
that
you
just
yeah.
If
you're
familiar
with
PP
you
just
sorry
with
python,
then
you
just
peep,
install
it
and-
and
you
have
it
so
now,
I
will
very
briefly
go
over
how
the
project
is,
is
more
less
organized
and
a
high
level
at
a
high
level.
How
how
it
looks
so
we
have
two
main.
B
So
in
the
repository,
you
have
two
crates,
the
main
one
containing
the
library
which
implements
pretty
much
all
the
functionality,
and
it
also
includes
the
binary
part
so
yeah,
so
the
the
lowest
hanging
fruit
for
us
for,
like
for
full
transition
to
rust-based
tool,
was
to
first
start
with
the
CLI.
So
that's
why,
along
writing
the
Library
part
that
does
all
the
actual
work.
We
also
run
the
binary
that
users
could
use
hopefully
soonish,
and
there
is
a
second
grade
set
RS
which
only
is
concerned
about
providing
python
biting.
B
So
it's
a
very
thin
wrapper
over
the
the
main
set
crate
and
here's
an
example.
So
still
it's
unreleased
code,
but
yeah
it
would
pretty
much
have
similar
experience.
I
I
would
expect
later.
So
that's
a
CLI
of
a
purely
purely
now
done
in
Rust,
where
you
can
create
your
encrypted
packages,
decrypt
them
transfer.
You
can
add
your
keys
and
now
yeah
and
because
of
a
sequoia
philosophy,
you
could
also
use
other
tools
to
interact
with
the
keys.
So
so
yeah,
it's
it's
very
nice
because
then
the
tools
are
interchangeable
and
this.
B
So
this
command
line
interface
actually
nicely
reflects
at
the
high
level
Library
API.
So
here,
I
just
give
an
example
of
how
the
encrypt
API
looks
like,
but
you
you
can
expect
this
a
similar
thing
for
decryption
for
transfer.
So
we
try
to
to
write
a
very
simple
high
level
API.
So
in
this
case
you
have
just
a
simple
encrypt
function:
taking
two
arguments:
encryption
options
so
something
like
files
that
you
want
to
encrypt
recipient
signer.
B
That's
a
compression
level
that
you
want
to
to
use
so
just
some
basic
information
that
you
need
to
create
a
package
and
destination,
and
here
we
want
to
try
something
more
generic.
So
so
in
principle
you
can
encrypt
either.
So
that's
that's
what
that's
the
only
thing
that
the
python
implementation
can
do
actually
is
to
encrypt
to
to
a
local
file
system
with
the
Ross
implementation
we
are
already
able
to
not
only
Implement
local
but
also
SFTP
remote
destination.
B
S3
is
still
a
work
in
progress
which
also
maybe
hints
on
what
we
are
trying
doing
under
the
hood.
So
this
is
already
Christian
mentioned
that
we
need
to
support
actually
quite
a
massive
data
sizes,
sometimes
so
sometimes
in
the
order
of
terabytes.
So
we
need
to
be
careful
with
how
we
use
we
use
resources
so
pretty
much
the
entire
workflow,
it's
Stream
So,
basically
in
memory
passing
from
one
pro
one
function
to
call
it
to
another,
so
compression
check
summing
encryption
and
then
saving
to
a
destination
happens
in
a
single
stream.
B
So
there
is
no
data
stored
temporarily
or
in
case
of
a
remote
destination
that
needs
saved
directly
at
the
remote.
So
no
extra
storage
is
used
on
the
computer.
That
is
performing
the
package,
creation
and
yeah,
and
one
have
it
here.
So
the
code
that
you
see
here
is
still
a
bit
of
a
mock
up.
So
it's
it
should
soon
land
on
the
main
branch,
but
it's
not
there
yet
so,
hopefully
we
can
make
the
high
level
API
as
nice
as
possible
and
yeah.
B
It's
also
important
for
us
to
do
it
that
way,
because
then
this
this
API
will
consume
ourselves
in
the
CLI
and
also
we
need
to
expose
it
to
python.
So
if,
if
it's
as
simple
as
possible,
then
it's
very
good
for
ourselves,
even
okay
and
as
promised.
So
this
is
how
you
would
then
expose
your
code
to
python.
So
here
you
will
see
that
yeah,
it
mostly
looks
like
I
mean.
B
Actually
it
is
mostly
rust
code,
but
with
some
extras,
so
you
will
see
here
the
two
macros
and
they
are
coming
from
pi
over
three
and
basically
they
turn
this
function,
definitions
into
something
that
can
be
done
packaged
into
a
python
module.
So
the
first
one
just
converts
a
rust
function
into
a
python
function.
The
second
creates
a
python
module,
and
here
you
will
see
that
once
you
create
a
pi,
a
module,
then
you
just
add
things
to
it
like
in
this
case,
this
encrypt
function
and
you
can
also
add
classes.
B
You
can
add
pretty
much
everything
that
you
you
would
need
in
an
easy
way
and
yeah,
and
then
here
in
the
comments,
you
will
see
how
you
would
call
it
in
Python.
So
just
like
any
other
python
code,
you
call
a
function
with
arguments
and
that's
it
pretty
much
and
yeah,
so
pile
3
gives
a
lot
still,
it's
not
perfect.
So,
for
example,
it
doesn't
work
with
generics
at
least
not
yet
so
so
yeah.
That's
why
this
this
API
is
a
bit
uglier
than
the
one
that
I
showed
on
the
last
slide.
B
Okay,
then
I
will
switch
a
bit
gears
and
talk
about
maintenance
or
something
maybe
not
very
exciting,
but
definitely
something
that
most
people
need
to
do,
and
especially
in
this
project,
we
need
to
support
a
lot
of
platforms
and
yeah,
so
basically
compile
against
a
lot
of
targets,
which
means
that
we
yeah
we
need
to
somehow
make
it
easy
for
ourselves
automate
as
much
as
possible.
So
here
I
will
show
you
a
bit
what
we
do.
C
B
This
context-
and
here
also
are
linked-
a
very
nice
talk
from
Ross
Nation
UK,
which
cause
much
deeper
into
the
maintenance
of
a
rust
project,
but
there
are
two
messages
that
I
took
from
it
that
are
relevant
here.
It's
like
be
careful
on
which
dependencies
you
take
because
you
need
to
to
live
with
those
choices.
Of
course
you
can
replace,
but
sometimes
it
comes
at
the
cost
and
second
yeah
try
to
be
up
to
date.
So
it's
okay
to
delay,
update
it's
but
yeah,
just
don't
ignore
them
and
yeah.
B
So
then,
as
I
mentioned
yeah,
we
try
to
make
a
heavy
use
of
CI
and
automation,
because
what
we
need
to
do
so
here
you'll
see
like
an
overview
of
our
CI.
So,
of
course
we
want
to
test.
We
also
need
to
test
against
multiple
platforms.
Unfortunately,
on
GitHub
we
can
have
native
Runners
for
Linux,
Mac
and
windows,
so
at
least
we
can
make
sure
that
it
compiles
natively
on
those
platforms.
B
We
also
need
to
build
python
packages
for
three
different
operating
systems
for
two
different
architect,
architectures,
so
x86
and
rm64,
and
also
the
the
raw
CLI
binaries.
Also
for
three
operating
systems,
or
so
for
two
architectures,
and
also
for
Linux,
we
built
for
both
Gypsy
and
muscle.
So
hopefully
this
will
even
satisfy
Centos
7.
B
and
find
and
yeah.
So
that's
that's
what
what
runs
usually
in
the
pipeline.
So
we
try
also,
since
we
have
a
lot
of
dependencies
and
the
builds
can
run
quite
long.
So
we
also
do
heavy
caching
and
also
since
the
last
release
or
two
releases,
a
goal
of
rust.
There
is
sparse
registry
that
also
greatly
speeds
up
the
CI,
so
yeah
we
are
trying
to
make
use
of
all
of
the
new
good
things
coming
and
yeah.
B
Also
so,
and
once
we
are
ready
with
for
release,
then
we
can
just
push
new
tags
to
to
the
to
the
repository
and
the
CI
then
create
python
packages
for
for
the
python,
The,
Binding,
crate
and
and
CLI
binaries
for
for
the
main
crate.
So
here
you
would
see
an
example
of
I
think
it's
a
previous
release
of
the
python
binding.
B
So
if
you're
familiar
with
python,
then
you
would
notice
that
it's
a
python
package
index-
and
here
you
will
just
see
those
packages
as
as
you
would
see
for
any
other
purely
python
project
and
as
for
the
CLI,
we
store
it
at
this
moment
at
least
just
on
a
gitlab
packages,
I
think
it's
called
so
yeah.
So
then
you
can
just
pick
binaries
from
there
and
here
exactly
you
can
see
all
the
targets
that
we
are
compiling
and
yeah
this
cross
yeah.
B
B
Likes
nice,
changelog
and
when
stuff
it's
nicely
documented,
but
it's
extremely
annoying
to
write
yourself
especially
try
to
figure
out
what
I
would
change
since
the
last
release.
So
we
also
want
to
automate
it,
and
here
we
yeah.
So
first
we
always
write
our
commits
in
the
conventional
commit
syntax.
So
then
they
are
actually
nicely
machine,
readable
and
also
human
readable.
And
then
we
use
this
tool
called
the
git
glyph,
which
happens
to
be
also
written
in
Rust,
and
it's
it's
an
extremely
flexible
generative
for
changelog
and
yeah.
B
So
that's
pretty
much
how
we
get
nice
change
loads
without
pretty
much
any
effort
and
also
yeah.
We
need
we
want
to
have
separate
releases
for
both
of
the
crates.
So
so
we
wrote
a
very
simple
script
in
this
case,
just
just
a
bar
script
that
that
uses
the
cliff
and
then
some
Unix
tools
to
to
nicely
and
to
create
changelog
for
each
of
the
crates
and
then
tag
them
with
correct
next
release
base
on
like
if
there
are
any
breaking
changes,
new
features
or
just
bug
fixes
and
as
of
staying
up
to
date.
B
So
it's
also
quite
annoying
to
just
check
if
there
are
any
new
versions
of
all
the
dependencies
that
that
you
have
so
we
looked
on
some
solutions,
so
a
popular
one,
it's
I
think
called
the
panda
bot,
but
it
only
works
for
GitHub.
So
it
was
not
a
choice
for
us.
We
came
across
a
renovated
bot
which
actually
works
quite
well
for
us
and
what
it
does.
So.
B
So,
let's
see
here,
you
notice
that
there
was
a
new
rust
release,
so
it
created
Denmark
and
if
you
see
here
in
the
bottom,
the
pipeline
effect,
which
is
which
is
maybe
a
bit
unfortunate,
but
at
least
it's
something
that
we
can
easily
check
and
actually,
in
this
case
there
were
some
new
clippy
linters
which
improved
our
such
as
the
Improvement
to
our
code,
which
actually
were
quite
nice.
So
so
without
much
effort,
we
just
fixed
the
linking
problems
that
we
are
running
again
at
the
latest
raster
version
and
yeah.
B
So
the
same
happens
for
for
crates
that
you
are
depending
on.
So
that's
that
saves
us
quite
a
lot
of
time
in
the
long
term
and
also
it
so.
This
renovate
bot
creates
a
like
a
parliament
issue
that
that
keeps
track
of
like
what
Mrs
have
already
been
opened.
You
can
also
close
some
if
you
really
don't
want
to
update
a
certain
dependency
and
also
you
can
see
what
will
be
created
in
the
next
Mr
window.
B
So
so
you
can
also
fast
forward
it
or
just
wait
for
for
the
for
the
window
and
yeah
then
going
to
the
next
section.
Let's
talk
about
the
future
and
yeah,
so
we
need
to
predict
and
predictions
are
hard,
especially
like
if
we
think
about
the
future,
so
so
yeah,
it's
always
difficult
to
predict,
but
we
we
at
least
I
identified
a
couple
of
things
that
we
know
that
that
we
need
to
improve.
B
So
some
of
the
things
that
they
are
present
in
the
python
code
base
are
still
need
to
be
migrated
to
teras.
But
it's
it's
already,
not
so
many
things
to
do.
There
are
other
things
that
we
identified
as
improvements,
for
example
we
okay,
we
measure
that
we
are
actually
quite
good
in
terms
of
performance,
so
we
are
already
faster
than
the
python
version,
which
wouldn't
maybe
be
surprising,
but
actually
the
the
the
python
code
call
calls
underlying
like
C
libraries.
B
So
so
it's
not
like
we
are
competing
against
interpreted
code.
We
are
competing
against
also
compiled
code,
so
so
it's
nice
that
we
already
faster
but
I,
think
we
can
do
better.
So
here
in
this
plot,
I
mean,
if
you
want,
you,
can
also
open
in
the
new
tab
and
explore
it.
It's
an
interactive
profile,
but
but
you
don't
need
to
read
much
in
like
text
on
it.
I
can
just
explain
you
that
yeah
there
are
these
two
blocks
and
which
represent
like
two
stages
of
the
workflow.
B
That
takes
the
most
time
the
left
one.
It's
about
compressing
the
data
and
the
right
one
is
it's
about
encryptical
data
and
those
two
run
on
separate
threads,
so
in
principle
they
run
in
parallel,
meaning
that
the
the
speed
of
the
entire
workflow
is
limited
by
the
by
the
longer
one
so
compression
and
yeah.
Probably
we
cannot
do
much
more
about
speeding
up
compression
other
than
multi-threading
it
and
there's
a
nice
Library,
rust,
Library
gzp,
which
I
really
try
to
include,
but
yeah.
B
It
has
some
strong
cons
constraints
on
what
types
you
can
use
it
with
so
yeah.
So
it's
a
bit
hard
to
to
to
satisfy
the
compiler
to
work
in
our
workflow.
So
this
will
require
more
work,
but
obviously
it's
something
that
we
need
to
do
at
some
point,
especially
to
to
speed
up
this
like
multi-terabyte
data
transfers
and
second
big
one
is
as
I
mentioned
yeah.
B
We
now
have
support
for
local
file
system
for
sfd
remote
SFTP,
but
as
Christian
mentioned,
we
are
slowly
moving
to
S3
because
it
feel
fits
our
overall
architecture,
much
better
than
SFTP
and
yeah,
but
still
we
don't
have
support
instead
for
direct
encryption
to
S3,
which,
like
okay
for
SFTP,
it
was
actually
surprisingly
easy
to
do
because
of
the
type
system
yeah.
The
type
system
basically
guided
us
how
how
to
how
to
implement
it
for
SFTP
for
S3,
the
API
is
quite
different
and
it's
unclear
how
to
do
it.
B
I
mean
definitely
possible,
but
we
just
need
to
spend
a
bit
more
time
on
it
and
yeah.
What's
maybe
also
quite
surprising,
is
that
yeah
we
didn't
Diplomat
it
in
Python,
because
actually
it
wasn't
obvious
how
to
do
it,
so
it
was
easy
easier
in
Rust
and
yeah.
The
next
thing
that
we
still
want
to
improve
so
packaging,
as
I
mentioned
in
the
University.
B
It's
it's
a
has
a
already
a
good
story,
because
you
have
just
a
binary
that
you
can
pretty
much
give
it
to
a
user
and
run
it,
but
our
users
are
not
very
technical.
So
still,
some
having
like
an
easy
installer
native
to
our
production
would
be
great,
including
a
self
updater.
So
there
is
a
new
release.
They
get
a
pop
out
here.
They
click
yes,
I
really
want
to
update
and
that
that
would
be
it
and
then
yeah.
B
So
the
big
thing
that
is
still
missing
is
GUI,
and
once
we
have
GUI
it
would
be
nice
to
have
just
a
single
executable
that
could
be
used
with
both
interfaces
yeah.
So
that's
that's
for
the
distribution,
but
yes,
I
mentioned
the
big
missing
component.
Is
the
GUI
and
a
GUI
in
Rust
is
a
bit
of
a
complicated
topic.
So
if
you
check
this,
this
page
at
least
maybe
not
most
but
there's
still
a
lot
of
GUI
Frameworks
in
us,
they
have
all
different
features,
different
problems
and
so
yeah.
B
It's
unclear
at
least
to
us,
which
one
will
be
the
best,
but
at
least
from
what
we
saw
and
think
that
or
the
features
that
they
would
provide.
So
so
we
identified
tari
at
least
as
the
one
that
would
be
solving
the
installers
or
self-updator
integration
to
the
operating
system,
a
problem
yeah
so
so
that
would
be
already
great.
The
one
problematic
or
maybe
not
problematic
is
a
design.
B
Choice
yeah,
it's
so
Tara
is
a
backend
for
which
you
need
to
write
the
front
end
so
like
the
popular
ways
are
to
just
use
some
JavaScript
Frameworks
like
react
and
yeah,
which
we
would
really
prefer
to
avoid,
and
so
so
here
I
think
we'll
try
to
use
U,
which
is
a
rust
front-end
framework
compiling
to
a
web
assembly
so
perfectly
compatible
with
with
tari
and
actually
so
yeah.
So
this,
actually
just
it's
a
real
example,
so
a
screenshot
of
a
real
app.
So
definitely
it's
possible
to
do
it.
B
Yep
probably
will
go
for
it
and
that's
maybe
one
of
you
has
already
great
experience
with
another
GUI
framework,
so
I
would
be
happy
to
talk
about
it.
But
if
we
were
to
go
for
you,
this
means
that
maybe
Christians
dream
of
having
everything
in
the
browser
could
be
fulfilled,
so
yeah
front
end
would
already
be
built
to
with
a
web
assembly.
B
B
So
that's
why
we
chose
to
move
in
this
case
to
rust,
which,
despite
its
steep
learning
curved,
proved
to
be
a
very
good
choice,
allowed
us
to
for
early
bug,
detection
and
easy
maintenance
and
distribution
and
yeah.
Also,
a
good
good,
CI
and
automation
saves
you
time
so
so
yeah,
that's
also
what
we
can
really
suggest
to
anyone
who
didn't
want
to
spend
time
on
it
yeah
it's
worth
it
and
yeah.
So
for
now
that
would
be
it
and
yeah.
So
now
will
be
time
for
questions
and
then
time
to
relax
and
yeah.
B
D
B
I'm
not
sure
if
you
already
saw
this
slide,
let
me
try
to
go
back
to
it.
Just
I
think
pretty
much
the
one
of
the
first
slides
exactly
so
so
we
have
this
Central
service
called
the
portal
or
the
Biomet
ID
portal,
which
which
pretty
okay
so
for
for
the
keys
themselves.
We
use
these
Keys
openpgp.org
key
server,
which
I
mean
which
holds
the
keys
and
does
the
basic
verification.
B
So
it
will
send
you
an
email
which
would
confirm
that
at
least
this
key
belongs
to
a
person
who
can
access
this
email
address.
So
at
least
you
you
are
reasonably
convinced
that
the
key
is
associated
with
this
email
address,
and
so
that
would
be
one
layer
and
second
layer
on
this
portal.
We
require
users
to
to
register
their
keys,
and
also
a
portal
has
like
two-factor
authentication
with
a
trusted.
Identity
provider
so
probably
are
familiar
with
switch
audio
ID.
So
it's
it's
like
a
Swiss
educational
identity
provider
yeah.
B
So
so
this
allows
also
to
link
so
at
least
link
their
account
with
real
personal,
real
researcher,
let's
say,
and
then
also
we
require
that
the
email
address
would
be
matching
between
this
identity
and
the
key
and
the
email
address
has
been
verified
by
the
key
server
and
also,
if
necessary,
we
can
because
it's
also
a
human
Act
of
approving
the
key.
So
then,
if
a
human
is
unsure
about,
so
it's
like
a
central
Authority
in
the
end.
B
So
if
the
central
Authority
is
unsure,
then
they
consider
reject
the
key
or
ask
for
additional
confirmation.
So
that's
that's
within
the
Biomet
ID
outside.
No,
we
don't.
We
cannot
assure
it,
but
at
least
like
still,
you
can
not
maybe
use
like
the
classical
web
of
trust,
but
still
you
can.
You
can
use
some
form
of
it
if
you
want
locally.
B
F
B
E
B
C
B
Also,
that's
that's
one
of
the
other
problems
like
yeah.
How
about
your
colleagues
like
if,
if
you're
working
in
a
team
that
at
least
you
need
to
convince
them
that
Russ
is
a
good
choice,
so
they
can
learn
it
if
they
don't
know
it
already
yeah.
So
that's
that's
also
a
bit
challenging
at
the
beginning.
B
Guess
you
all
key
management
is
actually
quite
yeah,
I
mean
very
important
for
us
and
also
quite
tricky
so
yeah.
So
we
also
looked
at
the
the
JavaScript
library
that
Christian
mentioned,
but
obviously
it
wouldn't
be
a
right
choice
and
there,
if
so,
that's
made
by
a
company
con
called
proton
which
is
behind
the
protonmail
product,
and
they
also
have
a
go
library
for
the
backend
part.
B
But
yeah
I
mean
none
of
us
knew
go
and
also
like
yeah
I
mean
there
are
some
strong
opinions
about
both
rust
and
go,
but
yeah
at
least
like
before,
knowing
one
of
those
languages
yeah.
So
somehow
we
would
need
to
make
a
choice,
but
still
rust,
like
from
the
opinions
and
also
from
Gary,
of
course,
was
a
more
a
better
choice
for
us
than
going.
The
goal.
Direction.
B
Are
so
now
they
are
simplifying,
also
splitting
a
bit,
so
they
they
at
the
beginning,
I
think
they
only
had
a
monorepore.
Now
they
try
to
create
a
more
separate
repositories
where,
when
it
makes
most
essential,
for
example,
the
SQ
binary
until
like
a
few
weeks
ago,
it
was
part
of
the
the
main
repository.
Now
it
has
its
own.
Also,
the
key
store
now
has
I
mean
they
are.
They
just
started
with
the
key
store
and
they
created
a
separate
repository
for
it.
B
F
C
B
That's
also
a
problem
that
we
were
not
able
to
solve
until
we
moved
to
rust
because
of
supporting
yeah,
the
infamous
Center
7.
We
need
to
support
gnu
PG,
a
version
that
is
long
outdated
and
doesn't
support
almost
actually,
no,
no
I
think
it's
any
elliptic
curve
algorithm,
so
pretty
much
you're
stuck
with
RSA.
So
that's
why
we
were
not
able
to
enforce,
but
now
once
we
completely
migrate
to
Sequoia
we
can.
We
can
start
enforcing
modern
encryption
algorithms.
Yes,.
E
B
The
protocol
level,
so
just
oh,
we
do
have
so.
Actually
we
have
kind
of
a
double
encrypt,
so
protocol
you
mean
the
transport
layer
yeah,
we
do
so
I
mean
like
so
SFTP
and
S3
are
encrypted
as
but
still
the
idea
is
to
not
trust
the
intermediates.
So
the
package
actually
travels
through
the
network
and
it's
stored
on
a
disk,
so
in
principle
like
sysadmins
could
read
it
and
it
and
what
we
are
transferring
is
like
a
patient
data
from
hospitals.
F
B
B
So,
at
least
at
this
point
we
are
not
really
concerned
about
the
the
transfer
speed
because
we
are
really
limited
by
either
compression.
Okay
data
could
be
compressed
already.
Then
you
can
skip
compression
step
and
just
go
purely
for
encryption
and
okay
I
would
need
to
double
check
it,
but
I'm
quite
sure
that
still
transfer
it
slower
than
the
encryption.
B
A
A
B
B
So
it's
very
similar
because
at
least
like
the
the
heavy
part
is
it's
written,
I,
think
in
Rust,
so
or
actually
that's
I'm,
not
sure
if
it
matters
but
yeah
anyway,
so
like
from
performance
perspective,
I
think
I
didn't
notice
any
difference,
so
it's
mostly
like
here
would
be
like
how
portable
it
is.
So
muscle
is
like
the
most
portable
thing,
like
especially
all
Santos.
B
B
E
C
C
B
C
E
E
F
E
E
E
B
S,
okay,
I
mean
I,
mean
the
things
before.
Writing
may
be
anything
bigger
than
a
prototype
in
Towery.
We
we
will
also
try
it
in
iced,
maybe
in
iguri,
and
and
see
a
lot,
not
to
repeat
the
pro
like
the
thing
with
python.
Okay,
python
works,
so
let's
just
go
for
it
like
yeah,
it's
better
to
have
something
that.
B
F
F
B
Yeah,
it
seems
that
way,
but
the
zip
is
uncompressed.
So
so
we
don't
do
double
cooperation
so
zip
here
is.
It
purely
serves
a
container
role
so
just
to
have
a
single
file
because
also
like
yeah.
We
wouldn't
trust
users
to
keep
more
than
one
file
like
organized.
So
as
long
as
it's
they
move
a
file,
they
move
everything.
B
So
I
mean
so.
This
is
a
very
long
shot
and
definitely
not
something
that
you
want
to
try
this
year,
probably
not
till
the
next
year
but
yeah.
So
you
just
comprise
to
webassembly
because
it's
a
front
Library.
So
it
needs
to
be
deployed
at
something
like
a
browser,
but
in
principle
you
can
compile
pretty
much
any
rust
code
into
webassembly,
so
the
I,
the
long
shot
idea
would
be
here
instead
of
like
Distributing,
desktop
applications
or
like
command
line
applications
just
compile
everything
to
web
assembly
and
use
it
from
browser
yeah.
B
And
here
the
challenge
is
that
we
still
have
some
C
dependencies
like
openssl
leap.
Ssh
could
probably
go
away
if
we
move
fully
to
S3
and
I,
don't
remember
other,
maybe
some
compression
stuff.
It's
also
like
has
a
dependency
on
the
C
library.
So
those
I
mean
you
can
compare
C
to
webassembly,
but
the
story
is
more
difficult
and
threading
from
what
I
read
it's
very
early.
So
it's
a
prototype
stage
so
yeah.
So
if
there
were
soft
threading,
then
of
course
a
performance
would
suffer
a
lot
from
that,
but
I
mean.
B
B
Yeah
exactly
exactly
process
in
general
yeah,
so
yeah
I
mean
I,
think
it
was
as
smooth
as
possible,
but
still
yeah
interfacing
between
two
languages.
It's
a
bit
difficult,
so
pile
three
does
a
great
job
but
still
doesn't
convert
any
type
to
any
type.
So
sometimes
you
need
to
to
to
to
to
to
to
some
wrappers
and
to
maybe
infer
some
of
the
types
yeah
which
of
course
can
produce
an
error
and
also
yeah,
because
you
need
to
let's
say,
report
the
progress
of
compression
and
transfer.
B
Then
you
need
to
somehow
have
callbacks
from
rust
to
python,
which
was
also
I
mean,
maybe
not
complicated,
but
quite
annoying
to
implement.
So
there
was
like
these
small
things
that
were
annoying
but
manageable,
so
so
yeah
I
mean
if
any
one
of
you
is
the
point
of
like
okay
I
have
a
python
project.
I
would
like
to
solely
translate
it
into
Russia.
I
can
definitely
recommend
it.
It's
it's
not
that
not
bad
I
mean
so
it
actually
is
quite
good.