►
From YouTube: Kubernetes + Encrypted Memory = Security * Privacy - Harshal Patil & Pradipta Banerjee, IBM
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Kubernetes + Encrypted Memory = Security * Privacy - Harshal Patil & Pradipta Banerjee, IBM
The Memory Encryption on hardware is coming soon. From Intel's TME/MKTME[1] to IBM's Ultravisor[2], hardware manufacturers are aiming to make sure 'what's written by the process stays within the process'. Once the hardware is out, it will change the way we perceive the security and privacy in the cloud. In this talk, we will discuss briefly on the upcoming memory encryption technologies and how we modified kata container runtime to handle kubernetes' Ephemeral Volumes (aka, EmptyDir volumes) to keep your data and application protected from the container image registry (encrypted at rest) to runtime (protected by memory encryption). For the demonstration, we run a container image with the encrypted TensorFlow model using kubernetes such that even the root user on the worker node won’t be able to read the model parameters. [1] https://goo.gl/Xt3MJf [2] https://goo.gl/X2A5yx
https://sched.co/MPdQ
A
A
A
We
take
a
quick
look
at
ways
to
secure
the
data
at
every
stage
of
its
life
cycle,
followed
by
introduction
to
memory
protection
and,
of
course,
encryption
and
then
conclude
with
kubernetes
integration.
So
this
is
the
high-level
flow
that
we
will
follow
as
part
of
this
session
today.
So
let's
get
started
next
slide,
please.
A
So
how
do
we
secure
data
and
code
in
order
to
dig
deeper
into
this?
Let's
envision
a
scenario
where
you
have
created
a
machine-learning
inference
pipeline,
you
have
trained
the
model,
and
that
is
the
one
that
is
used
for
influencing
the
models.
The
trained
model,
the
inference
code
is
your
sensitive
data,
that's
the
one
you
would
want
to
protect
from
any
unauthorized
access
across
its
various
stages.
A
You
can
protect
the
data
which
is
at
rest
using
any
current
encryption
technologies
that
are
there
right.
While
the
data
is
in
transit,
you
can
secure
it
using
HTTP
and
TLS,
but
how
do
you
protect
the
data
which
is
in
use
right
when
the
data
is
in
memory?
How
do
you
protect
it
from?
Let's
say
other
programs
which
has
privileged
access?
How
do
you
protect
that
data
from
malicious
admins
who
have
access
to
the
entire
system
memory?
A
How
do
you
protect
it
from
hackers
who
have
been
able
to
gain
admin
privileges
by
exploiting
the
system
owner
abilities?
For
example,
we
did
see
some
of
these
cases
with
respect
to
dirty
cow
and
the
Runcie
vulnerability
that
that
came
in
there
are
even
side-channel
attacks
which
could
take
the
data
out
from
the
processor,
cache
and
and
so
on.
So
how
do
you
prevent
the
data
that
is
in
use?
We
will
see
in
subsequent
slides?
How
do
you
protect
the
data
which
is
in
use
and-
and
that's
the
focus
of
this
session
next,
like
this.
A
Ok,
so
the
protection
of
the
data
that
is
in
use
is
a
huge
focus
area
today
that
everyone
across
the
industry
is
working
to
ensure
how
to
secure
the
data
that
is
in
use.
The
work
is
happening
across
CPU
architectures
Intel
was
the
first
one
to
came
up
with
secure
guard
extensions,
the
XE
X,
which
creates
a
secure
and
cliff
then
Intel,
has
also
come
up
with
total
memory.
A
Encryption
MD
has
similar
technologies,
which
they
call
as
secure
memory,
encryption
and
and
secure
encrypted
virtualization
IBM
Power
has
similar
technology,
which
is
termed
as
secure
VM
and
protected
execution
facility.
One
thing
to
note
is
that
that
these
technologies
are
continuously
improved,
to
make
it
easier
for
end-user
consumption.
For
example,
today
there
are
cloud
offerings
leveraging
Intel
SGX,
for
example,
there
is
OBM
cloud
data
shield
and
as
your
confidential
computing,
which
leverages
SDX,
but
then
we
are
taking
the
technology
to
the
next
level.
How
do
you
protect
the
entire
memory?
A
A
So
what
these
technologies
do,
what
is
fundamental
to
these
technologies
that
we
saw
like
IBM,
protected
execution
facility,
a
total
memory,
encryption
and
and
so
on?
What
it
does
in
very
simple
terms:
is
you
create
a
black
box
in
memory
and
put
your
stuff
inside
that
black
box,
so
whatever
is
inside
that
black
box
is
protected
from
anything
that
is
outside
and
that's
the
fundamental
concept
and
and
how
it
does
that
we'll
see
in
the
upcoming
slides
next,
please.
A
Okay,
so
as
we
as
we
saw
in
the
previous
ones
right
so
what
we
have
to
do,
we
have
to
create,
or
we
create
a
black
box,
and
we
have
to
put
the
sensitive
stuff
into
the
black
box
for
the
IBM
power
hardware
case.
Secure
VM
is
the
black
box
all
right.
So
so,
let's
go
through
the
these
the
layers
bottom
up,
so
we
have
the
PF
the
protected
execution
facility
capable
hardware.
That's
uses
TPM
for
secure
and
trusted
boot
of
the
firmware.
A
Now
the
firmware
and
the
hardware
ensures
that
we
have
memory
protection
for
secure,
VMs
and,
and
so
we
have
both
normal
VM
and
secure
VM
that
you
see
in
the
diagram
right
so
both
runs
on
top
of
Linux
k,
vm
the
arrows,
the
arrows
that
you
see.
Those
indicates
that
you
cannot
access
the
SVA
memory
from
either
the
hypervisor
that
is
k,
vm
or
any
other
vm,
for
example,
the
any
app
on
the
hypervisor,
whether
it's
privileged
or
not
privileged,
cannot
access
to
memory
inside
this
VM.
A
Similarly,
it
and
onion
secured-
or
let
me
call
the
unsecured
VM-
cannot
access
the
SVM
memory.
Even
another.
Svm
cannot
access
memory
belonging
to
other
a
sphere,
so
what
we
see
is
anything
that
is
out
sites.
Svm
cannot
access
any
data
that
is
used
inside
it.
So
SVM
is
our
black
box,
which
gives
that
protection
boundary.
Now
what
we
do
is
we
extend
this
SVM
concepts
to
containers.
A
Now,
if
you
see
the
SVM
image
this
VM
beam,
a
VM
image
is:
is
the
encrypted
route
FS
plus
lock
boxes
that
you
see
and
then
cryptic
secrets?
Now
this
lock
boxes,
if
you
see
R,
can
have
any
secrets
that
you
that
you
would
want,
let's
say,
for
example,
the
lock
boxes
can
have
secrets
that
you
might
want
to
use
for
decrypting
the
container
encrypted
container
image
itself
and
and
in
subsequent
slides
I.
A
Shall
we'll
talk
about
in
more
details
about
how
this
is
done,
but
this
lock
box
is
again
wrapped
with
the
system
for
a
system
public
key.
If
you
see
this,
this
IBM's
server
or
the
system
with
protected
execution
facility
has
TPM.
So
this
lock
box
is
wrapped
with
the
TPM
public
key,
what
it
means
that
this
can
be
decrypted
only
on
the
systems
with
which
with
using,
which
probably
keep
this
has
been
encrypted.
A
So
that's
the
whole
concept
of
that,
and
one
of
the
things
from
an
end-user
point
of
view
is:
there
are
no
code
changes
that
are
needed
for
your
application
or
the
container
absolutely
no
code
changes
needed
for
your
application.
All
the
changes
are
in
the
stack
in
with
respect
to
the
qmu
that
kernel
and
the
firmware
and-
and
that
takes
care
of
securing
your
containers.
If
you
want
to
know
more
details
about
the
underlying
technologies,
the
hardware
architecture
changes,
the
kernel,
changes
and
so
on.
There
is
a
link
that
is
there
on
the
slide.
A
Ok,
so
now
we
have
your
secure
container,
which
is
the
container
embedded
inside
this
VM.
But
how
do
we
use
this
with
kubernetes
right
now?
What
we
do
is
what
we
need
is
a
VM
runtime
right.
We.
What
we
need
here
is
a
VM
runtime
which
can
run
the
containers
inside
SVM.
So
what
we
do
is
we
leverage
the
cotta
container
runtime,
which
is
a
VM
runtime.
B
For
it,
so
as
fourth
dimension,
we
need
to
use
cotta
in
order
to
leverage
SVM
and
SVM
allows
you
to
have
your
memory
protected
from
the
hypervisor,
which
means,
if
you
happen
to
run
containers
inside
the
SVM.
The
memory
page
is
used
by
this
by
the
containers
will
be
a
protector
from
the
hypervisor
and
the
system.
B
So
a
choice
here
is
Cotta
which
which
can
be
used
to
launch
Qumu
based
VMS,
and
so
we
will
be
able
to
use
SVM
on
it,
but
having
just
launching
the
VM
is
not
in
a
continuous
use,
lot
of
other
aspects
where,
like
ephemeral,
volume
and
I/o
and
whatnot.
So
let's
see
how
we
can
integrate
ACM,
Carta
and
kubernetes
together.
So
typical
workflow,
if
you
see
how
Carter
works,
gets
integrated
into
SVM
into
into
kubernetes
is
like
you
have
a
cube
layer
and
then
he
talks
to
continuity
or
cRIO.
B
Moving
on
to
this
thing,
so,
let's
see
a
typical
workflow
that
we
see
in
a
kubernetes
one
right,
so
you
have
a
host
and
then
there
is
a
part
that
you
launch-
and
you
have
this
multiple
in
the
unit
containers
and
there's
a
finally
app
containers
and
these
containers,
while
they
run,
can
communicate
with
each
other
using
a
memory
back
ephemeral,
storage.
Typically
in
kubernetes
one.
This
is
called
an
empty
directory
type
volume.
B
So
how
do
we
take
a
typical
application
like
this
and
try
to
make
it
more
as
secure
as
possible
using
the
SVM?
So,
let's
see
what
how
how
these
runtimes
differ
in
the
way
they
deal
with
ephemeral
volume
on
empty
datatype
volume.
The
work
we
did
with
cata
enables
us
to
move
the
ephemeral
volume
inside
the
conference,
a
virtual
machine.
Now
you
know
that
the
ephemeral
one-time
volume
or
empty
rate
eight
volumes
has
a
lifecycle
tightly
coupled
with
a
part.
So
that
means
there.
B
There
is
no
need
to
keep
this
data
on
the
host
when
where,
as
long
as
the
part
is
running
so
when
this,
when
we
use
the
cata
runtime,
this
volume
is
more
inside
the
confines
of
this
virtual
machine
and
when
you
will
use
SVM,
this
ephemeral
volume
and
its
contents
will
be
also
encrypt
are
protected
from
the
host,
just
like
any
other
memory,
content
of
the
containers
so
going
back
to
the
earlier
flow.
If
you
reimagine
the
earlier
flow,
you
have
multiple
unit
containers.
B
You
have
app
container,
so
you
have
the
same
memory
back,
ephemeral,
storage,
but
now
it's
running
inside
svm,
protected
memory
project.
So
everything
that
you
see
inside
the
red
thick
border
is
protected
from
the
host.
There
is
nothing
that
visible,
so
any
unit
container
that
writes
into
him.
Although
ephemeral,
storage
or
reads
from
the
infamous
stories
or
a
container
that
might
read
for
my
fuel
storage,
everything
is
confined
within
this
boundary
read
boundary
which
which
host
is
not
able
to
see.
B
Even
even
if
oz
tries
to
take
a
memory
dump
of
VM,
the
the
pages
will
come
encrypted
and
there's
nothing
that
what
can
read.
So,
let's
imagine
the
typical
work
scenario.
Let's
say
if
you,
if
you
have
a
a
tensor
flow
model
which
you
want
to
protect
it
from
the
host,
and
it's
it's
probably
your
IP.
What
you
can
do
here
is,
you
can
have
a
let's
call
it
a
data
container
where
you
can.
B
You
can
embed
this
encrypted
tensorflow
model
and
when
you
want
to
run
this
scenario,
what
we'll
have
to
do
is
you
can
get
a
say,
a
symmetric
key
which,
from
your
kms
or
wherever
you
want
to
fetch
your
keys
from
put
that
key
inside
a
female
volume.
Then
again,
this
encrypted
tensorflow
model
d,
keep
listing
so
flow
model
using,
say,
GPT
tools.
And
finally,
you
run
your
inference.
Everything
that
happens-
and
everything
here
that
is
happening-
is
completely
invisible
to
the
host.
B
So
I
have
a
small
demo
here,
where
we
have
this
lovely
Persian
cat
and
they
said
tensorflow
inference
demo,
where
we'll
try
to
in
query
our
model
to
the
to
the
tensorflow
model,
to
the
vendor
inference
engine,
and
we
should.
We
should
be
able
to
see
whether
we
are
able
to
hide
the
our
encrypted
tensorflow
model
from
the
host
or
not.
Let's
go
to
the
demo.
B
So
there
we
are
starting
your
demo
and
we
have
this
run
times.
If
you
that's
a
runtime
class.
This
is
what
it's.
This
is
very
defined,
whether
you
are
to
run
the
your
workload
inside
a
kata
or
not,
then
we
have
app
container
and
then
we
have
a
fetch
container.
Imagine
this
container
very.
You
will
like
to
fetch
the
decryption
keys.
B
Then
you
have
a
date
and
then
you
have
a
data
container
where
you
might
in
this
particular
server
using
it
for
a
tensorflow
model
which
is
encrypted
and
the
tools
container
art
the
choice
of
tool
here
is
GPG.
You
could
use
anything
else
and
then
he
is
an
empty
directory
type
volume
that
we
are
using,
which
is.
We
are
calling
cache
volume.
B
B
There
you
go,
and
if
you
see
here
the
string
values,
the
first
element
is
Persian
cat
and
it
has
a
confidence
of
8.7.
Third,
so
the
higher
the
confidence,
the
more
the
accuracy
HP
is
nearby
10.
So
we
are
pretty
sure.
That's
a
purse
string
cat,
but
that's
the
interesting
part
is:
let's
see
if
you,
if
you
are
able
to
see
this
model
from
the
host,
you
may
remember
in
tensorflow
model
was
encrypted
right.
B
But
if
you
go
to
this
particular
T
wire
leap,
cubelet
parts-
and
if
you
go
to
the
part
three
volumes,
kubernetes
empty
tray
type
folder
inside
that
you
should
be
able
to
see
the
the
model
that
is
right
available
for
the
host
to
see,
and
this
kind
of
is
not
desirable
solution
in
a
secure
environment.
So
what
let's
see
what
what
we
want
to
behavior
when
we
use
a
cutter
and
time.
A
B
B
B
Let's
see
if
we
are
able
to
see
it
this
time
and
as
you
can
see
this,
that
particular
model
which
was
visible
earlier
on
the
host,
it's
no
longer
visible.
So
that
concludes
the
demo.
Let's
resume
the
presentation,
however,
the
story
is
not
yet
complete.
We
just
address
the
one
aspect
of
it.
When
you're
running
secur,
secur,
secur
vm,
you
want
entire
data
to
be
completely
encrypted
or
protected
from
any
unauthorized
access.
B
Here
we
have
to
note
that
you
know
the
registry
is
not
encrypted
and
even
if
you
use
registry,
which
is
password
password
protected,
you
still
have
admin.
You
still
have
the
unauthorized
admin
access
problem
to
deal
with,
also
the
images
the
way
the
images
are
extracted
on
the
host.
So
whenever
you
do
a
pull
of
a
continue
image
using
using
continuity
or
cRIO,
the
images
are
pulled
on
the
host
and
extracted
and
then
the
data
and
time
we'll
use
that
extracted
image
tool
to
use
the
launch
the
container
inside
the
virtual
machine.
B
This
is
also
not
desirable
because
that
will
expose
our
container
images
to
the
hosts
vulnerable
hosts
on
the
root
on
the
host.
So,
whatever
you
be
doing
in
that
direction,
well,
we
are
trying
to
encrypt
the
continuing
images
themselves.
So,
as
of
now,
we
don't
have
the
support
for
container
images
to
be
encrypted
in
OCS
spec,
but
we
are
there's
ongoing
effort.
B
We
presented
this
idea
in
dhaka
con
that
happened
just
last
month
and
then
we
have
a
ke,
p,
kubernetes
enhancement
proposal
to
add
the
support
for
encrypted
container
images
in
kubernetes,
and
we
will
be
talking
about
it
in
Shanghai,
2019
in
coupon,
so
come
and
join
us
there.
Also,
as
I
mentioned,
the
continuing
image
is
getting
extracted
and
a
host
problem
right.