►
From YouTube: Keynote: Seamless Customer Experience at Walmart Stores Powered b... Maneesh Vittolia & Sriram Komma
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Keynote: Seamless Customer Experience at Walmart Stores Powered by Kubernetes@Edge - Maneesh Vittolia, Principal Architect & Sriram Komma, Principal Product Owner, Walmart
At Walmart, while major application software can and does operate in the cloud, stores or any client edge compute cannot avoid the intermittent network events that can create less than ideal availability and performance of the software during those times. This can lead to poor customer experience and/or failed transactions during checkout. Because of Walmart's scale of serving around 265 million customer every week, the comnbined effect on customer experience as well as the loss of revenue is pretty huge. To overcome the issue between Stores and cloud, Walmart is building and rolling out the next generation of Point of Sale (POS) systems on highly resource constraint edge computing environment using modern service mesh based technologies designed to allow maximum business flexibility, extreme performance and rapid deployment and powered by Kubernetes.
https://sched.co/UdJf
B
A
I'm
not
going
to
make
you
wait.
I
know
waiting
is
really
difficult.
It's
very
hard,
like
waiting
at
the
airport
to
board
a
plane
or
even
waiting
for
a
simple
download
to
complete
on
your
laptop.
It's
really
hard.
Similarly,
waiting
at
checkout
register
within
a
store
for
a
few
minutes
is
like
an
eternity.
We
at
Walmart
believe
checkout
register
as
the
last
and
critical
step
in
the
customer
journey.
We
needed
to
ensure
that
customer
continues
to
get
best
experience
throughout
that
shopping
journey.
B
Well
with
that
said,
who
here
do
not
know
what
are
who
Walmart
is
marae's
offense
a
bit
curious
to
see
that?
Well,
nobody
great,
but
anyways,
created
a
slide,
so
I'll
just
go
through
them
to
give
a
brief
history
on
about
ourselves.
From
a
humble
beginning,
as
a
small
discount
retailer
in
Rogers,
Arkansas
Walmart
had
opened
up
thousands
of
stores,
both
in
US
and
internationally
through
innovation.
We
are
creating
a
seamless
experience
to
let
customers
shop,
anytime
and
anywhere
online
through
mobile
devices
and
in-store.
A
A
A
B
We
say
Walmart.
The
first
thing
that
comes
to
mind
is
scale.
So,
let's
quickly
talk
about
what
one
much
scale
is
like,
we
are
just
huge
wal-mart
operates
around
11,000
retail
stores.
Worldwide
we
serve
about
265
million
customers,
both
online
in-store
on
a
weekly
basis
and
Walmart
employees,
approximately
2.2
million
employees
around
the
world.
We
have
significant
presence
in
e-commerce
world
as
well,
and.
A
We
have
not
been
able
to
achieve
this
scale
without
the
help
of
technology.
As
an
example,
in
80's
we
installed
largest
private
satellite
system
in
u.s.
connecting
our
operations
with
voice
data
and
video
communications.
Almost
a
decade
ago,
we
started
using
RFID
to
improve
our
supply
chain
efficiency
and
point-of-sale.
Checkout
application
within
the
store
is
no
app,
no
stranger
to
accelerated
technology
advancement.
We
have
various
ways.
We
help
our
customers
to
complete
their
last
step
in
their
shopping
journey,
like
like
self
checkout
checkout
with
me,
scan
and
go
and
check
out
register.
A
So
it's
it's
important
to
understand
some
of
the
background
on
checkout
application.
So
last
couple
of
decades
we
have
been
using
legacy
point-of-sale
checkout
system,
and
it's
still
going
strong.
This
system
was
built
with
highest
level
of
resiliency
we're
required
business
data
was
presented
mr.
land
on
the
Stu
edge.
I
would
say
it
was
our
first
edge
compute.
A
We
have
had
where
stores
specific
data
was
on
in-store
servers,
so
registers
will
reach
out
to
install
servers
and
get
data
from
data
located
in
data
center,
and
any
transactions
data
would
be
stored
and
forwarded
to
back
to
the
data
center.
So
more
than
four
years
ago,
we
started
rolling
out
our
first
cloud
native
checkout
application
as
a
move
towards
moving
from
legacy
to
modern
stack
architecture
where
data
and
the
application
was
located
on
cloud.
A
They
said
some
limitations
just
accessing
the
applications
over
when
network
over
remotely
check
out
applications
for
customer
experience.
As
an
example,
regional
network
issues
can
happen,
and
that
can
happen
because
of
some
inclement
weather
constructions,
related
activities
in
the
stores
or
around
areas,
a
bridge
not
going
as
planned,
or
even
some
bad
changes
being
rolled
out
into
cloud
and
customers
are
impacted
because
of
that
because
of
our
scale
and
large
number
of
customers
who
walk
into
the
store.
B
A
So,
looking
at
the
potential
problems
and
followed
with
the
current
architecture,
we
decided
to
take
our
mana
application
check
our
applications
closer
to
the
customer
at
edge.
We
define
edge
as
where
customer
connects
with
Walmart
stores,
where
we
are
closer
to
the
customers,
where
interactions
with
customers
are
taking
place,
where
applications
can
run
in
offline
and
autonomous
mode
where
application
can
work
with
and
without
cloud.
A
This
helped
in
bringing
check
out
applications
closer
to
the
customers
within
the
store.
So
if
the
service
or
the
data
is
not
remotely
accessible
from
cloud,
customers
can
successfully
complete
their
last
step
in
the
shopping
journey
within
the
store
and
said
edge
is
still
functional.
However,
this
solution
didn't
come
without
our
fair
share
of
challenges
that
we
needed
to
solve
as
an
example,
first
being
like,
how
do
we
roll
out
services
and
Kuban?
It
is
cluster
across
thousands
of
store
across
the
world.
How
do
we
co
locate?
A
The
data
with
the
service
at
the
edge:
how
do
we
improve
the
time
to
market
how
to
bring
the
isolation
to
the
services,
so
that
application
is
fault,
tolerant
and
does
not
affect
the
customers
experience?
And
how
do
we
improve
the
productivity,
so
developers
can
quickly
build
it
and
deliver
to
the
business
apart.
B
From
this,
there
are
a
few
more
questions
that
needed
to
be
addressed:
how
to
manage
orchestrate
and
configure
a
given
service,
along
with
making
sure
we
keep
it
secure
and
achieve
resiliency
by
making
best
practices
default
and
biggest
of
all,
given
the
scale
and
we
operated.
How
do
we
manage
and
keep
the
deployments
consistent
across
all
the
stores?
B
After
some
exploration,
we
went
ahead
with
micro-services
managed
by
kubernetes,
as
our
solution,
as
a
rule
might
already
be
aware
of
open
resource,
orchestration
deployment,
configuration
resiliency
and,
most
of
all,
with
little
bit
of
hard
work.
Security
and
micro
service
architecture
helped
us
to
solve
few
of
other
parameters
as
well.
Well,
with
out
of
the
doorway,
came
out
the
hard
part,
our
journey
to
bring
a
well-thought
solution
to
life.
A
So
we
we
have
achieved
a
lot
as
a
part
of
this
journey
like
localizing
the
data
at
the
edge,
simplifying
the
consumption
of
Cuban
ideas,
platform
in
the
organization,
improving
the
observability
and
making
sure
that
platform
is
well
integrated
and
resilient
enough
as
a
that's.
What's
the
our
need
for
the
application?
Was
these
topics
may
itself
take
dedicated
session,
but
we
are
going
to
quickly
go
through
each
of
them.
So
when
we
started
proof
of
concept
of
humanity's
almost
like
two
years
ago,
one
of
the
important
thing
that
we
wanted
to
solve
is
like.
A
So
we
used
a
Kafka
based
architecture
to
bring
store
specific
data
from
cloud
for
any
changes
that
could
take
place
in
the
cloud
and
that
gets
persisted
on
the
data
database
located
within
the
store
outside
cubanía
cluster
on
a
VMware
for
any
data
that
is
produced
at
the
store
would
be
stored
and
forwarded
back
to
the
cloud.
This
is
how
we
synchronize
data
at
edge
for
any
cloud
specific
changes.
So
this
is
how
we
kept
store
as
independent
as
possible.
Next
in
the
journey
was,
how
do
we
simplify
the
consumption
of
humanity's
platform?
A
B
Right
now,
getting
an
app
deployed
equipment
is
clustered
but
little
painful
process
with
all
the
manifests.
That
needs
said
that's
needed
specifically,
while
deploy
into
thousands
of
stores,
so
get-ups
model
made
more
sense
here.
Basically,
we
went
basically
wanted
app
team
to
worry
about.
Apps
are
not
about
how
to
get
their
apps
into
kubernetes.
B
We
created
our
own
custom
3d
pipelines
where
once
app
teams
dropped
their
code
in
github,
everything
is
taken
care
from
building
an
image
and
pushing
it
to
Walmart
repository
and
using
the
same
image
and
getting
the
app
deployed
into
kubernetes
using
charts.
We
will
notify
the
app
users
between
each
and
every
stage
of
these.
A
Next
step
in
the
journey
was:
how
do
we
improve
the
observability
of
the
services,
the
system
across
thousands
of
cluster?
This
was
very
important.
It
was
easy
to
do
it
in
cloud
we're
here
like
few
regional
compute
environment,
but
here
we
are
talking
about
thousands
of
communities
cluster
located
across
thousands
of
location,
not
just
in
u.s.
even
outside
us.
It
was
a
big
challenge,
challenge
of
us.
How
do
we
get
this
logging
in
metrics
data
from
these
stores
on
thousands
of
clusters
back
to
our
site,
reliability
team?
B
Observability
is
very
critical
for
any
application
journey.
Point-Of-Sale
at
edge
makes
it
even
more
complex
because
of
the
scale
that
they
operate
in
all
the
way
from
making
sure
logs
are
available
to
getting
metrics
and
being
able
to
start
clearing
application
performance
problems.
We
started
using
fluent
different
bit
to
ship
our
logs
to
a
splint
cluster
and
the
same
logs
were
sent
to
cold
storage,
which
can
be
retrieved
for
audit
needs.
Along
with
that,
we
use
Prometheus
for
metrics
scraping
and
for
alerting.
B
We
use
our
in-home
custom
notification
bridge
to
centralize
app
metrics
across
multiple
stores.
We
ship
certain
metrics
to
a
version
of
federated
metrics,
back-end
and
stored
them
for
little
over
a
year.
That
way
we
can
start
mapping
the
trends
post
our
busy
holiday
season.
We
also
leverage
application
performance
management
for
our
app
clergy.
A
So
next
in
the
journey
was
since
it's
going
to
be
deployed
in
stores.
The
question
was
like
how
secure
this
platform
is,
and
we
didn't
want
it
to
get
our
customers
with
security
risks.
Another
thing
was
like:
how
do
can
we
deploy
these
clusters
across
large
number
of
stores,
even
in
network
bandwidth,
environment,
to
distance
locations,
and
how
well
integrated
is
this
platform
is
with
other
open
source
technology.
So
when
there
are
some
platform
support
related
issues,
we
continue
to
support
it
correctly
and
provide
best
experience
to
our
business
like.
B
Money
said
secure
is
very
critical
for
point-of-sale.
We
start
looking
at
a
platform
and
said
to
ourselves
app
security
should
be
a
default
offering
from
making
app
communication
secure
by
using
walmat
sign,
sets
to
making
sure
secrets
remain
a
secret.
We
basically
wrote
our
custom
operators
for
these
use
cases
in
order
to
make
sure
we
sustain
in
network
outages.
We
face
the
secrets
from
our
vault
instance:
that's
neared
for
the
app
and
keep
them
may
well
been
the
cluster
8
CD.
B
B
There
are
few
remote
stores
running
in
low
bandwidth
in
distant
locations
and
during
deployments
we
certainly
don't
want
to
over
him
or
van
network
and
cause
any
outages.
So
we
implemented
a
local
casting
solution
with
network
rate,
limiting
the
download
that
were
throttled
to
make
sure
there
is
no
impact
to
the
store
bandwidth
image
layers
already
pulled
as
part
of
the
previous
deployments
became
available
locally
on
the
clusters
they
required
layers
for
the
new
deployments
would
get
downloaded
by
now.
B
Everyone
should
have
guessed
the
next
time
that
I'm
going
to
use
walmat
scale
with
the
scale
we
need
to
operate.
We
needed
a
simple
way
to
manage
fleet
of
clusters
across
multiple
stores
and
other
environments,
so
we
built
our
own
custom,
in-home
federated
environment,
where
we
can
manage
kubernetes
clusters.
Basically,
we
manage
covenant
is
using
kubernetes.
We
started
using
federated
control,
prin
for
various
purposes,
from
inventory
management,
namespace,
reservation
app
and
cluster
lifecycle
management,
to
name
a
few.
B
A
So
now
everything
looked
great:
we
have
edge
based
community
cluster
and
we
have
cloud.
But
what
happens
if
edge
based
community
cluster
becomes
unstable
or
even
like
server
goes
bad,
and
we
wanted
to
have
a
resiliency
at
that
point
so
that
a
customer
can
are
not
impacted.
So
we
built
resiliency
at
the
checkout
register
level
outside
of
communities.
A
They
are
going
to
be
making
calls
to
H
clusters
over
on
wire
and
grace,
and
if
the
service
or
even
cluster
or
even
the
source
store
server
is
not
healthy,
it
will
fail
over
to
the
same
set
of
services
running
in
the
cloud.
So
that's
how
we
build
a
resiliency
from
check
or
register
standpoint,
but
what
about
application
level
resiliency
check
out
application
is
composition
of
dozens
of
micro
services
deployed
in
cubanía,
t's
cluster
same
set
of
services.
We
deployed
it
into
cloud.
A
A
The
call
is
going
to
be
paid
over
envoy,
sidecar
and
service,
be
located
in
cloud,
are
going
to
be
act
as
acting
as
a
secondary
backup
and
when
service
B
goes
bad
envoy
is
going
to
be
sending
a
request
back
to
service
B
in
cloud
and
when
service
B
goes
back
up
its
recovered
envoy,
sidecar
proxy
is
going
to
be
switching
the
traffic
back
to
service
B
in
in
humanity
cluster.
This
is
how
we
make
sure
and
check
out
system
is
resilient
to
the
problem.
A
B
A
Now
so
far
we
have
rolled
out
250,
plus
stores
in
us
and
1h
in
China
and
Japan,
and
along
this
journey
we
had
a
lot
of
learnings
as
an
example
like
first
being,
integration
is
time-consuming.
We
operate
in
brownfield
environment
and
we
have
to
integrate
our
system
with
this
new
Cuban
ready
platform
with
the
existing
system
that
exists
in
Walmart,
and
it
takes
more
time
even
to
move
services
from
cloud
to
educate,
compute
environment
in
humanities.
A
It's
not
easy
to
lift
and
shift
the
services
even
for
the
simplest
services
and
data
needs
to
always
follow
the
service,
and
one
of
the
important
thing
is
like
edge.
Environment
often
require
different
strategies
and
different
platform
component.
Considering
physical
and
technical
constraint
environment
we
operate
in.
B
Well,
like
people
earlier
called
out,
make
miss
prices
default
by
leveraging
templates
and
charts,
which
will
help
with
easy
consumption,
automate
common
workflows
using
orchestrators
and
operators
where
necessary,
but
last,
but
not
the
least
for
edge,
be
prepared
for
any
spirit.
For
example,
bandwidth
limitations
that
was
called
out
earlier.
A
And
our
agenda
does
just
doesn't
end
here.
We
have
a
lot
to
achieve
and
they're
our
future
open
challenges
that
we
wanted
to
solve
as
first
being
providing
data
persistency
within
kubernetes
cluster.
Currently
we
are
persisting
data
outside
community
on
a
VMware
VM
located
within
store
servers.
We
wanted
to
bring
it
inside,
so
it's
more
simplified.
Next
being
is
we
wanted
to
reduce
compute
in
the
store
environment
and
being
able
to
run
humidity,
is
in
the
smallest
footprint
as
possible
and
third
being
regionalizing
the
deployment?
A
Now
today
we
are
deploying
all
our
services
and
community
cluster
from
u.s.
cloud
region
and
due
to
distance
location
deployment
over
market
store
has
been
time
consuming
and
it
effects
time
to
market.
So
we
wanted
to
bring
the
market
proximity
for
deployment
and
make
our
market
self
in
ml,
and
with
this
we'll
wrap
up,
thank
you
for
having
us
here.
We
are
hiring
on
and
we
are
hiring
at
all
location.