►
Description
Speaker: Dan Cundiff, Technical Architect Consultant at Target Corporation
This presentation will cover the problems we needed to solve, the journey we took to get there, and the lessons we learned along the way. We’ll cover the technical and non-technical aspects of this story.
A
A
So
I'm
curious
before
I
begin
just
as
a
show
of
hands
who
considers
themselves
working
in
like
a
large
IT
shop
or
an
enterprise.
Just
curious,
okay,
actually
really
good
portion
of
you.
So
I
guess
what
I'm
going
to
see
is
either
some
head
nodding.
Like
agreement,
yeah
I
went
through
this
too
or
you're
going
to
be
like
furiously,
writing
down
notes
to
take
back
to
your
company.
This
is
going
to
share
a
story
here
about
how
we
stood
up
Cassandra
at
target
and
there's
a
lot
of
good
lessons
in
here.
A
So
I
think
if
you
take
some
of
the
stuff
back
to
come
to
you
work
at
you
get
some
good
stuff
out
of
it.
So
so
I'm
Dan
cundiff
work
at
Target
I
work
on
an
API
team.
So
with
the
it's
useful
to
understand
the
context
of
all
the
stuff
I'm
going
to
talk
about
here,
because
they'll
give
you
an
idea,
is
I
talk
through
this,
what
we're
using
it
for
and
stuff
so
target's
api
platform.
A
So
we've
got
mobile
devices,
com,
vendors
partners
and
stuff
that
use
all
kinds
of
target
data
and
we
get
that
target
data
to
them
through
restful,
UPI's,
so
products
locations
inventory
all
kinds
of
them,
there's
about
30
api's.
Actually
we
have
so
we've
got
consumers,
you
know
that
are
actually
calling
right
guys
inside
of
target
and
outside
and
you've
got
systems
you
can
imagine,
and
maybe,
if
you've
worked
on
an
api
program,
this
data
comes
from
anywhere
really
old
systems
packages.
You
know
you
name
it.
A
So
if
you
so
think
about
that
last
one
for
a
minute
there,
these
providing
systems
in
many
cases
they
can
be,
they
can
be
pretty
slow
right.
I
mean
some
of
these
things
might
be
mainframe
systems
like
said
legacy,
packages
that
are
just
not
up
to
par.
For
once
we
expose
that
data
and
get
it
really
utilized
really
high.
You
can't
call
those
providing
systems.
A
Some
of
them
are
pretty
costly
to
call
directly
they.
Some
of
them
are
really
old.
Were
they
just
can't
scale?
We
can't
actually
make
them
go
any
faster
for
any
reason,
and
then
a
lot
of
times.
We've
got
to
take
all
this
data,
that's
in
multiple
layers,
so
imagine
like
location
data.
There
might
be
several
different
system
to
have
a
stated.
A
We
got
to
put
in
one
place
and
then
sometimes,
as
data
is
just
not
even
in
good
systems
to
begin
with,
sometimes
they're,
just
in
list
somewhere
or
something
else
so
a
lot
of
times
what
we
got
to
do
is
we
got
to
get
this
data
into
a
single
plays
so
that
that's
actually
what
our
problem
was.
Is
all
these
things
right
here
and
so
so
you
think
about
the
existing
tools,
probably
in
the
enterprise
right,
you
get
your
traditional
already
BMS
is.
Some
of
them
are
pretty
costly
right.
A
If
you
think
about
the
processes
for
your
traditional
databases,
they're,
probably
not
a
really
good
fit
for
what
you're
trying
to
do.
And
then,
if
you
just
look
at
your
the
suite
of
options
that
you
have,
you
probably
don't
have
that
many
vendors
or
tools
to
choose
from,
and
so
in
our
case
is
certainly
certainly
a
problem
for
us.
A
So
we
took
a
look
at
of
our
existing
relational
databases
in
one
of
the
first
things
that
we
one
of
the
very
first
things
that
really
jumped
out
is.
Is
there
not
multi-tenant?
You
know,
in
our
case
growing
in
multiple
data
centers,
the
data
is
being
spread
in
multiple
places.
We've
got
guests
all
over
the
country
and
we've
got
data
centers
only
in
certain
places,
and
then
you
know
what,
if
you
want
to
go
to
the
cloud
with
some
of
the
stuff?
A
A
If
there
is
like
a
process
for
making
changes
to
those
relational
databases,
you
feel
like
you
have
all
the
control
you
have
I
mean
I,
don't
think
we
did
availability,
you
know
systems,
it
might
have
outages
and
other
tools
that
you
had
and
then,
in
our
case,
I'm
gonna
talk
about
this
a
little
bit
more
later,
but
is
are
those
things
automatable
right?
Are
they
being
managed
by
hand?
Are
you
using
something
like
chef
or
puppet
to
manage
him
so
I?
A
Remember
it
was
about
2012,
went
out
to
q
con
and
was
actually
with
another
colleague
from
target
p
he's
here
in
the
crowd,
and
we
went
to
a
netflix
presentation
where
Adrian
cockcroft
was
talking
and
I
mean
I'd
been
one
to
have
a
really
good
reason
to
track
sonder
for
some
time
and
I.
Remember
listening
to
Adrian
talk
and
he
just
sort.
This
is
actually
just
sort
of.
If
there's
any
vendors
in
here.
This
is
sort
of
the
power
of
someone
really
influential
mentioning
your
product.
A
He
said
some
da
Dada
about
Cassandra
he's
like,
but
honestly,
if
you're
an
enterprise,
you're,
probably
gonna,
use
datastax
right.
He
just
said,
as
a
matter
of
fact
and
I'm
actually
had
never
heard
of
data
stacks
before
that
prior
I
mean
I
might
have
like
CrossFit
and
never
like,
stuck
in
my
mind
like
an
indelible
impression,
but
that
certainly
did
leave
me
with
an
indelible
impression.
I
went
back
and
it
was
took
a
look
at
it
and
I
mean
I
just--literally.
I
just
went
back
and
approached
pete
and
I
was
like
hey.
You
know:
we've.
A
This
problems
really
been
coming
to
a
boil
here.
We
should
probably
do
something
about
it,
so
I
just
went
to
Peter
like
hey,
let's
just
do
it
so
we
did
and
keep
in
mind.
We
tried
other
things
in
the
past
with
you
know,
meteoric
success
trying
other
things,
so
it
wasn't
like
we
just
rationally
ever
actually
made
a
decision,
but
I
think
the
real
power
here
is:
is
that
just
a
if
you're
going
to
try
something
just
do
it
and
let
the
results
speak
for
themselves,
try
to
succeed
fast
or
fail
fast,
either
one
right.
A
A
You
know
whether
its
eventual
consistency
or
just
performance
wise,
there's,
a
whole
range
of
different
ways.
These
api's
behave
some
of
them
they're
kind
of
batchi.
Some
are
more
real-time,
but
so
eighty
to
eighty
percent
was
a
really
good
fit.
The
other
things
about
Cassandra
is
just
the
years
and
development
had
enough
years
under
its
belt,
where
it
wasn't,
you
know
brand
new
technology
that
makes
it
attractive.
A
This
is
a
big
one,
and
it's
something
you
don't
underestimate:
I
think
with
existing
enterprise
tools.
You've
probably
done
this
yourself,
you
try
to
troubleshoot
them
or
you
want
to
make
it.
You
want
to
try
it
look
up
some
new
feature.
There's
probably
paywall.
You
have
to
go
to
right.
There's
you
got
to
sign
into
the
vendors
website
and
stuff
like
that.
The
results
have
got
to
be
googleable
I
mean
if
they're
not.
You
should
really
consider
as
a
very
heavily
discounted,
no
reason
not
to
go
with
that
tool.
A
In
this
case,
the
community
is
very
strong.
There's
lots
of
results
on
Google
that
actually
makes
a
lot
of
difference
because
the
people
are
going
to
fix
this
stuff
at
2am,
anyways
we're
just
going
to
google
any
how
to
learn
something
they
don't
know
and
then
just
a
company
would
support
it
right.
I
mean
you
can
go
with
kisan.
A
A
We
use
chef
at
Target
and
cassandra
is
entirely
shuffle
and
I'll
talk
about
cool
thing
we
released
for
the
community
later
here,
but
it's
it's
chef,
able
and
that's
really
important
to
us,
because
we
don't
like
to
do
things
by
hands
humans,
make
errors
and
stuff
like
that
right,
aligned
really
well
with
our
existing
investments.
So
you
know
we're
java
shop,
the
existing
hardware.
We
had
worked
out
pretty
decently
and
then
the
last
one,
this
stuff
matters
right,
because
it's
developers
you
you
want
to
make
the
right
choices
for
your
company
right.
A
You
want
to
make
really
smart
choices
so
that
your
company's
profitable,
sometimes
working
with
those
some
other
vendors
right.
You
can't
easily
do
that
right.
You
feel
like
you're,
getting
gained
when
you
think
your
gaming,
the
vendor
their
gaming.
You
right
you'll
never
win
their
masters
at
this
stuff
right,
so
this
frictionless
sales
model
is
really
important
to
me,
at
least
so
in
this
case,
is
just
very
straightforward.
Pricing,
there's
a
lot
of
other
tools
like
that
out
there
to
write
that
are
frictionless
and
I'm
I'm
drawn
to
those
in.
A
So
that's,
that's
a
pretty
big
reason
for
being
attractive.
So,
let's
think
about
these
adoptions
as
we
thought
and
decided
to
bring
it
into
IT
after
looking
at
the
reasons
is
attractive.
Really,
the
nature
of
this
talk
is
bring
the
stuff
into
enterprise
IT
right
of
the
nature
of
it.
I
think.
Probably
a
lot
of
you
can
empathize
with
all
this
sort
of
things
you
run
into.
You
know
one
of
the
persons
being
you're
going
to
have
to
sell
it
to
people
right.
A
You
just
can't
cavalierly
make
the
choice
to
do
it
on
your
own
and
go.
Do
it
you're
going
to
talk
to
certain
people
who
actually
care
about
these
outcomes
that
might
have
some
concerns,
etc.
It
just
takes
sticking
the
key
message
and
talking
to
the
right
people,
so
this
is
I
mean
no
sequel.
For
the
first
time,
I
mean
I
think
there
were
probably
skunk
work
projects
of
it
in
many
places,
but
this
is
going
to
be
production
on
a
much
larger
scale,
and
so
for
some
people.
A
You
know
you
know
you
have
to
do
the
extra
work
to
tell
them
like
yes,
this
is
actually
gonna,
be
the
first
time
we're
using
no
sequel
and
working
through
that
conversation.
It's
real
important.
Don't
underestimate
that
and
then
just
the
idea
we're
going
to
be
automating
this.
That
was
also
a
really
big
emerging
thing
a
year
ago,
and
so
you
couple
that
with
us
bringing
this
technology
kind
of
scary
to
do,
but
we
had
to
sell
that
as
well
right.
A
A
The
political
aspects
of
it
right
so
just
having
a
really
good
leader
on
your
team
who's
going
to
go
to
bat
for
you,
who's
going
to
not
just
write
you
the
blank
check,
but
also
defend
your
choices
and
help
sell
that
to
all
the
other
leaders
across
the
organization.
Who
are
you
know
questioning
what
you're
doing
they
were
curious,
etc.
So
that's
that's
really
important.
If
you
don't
have
that,
you
should
definitely
advocate
for
that,
ask
for
that,
raise
we're
in
a
sport,
etc.
That's
really
important!
A
So,
let's
think
about
once
we
decided
to
start
using
it.
What
were
some
of
the
things
we
know
we
were
going
to
run
into
so
a
lot
of
a
lot
of
these
providing
systems
for
these
api's
I
was
telling
you
about
have
only
sometimes
they
have
very
batchi,
like
mechanisms
or
getting
the
data,
and
we
knew
that
in
our
case
we're
going
to
have
to
do
a
lot
of
bulk
loading,
and
so
that's
not
an
ideal
way
to
be
getting
data
into
Cassandra.
It's
entirely
possible
and
we
have
to
do
it.
A
We
just
knew
that
was
going
to
be
a
challenge,
keeping
Cassandra
in
sync,
just
all
these
providing
systems.
It's
it's
going
to
get
interesting
because
right,
a
lot
of
these
systems
aren't
event-driven.
They
aren't
real
time.
Some
of
them
is
packaged.
Some
of
them
are
packaged
software,
which
you're
really
limited.
Then
you
can't
modify
them.
It's
really.
You
have
to
the
few
interfaces.
Those
systems
have.
You
have
to
deal
with
them.
You'd
end
up
having
to
write
some.
You
know
custom
integration
stuff
to
actually
get
that
stuff
into
Cassandra.
A
So,
let's
talk
about
staining
it
up
couple
parts
to
this
here
right,
so
this
this
being
one
of
the
first
distributed
systems
and
the
true
since
I
mean
we
had
splunk
I
mean
guys
might
be
familiar
with
that
that's
a
distributed
system
and
we
maybe
had
some
other
ones.
Oh
sorry
about
that,
the
that
was
new,
two
teams,
and
so
you
know
you
have
to
take
the
time
to
actually
explain
that
to
all
those
teams.
A
You
know
your
need
to
work
with
your
hardware,
partners
etc,
because
they're
not
going
to
know
what
these
systems
are,
and
it's
really
important
for
them
to
understand
that,
because
they're
not
centralized,
there
are
things
that
they're
going
to
need
to
do
differently.
The
next
point
is
obviously
really
key
to
this.
So
and
then
this
is
a
common
theme
that
you
you've
even
heard
this
morning
in
the
keynote
you've
heard
any
time
you've
done,
engagements,
kisan
or
datastax
is
don't
you
saying
right,
and
that
was
actually
a
real
barrier.
A
I
think
that
all
of
us
hit-
and
you
just
make
you
cringe
a
little
bit
every
time
happens.
But
in
this
case,
you
know
is
something
we
had
to
advocate
for
to
say.
Like
look,
you
really
can't
you
say
in
this
distributed
system
you're
destroying
the
distributed
nature
of
this
by
making
it
centralized
right.
You
want
to
use
the
cheap
disks
on
those
cheap
servers
right.
A
Another
thing,
too,
is
that
all
of
this
emerging
kind
of
in
the
same
set
of
years
we're
talking
about
SSDs
here,
probably
a
lot
of
the
teams
where
your
infrastructure
partners
might
be
using
spinning
disks,
still,
obviously
you're
gonna.
You
got
to
work
through
that
type
of
stuff
and
we
had
to
and
then
there's
things
like
if
you're
going
to
get
these
servers,
they're
going
to
give
them
to
you
and
there's
an
existing
way
they
manage
them,
but
because
it's
just
distributed
system,
you
know
you
need
to
manage
it
a
little
bit
differently
right.
A
So
in
our
case
you
know
things
like
backups
monitoring
swap
stuff
like
that.
I
mean
those
are
things
you're
going
to
be
doing
differently
on
those
Cassandra
servers
that
you
probably
don't
want
your
existing
infrastructure
partners
to
to
set
up
in
a
certain
way
right,
so
you
have
to
talk
them
around.
Why
that
that's
these
distributions
real
bit
different
and
then
the
last
one
is
they're
going
to
give
you
servers
into,
at
least
in
our
case
we
had
a
set
of
only
a
couple
of
configs.
You
could
choose
from
right.
A
It's
I
understand
why
they
have
a
limited
amount
of
them,
but
in
this
case
remember
it's
a
distributed
system
and
you
kind
of
don't
want
huge
servers,
so
just
make
sure
you
don't
settle
for
whatever
they
give
you.
You
know,
like
I,
said
all
these
other
points
above
you're
going
to
work
with
them
and
you're
going
to
talk
to
them,
say
you
know,
here's
the
reasons
why
you
want
to
do
these
things
differently.
That
includes
the
type
of
server
they're
going
to
give
you
a
full
sack,
ownership.
A
I,
don't
know
if
that's
like
a
term
outside
of
target
or
not,
but
I
mean
in
this
case
I,
don't
know
how
it
works.
In
your
prize,
but
you
there
may
be
like
a
tendency
to
want
to
hand
the
systems
off
to
another
group
who's
going
to
manage
them
and
stuff
like
that.
I
mean
in
this
case
we
literally
take
are
quite
a
large
deal
of
pride
in
our
work,
and
you
know
we
actually
want
to
get
those
2
a.m.
calls
we
want
to.
We
want
to
make
sure
it's
stable
it's
random
way.
A
A
Let's
see
here
right
so
closer
to
the
problem
you're
best
suited
to
solve
it,
I
mean
that's.
That's
really
key,
therefore
use
keeping
that
over
and
over
that's
that's
really
key
to
kind
of
a
point
there
and
you
have
things
you
have
to
think
about
and
you've,
probably
if
you're
training
earlier
or
you
probably
been
in
other
talks
today,
you've
gotta
tune
Cassandra
to
meet
your
needs
right.
A
Other
thing
is
this:
these
skills
are
low
at
the
time
a
year
ago.
I
think
it's
getting
a
lot
better.
You
know
more
more
companies
are
using
it,
people
are
getting
in
the
job
market,
etc.
Don't
underestimate
training
your
people.
This
is
different.
It
is
a
different
technology
right.
So
don't
don't
underestimate
that
make
sure
you
to
training
they
need.
In
our
case,
we
we
were
very
willing
to
do
that,
and
it's
something
that
worked
out
well
for
us,
I,
think
and
then
the
last
one
is
be
wary
of
the
promises
of
consultants.
A
So
there,
especially
a
year
ago,
there
were
a
lot
of
you
know.
Companies
are
saying:
hey,
we've
got
people,
we
just
make
sure
that
you
grill
those
people
like
get
your
best
Cassandra
person.
You
have
on
staff
and
just
make
sure
you
really
know
their
stuff,
because
I
still
think
that
to
the
first
point
there
aren't
a
lot
of
talent.
A
Getting
the
you
know,
your
developers
used
to
knowing
how
to
write
queries
in
the
right
way
and
everything
else
not
hurt.
In
the
system
encouraging
developers
to
actually
you
know
care
about
this
right
care
about
the
outcomes
you
shouldn't
have
to
say
that,
but
I
mean
that
stuff
does
matter.
You
know
in
Kurt,
like
the
full
stack
ownership
stuff.
You
know
those
developers
are
part
of
that
ownership
as
well,
so
encourage
them
to
like
research.
A
Things
make
things
better
when
there's
problems
own
it
right,
there's
pretty
simple
things,
but
they
need
to
be
said,
I
think
and
then
the
last
one.
You
know
it
is
easy
to
hide
bad
queries.
Full
table
scans
filters
stuff,
like
that,
like
just
make
sure
that
everyone
is
sharing,
what
they've
learned
that
they're,
making
the
right
choices
and
obviously
take
the
time
to
give
the
feedback
whenever
you're
your
clients
of
Cassandra
and
doing
things
right.
A
The
other
thing
is
right:
a
lot
of
these
developers
have
an
RD
BMS
background.
You
have
to
take
the
time
to
explain
the
differences
there.
In
our
case,
we
start
out
with
asti
onyx
actually,
and
there
were
some
things
that
made
a
switch
to
datastax
driver
I,
mean
obviously
was
supported
by
data
sacks.
There
were
some
features
that
were
better
and
I.
Think
Asti
onyx
was
the
was
using
thrift.
The
time
whenever
we
started
using
it
I
think
it
uses
ce
qu'il
now,
but
de
sacs
driver
CQ
else.
A
All
right
so
running
it.
What
were
some
of
those
challenges
right,
lots
of
machines.
Don't
do
things
about
hand,
goes
back
to
the
automation
point.
I
was
saying
you
you're,
going
to
want
to
use
something
like
sheffer
puppet,
in
my
opinion,
tamanna,
Jose
or
ansible,
whatever
your
favorite
tool
is,
but
comparing
those
machines
by
hand
is
you're
just
going
to
introduce
there
that's
going
to
be
pretty
bad
stuff.
A
A
It's.
My
earlier
point
is
so
your
remember
those
your
infrastructure
partners,
it's
a
larder,
large
organization
right
and
I.
Don't
blame
this
from
the
than
for
this
right
there
they're
seeing
these
certain
thing
of
well.
These
are
configured
very
differently
from
our
normal
servers.
We
need
to
bring
them
back
to
how
they
should
be
config
and
they'll.
A
A
In
our
case,
we
can
see
anything
and
that's
extremely
helpful,
because
that
allows
us
troubleshoot
things,
make
performance
better
and
everything
else
in
our
case
and
I'm
under
depression,
that
not
everyone
does
this,
but
in
Cassandra
and
the
jmx
interface
you
can
take
that
data
and
send
in
real
time
to
one
of
your
favorite
logging
Liz.
In
our
case,
we
use
splunk.
So
I'll
talk
about
the
in
beams.
A
The
nice
thing
about
in
our
case,
how
we
did
this
and
maybe
you're
doing
the
same
thing,
but
if
you're
taking
that
event
data,
you
can
correlate
it
with
your
log
data,
which
is
depending
on
Cassandra
and
if
you've
got
those
logs
from
Cassandra
and
your
apps
in
one
place
and
you're,
seeing
spikes
and
latency
in
your
app
you're,
also
seeing
spikes
and
Cassandra
row
that
allows
you
to
make
that
correlation
very
easily.
So
you
know
I
I,
don't
know
what
logging
tools
you
use,
but
in
splunk
that's
very
easy
to
do.
A
A
We
not
only
do
we
like
operationally
use
them
to
look
at
the
health
of
the
system
and
things
like
that,
but
we
have
alerts
set
up
across
these
things
so
for
your
wood
logging
framework
you're,
probably
doing
some
monitoring
right,
you're
doing
some
alerting
of
some
type.
These
are
useful
things
to
actually
alert
against
as
well,
and
you
can
see
these
later
see
other
so
sorry
about
that
cookbook.
Actually,
just
this
week,
Danny
Parker
personal,
my
team
wrote
the
data
sacks
Enterprise
cookbook,
and
it's
actually
here
on
github,
it's
publicly
available.
A
It's
easy
to
remember:
/
target
/,
DS
DSC
cookbook
go
out
there
and
fork
it.
Pull
requests
are
encouraged.
It
works.
That
is
actually
the
cookbook
that
we
use,
so
this
is
not
watered
down
in
some
way.
This
is
actually
the
cookbook
we
use
and
we
have
really
really
good
luck
with
it.
So
we're
pretty
excited
about
this
and
then
another
thing
that
Danny
did
is
he-
and
this
is
in
relation
to
all
the
things
I
was
saying
above?
Is
it
wrote
a
great
blog
post
on
tuning
cassandra
so
check
that
out
as
well?
A
These
do
I.
Don't
remember.
This
is
just
target
github.
No,
that's
where
our
target
tech
blog
is.
So
what
were
the
results
of
all
this
I
mean
really.
We
went
from
total
newbies
to
production,
ready
literally
in
two
months.
No
joke
that
moment.
I
was
telling
you
about
earlier,
where
I
went
to
Pete
I
was
like
hey:
let's
go
do
this,
we
got
all
the
hardware,
did
all
our
testing
app
dev
and
deploy
to
prod
two
months-
that's
entirely
possible.
A
So
if
you're
thinking
about
that
in
your
own
company,
this
is
something
that
this
is
a
bar
I
mean.
Maybe
it's
that
kind
of
high.
Maybe
it's
that
average
I
don't
really
know,
but
know
that.
That's
something
can
take
back
to
your
leaders,
the
other
big
thing
about
this
and
you
imagine
the
retail
line
of
work
that
we
did
this
right
before
our
peak
season.
So
talk
about
thanksgiving
black
friday
and
stuff
and
it
went
off
flawlessly.
It
was
great.
A
A
What
is
the
overall
results?
So
we
think
about
us
starting
to
use
this
with
our
API
becoming
a
cool,
our
core
tool
used
within
our
API
stack.
You
know
the
number
of
resources
and
functions
we
opened
up
in
our
AP
eyes
and
just
growth
in
volume
and
usage
of
our
API.
So
it's
two
thousand
percent
growth.
A
You
know
that
in
turn
allowed
us
to
like
make
those
api's
a
lot
more
attractive
for
internal
and
external
developers
to
say:
okay
cool,
you
guys
are
doing
more
stuff
sweet,
I'm
going
to
use
your
api's
and
then
just
unlocking
things
we
can
do
before.
We
had
to
say
no
to
a
lot
of
feature.
Requests
that
we're
coming
in
for
api
is
because
we
knew
we
just
couldn't
do
it,
but
with
Cassandra.
Now
it's
entirely
possible
right.
A
It
opens
up
a
whole
different
set
of
doors
you
didn't
have
before
and
then
there's
other
stuff
like
you
know,
right
it's
it's
an
it's
something
that,
if
you're
using
automation
to
manage
it,
if
you
got
really
time
to
people
who
are
full
stack,
owning
it
and
everything
else,
and
you
can
do
things
like
a
jalando-
make
a
change
to
the
system.
You
know
in
a
two-week
sprint
and
then
gets
us
a
lot
closer
to
a
continuous
delivery.
So
we
use
we
are
full.
Continuous
integration
aspire
to
continuous
delivery.
That's
hard
right!
I!
A
A
Then
the
other
resulting
cascading
effects
right,
so
you
brought
it
in
and
in
our
case
we
did
more
teams
are
using
it
right.
So
you
know,
we've
got
other
people
here
from
target
who
are
now
using
it
and
other
major
projects
inside
of
target
we're
sharing
those
cookbooks
with
them.
The
lessons
learned
trying
to
share
that
knowledge.
It's
really
important,
you
know
everything.
You
learn
try
and
document
into
we
key
in
house.
A
If
you
have
that
or
whatever
way
you
want
to
share
knowledge,
and
then
it's
opening
up
the
door
to
other
distributed
systems
right,
like
Kafka.
Our
team
uses
that
to
again
you
can
come
talk
to
me
about
that
on
the
side.
These
are
all
distributed
systems
more
of
those
distributed
systems.
If
you
do
it
once,
it
makes
a
lot
easier
to
bring
in
the
other
ones.
So
what's
the
future
look
like
for
us
right
we're
going
users
across
more
AP,
as
it
makes
a
lot
of
sense,
get
rid
of
those
spinning
disks
we
actually
did.
A
I
just
was
reviewing
this
presentation
and
realized
that
when
I
wrote
this
we
actually
did
get
rid
of
those.
We
had
some
left,
but
we're
now
on
all
an
SS
keys,
and
it
makes
a
lot
of
sense
right.
We've
seen
several
talks
here
today
to
talk
about
get
SSDs
like
knock
it
off
quit.
Thinking
about
it,
you
should
definitely
be
using
SSDs
move
to
the
cloud,
so
I
mean
a
lot
of
stuff
actually
are
Cassandra
clusters.
A
All
in-house
makes
a
lot
of
sense
to
go
out
to
the
cloud
in
our
case
we're
looking
at
Google
compute
engine
fairly
attractive.
We've
done
some
tests
out
there
and
things
are
looking
actually
pretty
good.
It
makes
a
lot
of
sense
for
us.
Some
data
could
to
consider
that
there's
more
things
we
can
automate
down
to
the
infrastructure.
You
know
in
our
case,
we
want
to
scale
and
geographically
distribute
our
data.
Centers
into
that
cloud
makes
more.
You
know,
changes
to
them
for
infrastructure,
so
I
mean
there's,
there's
more
to
do
on
that
front.
A
Okay,
I
said
earlier
when
to
that
training
on
the
data,
modeling
really
opens
up
your
eyes
to
like
a
few
key,
more
advanced
things
or
like
wow,
that
data
modeling
stuff
really
matters
I.
You
know
you're,
probably
going
to
stand
your
data
model
up
the
first
time
with
your
set
of
applications.
You
can
make
a
lot
of
mistakes,
just
don't
put
yourself
in
a
corner
where
you
can't
back
out
of
that
easily.
A
Don't
underestimate
the
value
of
the
data
modeling
upfront
that
stuff
really
matters
performance.
It
matters
a
whole
lot
when
you
try
to
connect
the
two
from
data
model
down
to
how
that
data
is
stored,
ones
and
zeros
on
those
machines
like
it
matters
a
lot
to
understand
that
get
away
from
that
bulk
loading
stuff
like
there's,
probably
some
better
things
we
can
do
get
away
from
that
batch
stuff
chances.
Are
you
gonna
have
the
same
thing?
A
Think
of
that
stuff
up
front
if
you
can
gets
rid
of
that
compaction
process,
overhead
stuff
and
then
other
things
you
know
someone
here
earlier
today
talking
about
spark,
that's
great
stuff.
You
know
we
got
to
do
more
of
that
event
based
data
loading
and
stuff,
like
that
we
probably
got
a
fair
use
in
spark
and
Kafka.
So
when
we
talk
about
that
on
the
side,
if
anybody
wants
to
know
more
about
that
stuff
yeah,
what
are
some
crazy
things
like
docker
sounds
really
attractive.
Actually,
it's
entirely
possible.