►
From YouTube: Migrating from Packaged Releases to Continuous Delivery and Something In... - Gopinath & Balaji
Description
Migrating from Packaged Releases to Continuous Delivery and Something In Between - Challenges and Solution Patterns - Gopinath Rebala & Balaji Sivasubramanian, OpsMx
Speakers: Balaji Sivasubramanian, Gopinath Rebala
As organizations adopting CD has progressed from early adopters to early majority, there are significant challenges faced in the transition of existing software processes in monoliths and traditional software deliveries. Spinnaker is a powerful continuous delivery platform. This talk discusses common patterns in application architectures and challenges in adopting continuous delivery in terms of process automation. We will present some of the patterns adopted in hybrid environments and discuss pros and cons that of approaches and what worked in our implementation.
For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/
A
B
Thank
you,
balaji
welcome
everyone.
I'm
gonna,
we've
been
part
of
spinnaker
community
for
a
few
years.
Now
and
now
we
are
a
part
of
this
continuous
delivery
foundation.
We
are
excited
to
be
part
of
it
and,
as
balaji
said,
we
work
on
intelligent
delivery
solutions
and
today
we'll
go
through
some
of
the
projects
that
we
have
done
and
how
we
have
implemented
them.
A
Excellent,
thank
you.
So
here's
our
agenda.
I
will
briefly
touch
upon
our.
You
know
the
state
of
kenya's
delivery.
As
you
see
it
and
then
I'll
talk
about
some
of
the
challenges
with
the
package
delivery
process
today
and
then
gopi
would
go
through
a
few
use
cases
real
world
customer
scenarios
where
we
essentially
solved
some
of
these
challenges,
and
I
think
that
you
you,
you
know
if
you're
still
doing
some
of
the
monoliths
and
infrastructure
updates
as
part
of
your
as
part
of
your
software
process,
you
you
definitely
will
benefit
from
it.
A
So,
in
terms
of
software
delivery
itself,
I
think
the
way
we
look
at
it
is
that
the
software
development
has
been
transformed.
People
are
adopting
microservices,
more
and
more,
not
completely,
but
at
least
definitely
adopting
you
know
by
majority
of
majority
of
the
people,
and
software
process
is
faster
now,
with
with
the
adoption
of
agile
process
with
the
adoption
of
stateless
services,
whether
through
whether
my
containers
or
microsoft
or
functions,
there's
been
a
lot
of
progress
in
that
space.
A
The
the
on
the
other
side
of
the
it
side
are
the
infrastructure
side.
That's
also
been
transformed.
Now
you
can
bring
up.
You
know
hundreds
of
containers
or
the
infrastructure
in
seconds
right
compared
to
you
know
previously,
where
you
have
to
spend
weeks
to
do
that,
but
what's
really
still
not
solved
essentially
is
the
delivery
you
know
taking
that
artifacts
and
and
and
putting
it
in
the
production
servers.
A
You
know
the
need
to
do
it
at
speed
also
has
increased
as
businesses.
Innovations
are
requiring
them
to
make
many
updates
quicker
from
maybe
hundreds
of
releases
day
a
year
to
like
thousands
or
ten
thousands
of
releases
a
year
multiple
times
a
day,
for
example,
and
those
things
are
or
or
may
are
making
the
existing
process
difficult.
So
what
is
challenging
with
the
existing
process?
A
If
you
look
at
a
typical
cd
pipeline,
you
know
you
have
the
check-ins,
you
do
the
testing,
you
do,
the
staging,
you
do
a
production,
rollout
and
then
maybe
roll
back
if
needed,
the
current
process,
if
you're
not
using
something
like
spinnaker,
for
example,
the
orchestrating
this
whole
pipeline.
You
know
in
terms
of
we're
automating
the
workflow,
making
sure
that
you
can
deploy
to
multiple
clouds
be
able
to
quickly
roll
back
itself
is
a
challenge.
So
that's
first
part
of
the
problem.
Let's
assume
you
have
you're
actually
adopting
some
of
these
technologies.
A
The
other
thing
is
like:
let's
say
you
wanna,
you,
everything
pass
everything
from
a
security
check
passes,
but
what
about
the
actual
software
itself
in
terms
of
the
software
validated?
You
know
whether
it
does
any
performance
issues
or
other
kinds
of
issues.
So
you
need
to
be
able
to
verify
the
software
typically,
you
do
it
in
a
candidate
deployment
and
you
try
to
figure
out
if
the
new
version
is
good
or
not,
and
then
you
have
to
diagnose
this,
this
could
take
hours
as
well.
And
finally,
you
know
you
have
to
have
the
audit.
A
If
some
cases
you
need
to
do
audit
or
any
other
kinds
of
activities,
and
that
could
also
take
a
lot
of
time.
Let's
say
you
do
find
an
issue
like
this:
maybe
a
solarwinds
vulnerability
in
in
one
of
the
software
packages.
How
do
you
trace
back
everything
that
happened
and
make
sure
that
no
other
software
also
actually
has
the
same
package,
for
example?
So
these
are
different
challenges.
This
is
a
this
set
of
issues.
A
A
So
if
you
look
at
different
kinds
of
applications
that
you
see
in
a
typical
enterprise,
you
know
you
obviously
have
the
modern
applications
which
we
talked
about
with
microservices
or
functions,
but
they
also
you
have
other
kinds
of
applications,
traditional
application,
which
is
essentially
monoliths,
and
also
you
have
things
like
you
know
you
want
to
do
infrastructure
as
update
I
mean
that's.
People
are
maintaining
that
as
a
as
a
package
and
they
want
to
do
infrastructure
updates.
Maybe
you
want
to
do
a
schema.
You
know
data
schema
changes
and
databases
or
big
data
warehouses.
A
A
A
So,
for
example,
you're
making
some
changes
to
infrastructure,
but
then
that
may
have
some
implication
to
the
applications
that
you're
running
on.
So
when
you
make
these
changes,
you
have
to
make
sure
that
the
application,
performance
or
security
concerns
are
not
changed,
but
the
infrastructure
changes
you're
introducing.
So
this
thing
has
to
be
tested
together
and
the
last
one
is
the
database
updates
or
schema
updates.
A
So
you
have
obviously
application
changing,
but
also
you
may
have
to
change
your
database
schemas
in
the
backend
or
maybe
the
you
know,
the
warehouse
database
warehouse
things,
big
data
data
updates,
so
you
need
to
do
that
also
as
a
process
instead
of
doing
ad
hoc.
If
you
do
it
separately,
you
know
you're
going
to
have
situations
where
you
have
some
discrepancies
in
terms
of
these
things
happening
out
of
sync,
and
so
these
things,
what
we
call
as
package
releases.
A
How
do
you
ensure
these
monoliths,
this
infrastructure,
either
with
the
application
or
without
it
or
the
big
data
schema
changes
with
the
application
goes
smoothly
as
a
micro
service
application
right,
everybody
knows
how
how
it
works
for
a
kubernetes
app.
But
what
about
these
right
and
that's
the
goal
of
this
session,
so
some
of
the
high
risk
some
of
the
risk
that
these
these
introduce.
A
Essentially,
these
has
some
challenges.
Obviously,
with
these
releases,
one
is
the
inability
to
test
small
changes
in
isolation.
For
example,
you
have
multiple
components.
How
do
you
test
one
component
change
in
isolation?
You
need
to
basically
package
it
all
together
and
test
it,
and
that
may
that'll
be
difficult
and
also,
if
you
have,
if
you
have
to
need
to
roll
back,
you
need
to
draw
back
a
lot
of
set
of
changes,
because
these
are
all
a
bunch
of
changes.
A
A
B
Some
of
the
common
problems
are
these
integration
tests,
because
these
are
not
knowledge.
Large
number
of
components
coming
together.
The
tests
themselves
take
a
very
long
time.
It's
not
uncommon
to
take
for
the
integration
test
for
hours.
Let's
say
they
take
10
hours,
and
once
we
find
a
problem
in
eight
hours,
you
have
to
restart
the
test
once
that
is
fixed,
but
then
finding
the
fix
itself
is
an
issue
because
there
is
so
many
different
components
are
coming
together.
The
experts
in
each
of
the
systems
are
different.
B
How
do
you
triage
and
quickly
recover
from
the
problems?
So
these
are
some
of
the
things
that
microservices
solve
much
better,
but
we
look
at
some
of
the
challenges
that
we
face
and
how
we
address
them.
The
other
one
that's
getting
more
common
is
infrastructure
changes
themselves
becoming
part
of
the
application
now,
with
amazon,
aws
services,
azure
gcp,
providing
so
many
different
services
with
the
databases,
security
and
messaging
services.
B
The
applications
taking
advantage
of
them
are
now
using
so
many
of
the
services
provided
by
cloud
systems
and
any
changes
that
needs
to
be
made
to
the
application
security
application
performance
touch.
Many
of
these
things
in
the
infrastructure
as
a
code
is
becoming
a
vehicle
to
do
that
and
troubleshooting.
This
is
similar
or
delivering
things
faster
is
similar
to
the
problems
or
the
monoliths
that
yes,
so
these
are
some
of
the
things
that
we
will
look
at
today
to
break
it
down.
B
We
will
look
at
three
specific
use
cases.
One
is
a
delivery
system
that
is
implemented
at
a
large
semiconductor
manufacturer.
Who
has
this
big
package
system
that
they
deliver
for
emulating
the
hardware
systems?
The
other
one
is
the
sas
vendor
is
completely
opposite
of
the
semiconductor
manufacturer.
B
Is
they
have
this
infrastructure
built
on
aws
using
bunch
of
services
from
aws
and
then
applications
sitting
on
top
delivering?
These
changes
continuously
to
different
tenants
sitting
on
that
sas
platform
and
the
last
one
we
will
look
at
is
how
the
database
changes.
Snowflake
is
a
sas
application
by
itself,
but
there
are
applications
that
are
being
developed
on
top
of
snowflake.
How
do
you
manage
in
a
compliant
manner,
changes
that
are
being
delivered
to
the
databases
and
changes
to
the
applications?
B
B
The
challenge
here
is
that
there
is
a
significantly
advanced
cii
system,
so
hundreds
of
developers
developing
code,
multiple
components,
they're
all
coming
together,
but
they
still
have
this
manual
scripts
that
are
deploying
the
components
that
are
developed
and
then
testing
those
the
tests
themselves
run
12
hours
a
complete
integration
test
before
they
can
call
it
a
release
and
any
changes
like
we
discussed
problems
that
are
found.
They
need
to
rerun
the
entire
test
suite
and
the
diagnosis
itself
is
very
complex.
B
B
So
the
way
we
approached
it
is
there
is
a
ci
system
well-built
that
has
parallel
pipelines.
So
the
bottlenecks
are
in
deploying
the
system
with
new
components
that
are
built
on
a
continuous
basis
and
improving
the
test
process
itself.
The
instead
of
running
all
the
tests,
one
after
another
break
it
down
into
parallel
systems.
B
You
could
run
these
tests
in
parallel,
but
before
that
you
also
want
to
break
it
down
as
a
smaller
set
of
tests
that
allow
you
to
test
each
build
for
its
validity
and
then
integration
tests
you
can
run
in
parallel
and
the
third
one
that
we
have
done.
Is
this
intelligence
part
where
you,
you
implement
a
automated
analysis
that
would
associate
some
probability
to
the
components
that
are
there
in
the
system
and
then
run
an
automated
triage.
B
So
what
does
that
mean?
The
two
things
that
we
are
doing
here
right,
one,
is
build
the
pipeline
with
spinnaker
that
allows
us
to
do
automated
runs
with
a
rollback.
So
previously
it
was
and
horse
crips
that
were
deploying
the
system.
There
was
no
real
way
to
roll
back
a
component
and
that
we
fixed
by
using
containerized
platform.
Although
it's
not
a
micro
service,
you
could
still
build
a
container.
You
could
roll
it
back
and
by
using
the
automated
analysis,
you
could
identify
the
probability
of
these
components.
Now
you
can
run.
B
Let's
say
there
are
40
components
in
the
system,
identify
five
to
six
components
that
are
the
high
probability
for
causing
the
problem.
You
run
those
five
in
parallel
with
the
same
test
suite
with
the
shorter
term,
and
it
automatically
gives
you
the
version
of
the
component
that
could
be
causing
the
problem
and
it
improved
the
triage
process
tremendously
and
it's
the
other
thing
that's
benefit
that
came
out.
Is
this
collaboration
between
the
teams?
B
Now
we
are
able
to
see
the
pipelines
with
spinnaker
that
are
running
in
parallel,
that
give
you
what
versions
of
the
components
are
being
deployed
and
tested
and
where
the
test
is
failing
and
that
could
be
shared
between
groups
much
more
easily
and
the
logs
are
much
more
easily
shared.
It
really
improved
their
performance
on
this
triage
process
reduced
the
amount
of
time
they
need
to
try
out
and
over
time.
It
will
also
reduce
the
amount
of
errors
that
you
will
see,
because
these
things
could
be
automated.
B
So
the
platform
itself
is
being
deployed
infrastructure
as
code
using
terraform.
The
changes
are
not
synchronized
any
issue
that
shows
up
that
triage
process
is
again
very
involved
and
painful.
It
takes
a
tremendous
amount
of
time
from
different
groups,
because
the
configuration
changes
that
are
required
by
the
application,
if
that's
not
properly
tracked,
it
can
take
days
to
figure
out
what
the
issue
is
and
clean
up.
B
As
you
know,
the
terraform
structure
also
changed
between
10
to
12
12
to
14,
and
the
stage
management
with
the
terraform
itself
needs
to
be
properly
understood.
The
life
cycle
management
of
that
needs
to
be
managed.
So
here
we
looked
into
where
the
real
bottlenecks
are.
The
changes
need
to
be
tracked
vertically.
The
changes
that
are
made
to
the
infrastructure
as
a
code,
the
terraform
needs
to
be
matched
with
the
changes
that
are
happening
at
the
application
level
with
containers
and
together
they
need
to
be
tested.
So
we
built
these
pipelines
with
spinnaker.
B
As
the
change
happens
in
the
kit,
we
actually
run
a
parallel
pipelines
to
verify
the
application
functionality
on
the
new
infrastructure
that
is
built,
upgraded
infrastructure
and
with
the
new
uplink
new
application
code
on
top
of
it,
let's
call
them
tenants,
and
only-
and
these
things
are
successful,
we
would
allow
them
to
go
into
the
main
branch
and
get
and
then
anytime
any
given
time.
The
main
branch
is
ready
to
be
released.
B
B
Addressed
because
these
changes
are
small
and
easily
caught
so
this
again,
because
the
ability
of
spinnaker
to
provide
these
pipeline
views
and
the
testing
there
is
an
improved
collaboration
between
the
groups
and
the
overall
process
systematically
improved
by
addressing
a
bottleneck
one
at
a
time
and
the
last
one.
Is
this
a
1450
technology
vendor?
B
B
Here
the
goal
is
that
the
changes
that
are
being
made
to
the
snowflake
is
ad
hoc
scripts
and
there
is
no
process
the
ability
to
control
this
git
process
moving
from
one
environment
to
another,
and
these
are
also
custom
scripts
that
are
not
auditable
properly.
So
there's
there's
a
compliance
risk
that
was
involved
in
making
these
kind
of
changes.
B
So
you
have
these
custom
stages
that
we
build
in
that
allow
for
the
servings
actually
kind
of
approval,
and
because
these
pipelines
are
visible
and
auditable,
you
have
the
compliance
requirements
that
come
in
and
the
pr
merge
process
that
move
from
one
environment
to
another
for
the
scripts
is
also
implemented
as
part
of
this
process
and
at
the
end,
you
could
also
do
a
roll
back
of
the
schema
with
the
pipeline.
So
this
allowed
us
to
provide
the
ability
to
do
a
visible
compliant
pipelines
that
allowed
for
their
governance
policies.
B
So
we
went
through
some
of
these
use
cases
here
as
a
core
of
it.
As
you
can
see,
the
challenges
do
exist,
but
by
using
spinnaker
and
ability
to
construct
these
pipelines
identifying
the
bottlenecks
and
addressing
them
with
the
parallel
pipelines
having
an
intelligence
built
in
a
little
bit
in
terms
of
automated
triage,
we
were
able
to
improve
the
process
tremendously.
B
B
A
A
yeah,
no,
absolutely
thank
you.
Thank
you
for
walking
through
those
sort
of
three
use
cases.
Just
to
summarize,
I
think
we
leveraged
spin
echo
and
helped
customers
with
the
adoption
of
many
of
these
use
cases
and
there's
also
some
enterprise
features
that
we
added
to
enable
that
happen,
and
also
you
look
at
some
other
use
cases
in
order
for
it
to
be
a
fully
continuous
delivery
process.
You
need
to
have
these
intelligence
problems.
We
talked
about
earlier
ability
to
automate
approvals,
able
to
audit
every
verification.
A
Every
do
traceability,
all
of
those
things
are
built
in
and
that's
what
makes
it
a
very
robust
solution
so
that
you
can
actually
feel
like
you
know
you
are
not
you're
not
slowed
down
with
these
other
applications.
Non-Micro
services
applications
and
our
our
customers
are
able
to
kind
of
take
both
of
the
you
know,
both
microservices,
as
well
as
the
traditional
applications
and
do
a
good
job
of
it.