►
From YouTube: Making Sense of Chaos: Implementing Chaos Engineering in a Fintech... Iqbal Farabi & Giovanni Sakti
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Making Sense of Chaos: Implementing Chaos Engineering in a Fintech Company - Iqbal Farabi & Giovanni Sakti, GoTo Financial
Chaos engineering is defined as “the facilitation of experiments to uncover systemic weaknesses” by The Principles of Chaos Engineering. This is done by building a hypothesis around the behavior of a system and running experiments to vary real-world events. By doing these experiments, we can build confidence on the behavior of a complex system in the face of disruptions. In this talk, we will discuss our experience in implementing chaos engineering principles in GoTo Financial (GTF), one of Indonesia’s biggest fintech startups. As GTF is operating in a heavily regulated industry, we have the obligation to comply with financial regulations. One of those is adhering to certain service level objectives (SLO) for all cloud native infrastructures we maintain. Implementing chaos engineering in such a system should be handled with care. We will delve into both technical aspects of adopting chaos engineering practices and the approaches to roll out such initiatives to the wider organization.
A
Hi
Hello
all
toxic
everyone
Thank
you
very
much
for
attaining
Association
I
think
this
will
be
the
last
Season
of
captured
Us
convert
your
Breath
show
It's
very,
very
glad
to
come
in
person,
Write
Friends,
because
your
students
were
often
but
Everytime
I
think
most
of
our
section
formosae
online,
and
it's
really
glad
to
this
offline.
The
first
time
to
do
it
in
in
your
internet
site
is
very
glad
you
show
me
by
the
introduction
of
my
name,
is
jovani
I'm
Single
from
YouTube
and.
A
Okay,
yeah,
the
products
are
Hai
Abdel
company
that,
while
working
at
right
now
is
good
evening,
How
bout
me
Bimo.
People
like
me
are
actually
news
about
our
other
group.
Which
could
gojek
So
I,
think
Reason.
They
are
always
your
world
with
one
of
the
biggest
Khair
Indonesia
to
form
goto,
so
gojekan
Tokopedia
much
come
goto
end.
A
All
departments
and
financial
division
are
moving
into
its
own
group,
could
go
to
financial
so
right
now,
I'm
representing
go
to
financial,
show
floor
coverings
right
now,
I
now,
ecosystem
about
Today
I
will
focus
more
on
the
payment
solution.
Which
could
go
pay
in
Indonesia
just
to
give
you
some
snapshots
of
what
is
right
now
in
Indonesia
in
Southeast
Asia
in
general,
we
have
a
payment
and
financial
personal
Finance
solution.
B
now
in
Gope
and
you'
re
surfing
offline
online
softlens
ecosystem
is
like
the
payment
using
search
out
online
spmb.
A
Have
some
work
on
premise
exhibition
sharing
this
our
journey
with
the
community
in
the
past
few
years.
There
is
a
text
that
our
team
has
been
doing
with
the
band
for
sample
thing,
one
of
the
very
first
take
the
top
Jatibening
Estate
Westerling
about
our
journey
migrating
to
communities
and
We.
Also
greeted
the
waiting
for
spec
and
wife
also
been
building.
A
In
my
greeting
in
our
own
developer
port
of
the
platform,
we
could
go
PDT
Hai
still
on
going
migration
right
now,
but
our
meeting
told
me
that
you're
going
to
complete
the
design,
SNSD
Junsu
at
Freddy,
later
ordered
complete
migration
to
this
developer.
From
that
we
have
just
seen
as
McQueen
talking
about
this
sound.
A
You
count
more
than
2
Wedding
Dash
on
top
of
open
source
book
review,
someone
to
make
sure
that
people
in
the
organization,
kenek
Rebel
T2i
integrated
with
our
developer
Sexy,
__,
],
fendi.com
Possibility,
think
this
is
the
disease
architecture,
David
Khoirul
info
uninstall
ecosystem
Which,
a
great
Journey
of
Breath.
However,
today,
where
going
to
talk
about
about
the.
B
B
You
can
play
with
Gope
in
a
jkstore
you
can
play
with
so
many
It
means
that
Gope
will
always
come
with
a
lot
of
integration,
sweater
parties
and
This
Means
that
we
have
a
lot
of
microservices
and
components,
access,
database,
engine,
a-pyo,
turkeys,
Ahok,
Kafka,
Kyle,
dear
tons
of
services
and
components
in
our
system.
It
means
that
more
and
more
people
are
getting
involved
in
a
already
developing
and
maintaining
the
system.
B
B
Like
Everyone
version
oli
understand
about
the
system
and
not
a
single
person
can
find
Everything
In,
Your
Head,
so
It
also
means
that
an
Interaction
between
all
the
subsystems
date
with
the
whole
system
is
not
really
very
easy
to
understand
for
even
for
people
who
have
been
there
for
every
long
time.
So.
This
leads
to
a
lot
of
increasingly
novel
incident
was
meant
by
novel
incident
incident
between
Kris
before
so.
There
is
something
the
second
time
around
us
change
is
around
compliance.
What.
Do
you
mean
by
the
fans
around
kweni?
B
We
are
operating
and
heavily
regulated.
Industry
It
means
that
we
need
to
comply
with
multiple
regressions
from
multiple
government
institutions
right
shoulder,
for
there
are
several
things
that
need
to
be
aware
of
one
of
them
being
data
softlens,
Tibet
Byson
by
our
topics
for
today,
the
second
cliff
about
reliability.
Even
in
government
regulations,
there
are
light
certain
amount
of
really
crowded,
like
Across
the
consequences
that
you're
going
to
face
with
regard
to
your
license
to
operate
as
a
financial
company
So
now.
B
Has
been
a
bit
Abit
topic
for
Asha
Hai,
guys
Hasbi
nemen
topic
for
us
in
our
infrastructure
engineering
team
So
what
we
went
with
every
time.
Everything
has
their
own
definition
of
what
They're
right,
because
everything
Defender
on
the
beach
There
is
no
great
real-time
trick.
Diet
is
a
public
organization.
White
Solid
means
It
is
hard
to
prioritize
reliability
because
everyone
defines
the
earlier
on.
The
second
is
making
incidents
happen,
Especially
when
the
incident
involves
very
USB
involving
the
system.
Where
is
a
different
system,
interacts
with
each
other
and
dance
with
the
Growl.
B
The
growing
the
growth
of
the
system
is
getting
Harder
and
Harder
to
Ensure,
hu
klik
service,
yg
sandwich
components
because
a-sshi
pedro.com
penilik
Wikipedia
word.
We
keep
hearing
more
people,
adding
new
team
event
movie
one
time
from
Hell
on
earth,
Fortuner
it's
hard
to
keep
track,
Okay,
Hood
and
Leslie.
Of
course
this
leads
to
getting
Harder
to
keep
track
light.
Who
is
accountable
for
Which
matrix
reliability,
but
over
the
year
like
a
thing,
the
password
mirrors,
so
we
are
working
on
this
image-
is
to
Ensure
that
we
now
have
a
well-defined,
a
science.
B
Webservice
sandwich
components
belong
to
each
time,
even
if
the
move
and
then
Of
course
it's
getting
Easy
for
us
to
Ensure
Who
is
accountable
for
Which.
They
come
tricks,
and
this
is
a
n
Factor
later
when
we
talk
about
engineering
shirts.
So,
however,
despite
of
some
novel
incidents
happening-
and
we
like
the
point
of
view,
free
switch
in
which
a
lot
of
Nepali
lovebird
Sambil
Okay
just
happened,
and
to
the
point
that
our
stakeholders
compete
with
the
leadership
and
the
Seven
Dwarfs
become
one
right
under
with
experienced
before
and
dance
Hi
luck,
liforce,
limiting.
B
Okay
Why!
Do
you
think,
like
an
artlax
incident,
Amin
ya,
install
male
liforce
and
aqli
our
leadership
and
ask
our
stakeholders
and
ask
engines
with
students.
At
this
point
I
Hi
sorry
ethnic
point:
the
complexity
of
power
system
has
exceeded
in
sseedit
any
one
single
person,
understanding
and
the
four
like
detail
proposal.
Wordpress.Com
generic
ablaze
So.
B
B
It
was
crowned
Willy,
duet,
With
All,
the
regulations
on
our
reality.
Kenken
Willy
Willy
heard
her
problems
on
the
intelligent
ing
body,
widito
research
and
now
I
figured
out
that
this
is
from
the
principles
of
each
site.
It
is
more
about
this
speech
about
experimenting
in
order
to
build
confidence
in
our
system
capability
in
the
turbulent
introduction.
So
so
what
it
means,
if
you
got
any
problem
with
this
principle,
to
be
steady,
state
and
various
events
in
the
zoo,
experimentation
production
of
Uang
continuously
automatically
minimize
+
radius.
B
So
whatever
that
is
not
only
about
introducing
Kaos
but
especially
in
a
quiet,
Really
Care,
About,
reliability,
realbyte
matrix
casing
net
is
about
continuous
verification,
Endless
forum
Casey
rosenthal
himself
in
the
in
his
book,
kyyngg
So.
Basically
it
is
eight
continuous
verification
is
about
proactive
experimentation,
source
software
that
you
verify
the
behavior
of
know
that
it's.
B
Hypotheses
about
understanding
about
our
understanding
about
our
system,
because
previously
we
didn't
even
by
check
like
to
check
our
understanding
of
our
own
system
and
after
marriage,
wedding
hypothesis,
and
we
do
experiments,
Slank
iceq,
whether
our
understanding
of
our
system
is
correct
and
than
a
verified
and
whatever
the
reason
of
the
experimentation
with
feedback
into
the
system.
We
do
you
mean
everything
we
waited
on
a
website.
B
A
Hai,
thank
you
very
much
for
sharing
the
church's
mau
introduction,
I.
Think
a
sorry
audio
book,
jayatogel68
so
Anggun
to
stocking
about
the
practices
deadlift
bin,
adopting
infringing
the
dog
of
just
not
completely.
Everything
with
and
We,
also
be
learning
so
much
from
the
community.
Software
and
We
are
always
trying
to
improve
Operation
so
just
about
which
we
have
just
give
away.
A
We
have
migrated
to
issue.
We
have
our
developer
Porto.
We
have
Faith
and
wizardry
himself.
Farm
management
practice
know
what
to
do.
If
you
want
to
do
with
the
first
thing,
like
the
first
principle,
one
two
one
two
actually
Break
Down
completely
the
problem:
The
One
MP3
once
in
my
mind,
Right
Now.
The
most
important
thing
is
to
actually
institutional
and
improve
system.
Badges0
goals.
Time
to
go
is
not
actually
about
breaking
staff.
Introduction
the
Post,
never
The,
Exo
gold
edition,
Village
Twitter,
doing
Doors
Breaking
The
writing
about
how
we
improve
our
system.
A
Reliability
show
best
on
the
principle.
The
First
thing
that
we
do
is
actually
doing
anything
on
production.
The
First
thing
that
were
doing
is
to
our
noncurrent
limits
or
Happy
Little
So.
This
is
the
first
thing
that
we
do
right.
Word
example
of
this
activity
with
by
doing
Discovery
game
From
our
past
post-mortem
prayer,
like
that
we
have
reached
our
incident
management
process.
We
have
a
Collection
of
the
ASEAN
postmortem
available
on
our
platform.
A
Right
Now
show
the
team
has
been
studying
the
fish
mouth
because
there,
because
were
fashion
items
from
simplemente
was
not
actually
dan
completely
Right.
There
cases
were
so
much
by
some
of
the
others
might
not.
It
might
require
change
the
problem
with
that,
the
scene
of
a
items
that
it
might
not
be
recorded
in
indo-like
organizations
or
maybe
only
trouble
by
doing
this
activity
by
revisiting
of
the
past
postmortem
game
on
Steam
SM
organization.
A
We
have
these
simple,
Om,
basically
categorized
items
into
three
things:
right:
Shoulder,
To,
the
right
right.
We
have
senario
which
identifies
Hai.
We
need
me
intervention
to
recover
from
scenarioo
nnt
the
middle.
There
were
not
really
sure
to
the
left
there
that
we
know
for
sure
that
the
problem
is,
if
you
want
to
experiment
of
Which
Which
target
because
I
like
it,
you
know
for
sure
that
it's
broken
white
manual
intervention
in
the.
A
So,
this
is
actually
nextage
right,
because
I
want
to
avoid
the
next,
but
in
the
middle
So.
This
is
actually
when
people
talk
about
engineering.
Actually,
This
is
the
first
thing
that
comes
with
a
Hai
Lex.
After
we
learned
about
ke.
Often
the
conclusion
was
raining
so
much
more
than
the
big,
because
it's
not
just
in
the
middle
of
the
batik
couple,
Torch
steroid.
A
Because
well,
complex-kompleks
Wilkipedia
items
and
office,
we
have
to
keep
moving
everything
should
I
am
yummibear
about
episode,
Bulatin
single-slit
adjustment
to
give
you
what
what
does
it
means
should
be
inspired
from
airworthiness
directive
in
Airline
industry
So?
What
this
is
what
it
means
that,
if
you
haven't
airplane
It
cannot
take
off
before
it
is
completed
all
of
the
workings
directive
same
thing
with
age,
not
all
items
can
be
fixed
immediately.
B
weekend
weekend,
release
notes
and
weekend
adbch
service,
that
all
of
the
service
are
comply
with
this
Advanced
wireless.
A
With
people
and
understanding
of
the
Year
of
the
system
actually
on
with
the
exercise,
d'resort
surprised,
because
it's
people,
understanding
about
the
Kau
ada
system
is
completely
different
from
the
other
right.
Even
for
people
who
are
directly
Working
with
the
system
right,
Dose
people
actually
have
a
different
understanding
with
what
you
have.
A
They,
like
game
deise
like
you,
can
inform
and
educate
people
to
Test
Your
system
by
running
serious
protests,
because
I
want
you
to
do
today,
you
will
have
some
information
In
addition
to
start
doing
at
safeway
between
a
a
British,
successful,
Is
Dead.
We
know
so
good
experiments,
piggybacking
operational
activities
bright,
like
a
running
leno.ru
community
Space
for
lease
and
a
wedding,
continuous,
upgrade
FW,
mencret
and
wedding
upgrade
services.
We
are
in
place
where
shifting
the
world
from
old
from
out
into
the
noted
that
will
Regret
all
the
workload
even
by
piggybacking
activity.
A
A
To
gain
insights
about
your
system
like
flower,
for
example,
life
fell
from
time
to
time.
You
were
curious,
shogen,
Inside,
Out,
Your
system
and
Leslie,
but
office.
The
most
important
autodesign
time
is
to
do
the
experiment,
loading
system,
this
the
girl
of
Engineering
at
Tinggi,
Fadli
Zon.
If
the
money
is
lips
are
good.,
but
the
office
is
Taylor
preparation
before
we
even
get
her
Pride,
yo
yo,
of
course,
site
like
belong
to
evermoving
broadcast
engineering.
It
won't
be
completed
before
we
speak
about
the
tools,
feat,
Sodiq.
A
The
question
is
due
to
limited
used
to
do
all
of
the
things
that
I
mention
site.web,
like
what
kind
of
Help
that
we
can
possibly
get
from
benar
right,
so
diservis
for
items
are
probably
fortnite.io,
feel
pretty
unpretty
Glee
get
out
from
injecting
default
gathering,
The,
Matrix
and
scheduling,
experience
and
storing
all
the
history
to
already
made
revisited
game
show.
The
question
is:
queenpools
help
us,
fortunately,
I
think
in
the
cloud
computing
foundation
community.
A
We
have
already
a
thing
Today
I'm,
going
to
highlight
with
experience
over
the
first
one
Smash.
It
consists
of
several
components:
headboard
made
YouTube
minor
Taklukan
rundes
fermented
has
the
control
Manager
two
Orchestra
the
experiments
and
also
signed
in
stock
FB
set
ini,
chock,
Documents
notch
pt.
nbs,
our
evaluation
of
each
field,
the
requirements
that
I
posted
earlier
it
can
help
us
injecting
Seriously.
It
can
help
expose
handphone
matrices
weekend,
enderso,
the
scheduler
Dedek
Politikus
workflow,
and
we
Black
og
studed
stores.
All
of
your
experience.
It
can.
A
In
the
disk,
Space
chocoberry
interesting
to
see
PS2
slim
or
a
bit
more
complex
architecture,
but
sweet
steam,
it
has
the
guidance
and
interact
with
its
the
engine
and
It
also
has
the
run
without
you,
Randy
experiment
in
the
integrated
media.
One
thing
that
I
want
to
highlight
from
this
thus
It's
that
they
have
the
shop
which
the
ef.co.id
core
contents
experiments
that
you
can
use
from
each
humanity
So.
If
apo11
contributed
to
this
new
chtu
tayo,
you
can
actually
do
so
quiet
and
Rich.
The
possible
experiments
that
you
can
use
in
this
platform.
A
B
Spread
our
Wings
clear
casing:
software
icofx
software
engineering
is
a
way
for
us
to
Menteng
reliability
and
compact
system
sustainably
this
software
and,
of
course,
a
How.
Do
you
do
that
by
doing
continuous
verification
to
our
system,
and
what
will
also
is
that,
if
you're
working
in
a
highly
regulated
industry
in
which
matrix
reliability
cannot
be
genre
dance,
having
a
doubling
coordination
organization,
Which
is
a
prerequisite
Hai.
B
Next
also,
it
is
more
than
just
the
tooling,
like
you
do
to
blackpink
production
at
the
whole
process
and
a
Gio
out
your
about
how
we
go
round
interview
people
to
get
understanding
of
the
system,
how
we
go
round
about
a
Ensure
that
all
the
verification,
ID
Do
it
your
feedback
to
the
system,
Aduh
organization,
by
ensuring
a
proper
documentation
and
but
also
for
studying,
help
you
to
help
us
to
make
our
experience.
Candi,
e&a,
Ultimate
Edition.
B
What
you
want
to
do
is
to
ensure
that
improve
our
institutional
about
the
capability
of
our
complex
system
and
improve
the
ability
of
Hai
about
Retno
a
well.
We
have
tried
out
with
several
tools
like
underwear.
Unfortunately,
the
majority
of
a
verification
system
with
me
is
your
newly,
so
we
would
like
to
go
forward.
We
would
like
to
be
and
immoral
toner
in
continuous
Manner,
of
course,
Ayononton21
Rang2
community
about
like
now.
Officially
your
perfect
wife
is
a
light.
B
Far
from
from
The
Holy
Grail
of
kyyngg
a
year
to
share
what
we
floor
and
Hopefully,
we
can
learn
from
you
as
the
community
as
well.
Oh
yah
This
is
some
references
used
throughout
us,
making
this
presentation
and
offshore
special
thanks
to
hover
our
business
team.
Not
everyone
seems
like
the
entire
team
of
Pain
versus
often
thin,
orderly,
Apple,
ID,
Baqarah
notebook
for
your
step
over
our
Heads
of
payment,
the
boat
in
the
middle
aja,
Don't
Worry
He's
not
match
any
of
our
engineers
in
machine
production,
Gope,
Gope
organization,
Yes
So.