►
Description
Prometheus Is Good for Your Small Startup - Ignacio P. Carretero, ShuttleCloud
"ShuttleCloud is a small startup specialized in email and contacts migrations. We developed a reliable migration platform in high availability used by clients like Gmail, Gcontacts and Comcast. For example, Gmail alone has imported data for 3 million users with our API and we process hundreds of terabytes every month.
In this talk we'll explain our journey from having near-zero monitoring to having all of our infrastructure monitored with the necessary metrics and alerts. We will share with the audience some of the mistakes we did and what lessons we have learned. We currently have around 200 instances monitored with a comfortable cost-effective in-house monitoring stack based on Prometheus.
We want to demonstrate that you don't need to have a big fleet to embrace Prometheus and that it is a non-expensive solution for monitoring."
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Hi
everyone:
how
are
you
doing
cool
so
primate
is
good
for
your
small
shut
up
who
we
are
and
why
well?
My
name
is
natural
well
in
Matthew,
my
real
name,
but
if
you
happen
to
know
that
in
Oleg
matters
were
called
natural
yeah
like
the
Mexican
food
I
for
that
joke
before
you
can
contact
me
at
that
treated
so
carrot
tops
and
you
should
definitely
follow
shelter
and
shelter,
engineering,
especially
engineering,
because
we
usually
talk
about
some
technical
stuff,
and
if
you
want
to
participate
you
you
are
free
to
do
that.
A
A
We
are
based
in
Chicago
in
Madrid,
and
we
basically
offer
an
API
for
email
and
contacts
import,
who
are
our
customers,
usually
ISPs
and
email
and
address
book
providers.
Those
are
three
examples
because
they
are
very
I've
chosen
these
those
three
because
they
are
very
different
from
each
other.
So
Google
is
one
of
our
customers.
Stanford
University
can't
gasps.
We
have
a
you,
are
curious
about
it
and,
or
you
don't
believe
it
you
can
go
to.
Jimmy,
have
a
gmail
account
to
Settings
accounts,
an
import
and
the
people.
A
That's
importing
your
email
and
contacts
behind
Google.
It's
us
so
some
facts
about
us,
yeah,
yeah
I'm,
you
can,
you
read
it
I
hope
so.
So,
thanks
to
us,
gmail
has
already
imported
three
million
users
using
our
API.
We
are.
We
have
some
as
layers
of
high
availability
around
99.5
percent
and
we
comply
with
them
still,
hopefully,
we'll
keep
complying
with
them.
We
had
we've
had
some
peaks
of
six
terabytes
per
our
data
migrated,
usually
some
numbers
what
we
do
around
18,000
migrations
per
day,
around
30
million
emails
per
day.
A
First
of
all,
everything
I
explained
we're
only
six
people
in
the
engineering
team.
Actually,
there
are
more
people
in
this
room
that
there
are
in
my
company.
So
six
right
now
we
are
seven
dozen
one
more
so
I
want
to
thank
to
all
of
them
because
we
don't
have
any
first
thing.
I
want
to
explain
that.
Maybe
you
can
resonate
with
that
in
your
companies
that
we
are
you
don't
have
a
monitoring
department.
We
don't
have
proper
people
dead,
proper
Department
dedicated
to
do
that.
A
So
all
of
us,
each
one
of
us,
is
helping
or
trying
to
get
our
hands
dirty
with
it.
So
yeah
second
thing,
obviously
I'm
not
here
to
tell
you
how
to
do
things
and
don't
expect
the
perfect
model
on
how
you
should
implement
Prometheus
guys
have
not
the
one.
That's
gonna
tell
you
that
and
third,
a
lot
of
putting
memes
everywhere
so
I'm,
sorry
about
that
before
Prometheus.
First,
as
you
can
imagine
a
start-up,
we
didn't
have
as
many
project
circles.
A
We
have
right
now
we
didn't
have
as
many
costumers
and,
to
be
honest,
our
first
goal
was
to
grow
and
not
to
monitor.
Even
though
we
had
some
things
in
place,
not
perfect,
as
you
might
see
first
thing
we
had
some
automatic
scripts
that
actually
monitored
all
of
our
our
operation.
Metrics
such
as
yeah,
this
hard
drives
getting
full
I
know
there
are
a
lot
of
things
wrong
with
that
like
how
do
you
get
that
email
if
an
instance
is
not
available
or
it
sound
I
know
but
yeah?
A
Somehow
it
worked
for
some
of
the
things
that
we're
practically
monitoring
most
of
those
things,
but
we
have
those
emails
for
api's.
We
had
external
service,
we
still
have
it
by
the
way,
just
in
case
and
which
is
pinged
and
it
it
sends
pings
from
all
over
the
world
to
your
API
sign.
You
receive
beautiful
alerts
if
you're,
not
up,
which
is
nice
and
for
all
our
business
metrics.
We
trusted
in
that
dashboard
and
we
had
some
daily
and
monthly
and
weekly
reports
of
what
we're
doing
like
hey.
Yesterday,
you
migrate
for
this
project.
A
I
know
whatever
number
for
migrations
from
this
provider
from
this
provider,
and
things
like
that.
So,
as
you
can
imagine,
we
actually
did
a
lot
of
things
manually
like
we
reviewed
those
emails.
We
check
that
dashboard.
That's
not
that
doesn't
scale.
I
want
to
share
with
you
a
little
bit
how
it's
very
simplistic
model
about
how
we
got
those
metrics,
because
this
is
going
to
explain
afterwards.
Some
of
the
decisions
we
made
every
single
migration
will
run.
It
saves
us
status
in
that
status
database.
It's
a
non
relational
data
model
does
matter
in
our
case.
A
Couchdb
and
we
have
sort
of
a
system
that
processes
those
statuses
in
a
relational
model
like
hey
from
this,
that
in
the
status
database,
you
receive
all
services,
and
this
one
transforms
that
to
this
migration
from
this
provider
finished
with
that
set
that
status
at
that
time.
Great,
and
with
all
these
data
points,
we
had
another
system
that
a
view
controller
model
system,
that
the
controller
sort
of
processes
that
information
and
different
views,
one
to
push
the
data
to
the
dashboard.
A
Another
one
said
the
weekly
daily
monthly
emails
and
yeah
I
mean
this
is
the
kind
of
thing
when
you
talk
to
other
startups,
like
the
things
that
you
don't
want
to
share
like
we
don't
have
a
monitoring
system
in
place
for
everything,
but
we
realize
that
and
that
didn't
scale.
So
we
actually,
we
were
looking
for
and
monitoring
solution
and
one
day
it
was.
A
It
was
not
our
priority
before,
but
it
was
at
a
given
point
like
it
is
right
now
the
priority
so
I'm
gonna
tell
you
why
we
chose
Prometheus,
not
that
one
but
the
other
one,
and
because
this
is
usually
the
first
question
that
someone
can
do.
We
got
asked
like
why
you
chose
Prometheus
I'd
love,
to
tell
you
a
story
that
someone
recommended
promise
to
us,
but
it
was
actually
the
other
way
around
we've
got
sensor
was
a
versa.
A
We
asked
on
the
dev
ops
community
in
Spain
and
the
majority
of
people
recommended
sensor,
but
we
decided
to
do
that
in
a
in
an
agile
way.
So
we
wanted
to
try
like
let's
spend
a
couple
of
threatt
iterations
and
try
one
system,
and
if
it
doesn't
work,
let's
try
the
next
one
to
do
so.
We
first
analyzed
what
which,
which
one
is
gonna
be
the
first.
A
So
we
chose
primitives
to
be
the
first
one
first,
because
something
very
beginning
I
did
we
didn't
have
a
lot
of
idea
about
how
or
they
how
our
metrics
should
look
like
one
of
the
things
that
you
can
do
with
Prometheus.
You
can
you?
Don't
have
to
have
that
metric
model
decided
from
the
very
beginning,
so
you
can
add.
If
you
happen
to
know
and
I
guess
you
all
know,
you
have
labels,
so
you
can
add
those
labels
for
your
time
series
and
have
different
time
series
for
that.
You
can
remove
them
afterwards.
A
You
can
create
new
metrics.
So
that
was
a
very
good
point
for
us,
because
we
didn't
want
to
spend
a
lot
of
time
deciding
what
we
want
to
have
time
afterwards.
Then,
to
do
that
test
that
small
test,
you
don't
need
any
external
services
with
Prometheus,
for
example,
with
sensor.
You
need
our
messaging
systems,
such
as
radical
queue.
A
We
also
happen
to
have
at
that
time
for
a
different
story
and
internal
DNS
server
and
primitives
allowed
that
service
discovery
using
your
internal,
our
internal
DNS,
so
DNA's
will,
which
was
nice
and
it's
a
go
file,
so
it
was
very
easy
to
sound
test.
So
we
give
it
a
try,
and
this
is
the
first
things
we
did
sort
of
an
experiment
like
first
we
position
all
of
our
machines
using
Cybil,
so
it
was
very
easy
to
us
for
us
to
adapt.
A
One
playbook
to
manually
feed
all
the
in
front,
our
ansel
inventory
to
Prometheus
all
the
services
that
had
to
be
monitored.
Then
we
change
to
that
DNS
service
discovery
as
explained,
and
just
if
you
just
use
Prometheus
with
the
node
exporter,
you
happen
to
have
most
of
the
operational
metrics
we
had
before
just
by
default
and
the
ones
that
we
didn't
have
I'm.
Sorry,
because
you
cannot
read
anything
at
all.
A
We
use
the
ders
I'm,
not
telling
that
you
should
use
it,
but
we
did
sort
of
a
yeah.
Let's
in
something
quick,
we
use
the
text
file
in
the
Nordic
sport
that
you
can.
You
can
create
a
text
file
with
all
your
metric
metric
results
and
parameters
can
pull
the
data
from
it.
It's
not
perfect,
but
there
are
some
things
that
we
still
don't
know.
How
to
do.
A
I
will
explain
that
later,
like
it's
important
for
us
to
monitor
the
status
of
the
replications
in
CouchDB,
so
some
custom
things
that
we
need
to
be
sure
that
are
working
yeah,
some
of
the
first
alerts
for
this
experiment.
We
only
have
operate.
We
only
have
operation
metrics,
so
we
only
had
operation
alert,
obviously
so
hard
drive
usage,
for
example,
if
an
instance
are
up
and
down,
and
at
that
time
we
only
use
absolute
thresholds.
Yeah
the
hard
drive
reaches
85%
of
capacities
and
as
an
email,
90%
page
someone-
and
this
comes
by
default.
A
Like
the
node
exporter
world,
not
the
alerts
got
to
set
the
threshold
yourself,
but
the
metrics.
Just
by
installing
the
node
exporter,
you
have
all
those
metrics
available,
it's
something
so
at
that
moment
we
realized
that
it
was
quite
easy
and
easier
than
what
we
expected.
So
we
kept
on
working
on
Prometheus
I'd
love
to
tell
your
story
like
yeah.
Then
we
tried
another
system
and
then
another,
but
no
we
only
try
actually
Prometheus
and
we
get
with
it.
So
that's
it,
and
if
we
go
back,
if
you
remember,
I
explained
you
that
system.
A
So
once
we
have
previously
in
place,
we
need
the
business
metrics
where
we
have
there
up
metrics
the
business
metrics.
Well,
we
have
those
two
publishers
that
are
be
used
from
that
system.
So
it
was
pretty
straightforward
to
create
another
publisher
that
pushes
data
to
Prometheus
I,
said
push
and
not
pou,
and
that's
one
of
the
first
things
that
you
shouldn't
do
don't
do
that,
but
we
did
goes.
It
was
very
easy.
A
That's
why
I
wanted
to
explain
to
you
what
we
had,
because
it
was
very
pretty
straightforward
to
create
another
publisher
to
just
push
the
data
to
Prometheus.
It
happened.
There
happened
to
be
another
component,
which
is
the
bush
gateway
which
shouldn't
be
used
for
that,
but
in
case
you
have
one
a
village.
It
cases
where
you
should
push
data
to
Prometheus.
Instead
of
pooling
you
can
use
that.
But
it's
not.
It
was
not
our
case,
but
yeah.
A
We
used
it
as
a
metric
cache
so
that
readers
can
could
pull
the
data
from
it,
the
metrics
how
wrong
we
were
with
them
like
we
chose
three
metrics
for
the
business
metrics
incoming
operations.
Last
ten
minutes
incoming
statuses.
Last
ten
minutes
yeah
we
had
more
about
those
of
the
key
three
ones
on
migrations.
Last
five
minutes
we
chose
gauges
and
we
did
the
aggregation
in
the
metrics.
So
in
the
exporter,
in
this
case
a
publisher,
and
we
were
wrong
if
you
have
to
bring
some
one
IP
idea
from
this
talk.
A
Don't
do
that
so
really
consider
if
you
want
to
do
gauges
where
there
shouldn't
be
any
cages
and
some
of
the
unexpected
consequences-
and
this
is
a
real
story,
if
you
get,
you
cannot
see
that
I'm.
Sorry,
so
maybe
I
don't
really
know
yes,
sir.
Oh
you
get
there,
but
there
are
too
many.
Maybe
sorry
about
that.
B
A
Problem.
I'm
sorry,
so
this
happened
to
someone
someone
got
Bates
at
3
a.m.
in
the
morning
because
incoming
migrations
fraud
for
one
provider
we're
not
coming
so
that
was
critical.
He
woke
up.
He
thought
he
fixed
it,
but-
and
he
indeed
fixed
it,
but
the
alert
was
triggered
again
and
kept
trigger.
Why?
Because
if
you
see
the
second
the
one
below
if
operations
resumed-
and
you
are
aggregating
in
a
time
window
until
you
get
to
that
threshold-
imagine
it's
80%
in
the
last
10
minutes.
Imagine
it's
not
10
minutes
about
two
hours.
A
If
in
two
hours
you
do
not,
you
do
not
reach
to
80%.
It's
gonna
kick
so
that
you're
gonna
get
that
alert
triggered.
So
maybe
you
have
to
wait
an
hour
and
a
half
with
the
problem
already
solved
until
that
alert
is
not
trigger
anymore
and
at
3
a.m.
in
the
morning.
It
was
not
something
he
was
thinking
about
and
he
freaked
out
and
it's
understandable.
So
that's
one
of
the
unexpected
consequences
that
you
shouldn't
yeah
yeah.
A
So
if
you
happen
to
have
because
how
our
system
was
designed,
if
you
have
two
parameters-
publishers,
the
ones
we
design
pushing
data
to
the
push
gateway,
same
data
or
replicas
of
the
data,
we
got
wrong
information
and
it
was
not
because
it
was
that
everything
we
designed
at
that
time
was
not
right
because,
as
you
can
see,
publisher
code
is
not
really
tied.
To
instance
of
itself,
as
the
operations
were,
narrative
were
aggregating
data,
blah
blah
blah.
A
A
This
got
too
complicated
and
I've
got
that's
when
we
decided
to
do
a
revamp
and
another
iteration
like
a
yeah
using
legacy
systems
was
not
a
good
idea,
so
we
decided
to
start
it
from
scratch.
If
you
remember,
we
had
that
again
and
if
you
analyze
this
chart,
you
can
see
that
if
primitives
monitoring
the
status
of
your
system
and
the
data
incident
that
comes
to
be,
there
are
one
two.
Three
four
systems
in
between
that
can
fail
and
you
can
lose
your
data.
A
So
why
don't
we
create
an
exporters
of
the
Prometheus
way
that
it
should
be
that
pulls
the
data
from
doubt?
Couchdb
and
Prometheus
can
access
it,
and
this
is
what
we
did.
So
we
created
an
operation
exporter
that
was
in
a
standalone
app
and
not
a
chrome
based
script
that
we
had
before
and
we
change.
Finally,
the
metrics
to
be
counters.
We
trusted
the
aggregation
in
Prometheus,
which
is
much
better
than
do
the
aggregation
yourself.
A
If
one
counter
is
one
thousand
one
and
the
other
one
thousand
a
two,
probably
the
one
thousand
and
two
has
better
information
or
more
up-to-date,
so
pick
the
max
of
them.
The
metrics
finally
counters
operation
requests
total
operation,
statuses,
total
and
operational
errors.
Total
work
smooth.
You
can
do
the
aggregation
in
Prometheus,
fine,
the
alert
we
I
told
you
before
that
most
of
the
others
we
did
on
the
bar
beginning
where
absolute
thresholds,
I
I,
don't
really
know
what
I
was
expecting,
because
you
cannot
read
anything
I'm,
very
optimistic,
but
there
are
two
alerts.
A
The
first
one
just
sets
a
threshold
like
if
capacity
of
that
hard
drive
is
over
90%
sent
an
email,
and
we
don't
do
that
anymore.
There's
some
magic
sugar
function
predict
linear,
where
you
can
tell
hey.
If
this
hard
drive
with
a
current
rate,
it's
gonna
get
full
in
two
days.
Send
me
an
email,
so
you
get
an
email
to
this
before.
Well,
it's
an
estimation,
but
you
get
an
email
two
years
before,
like
with
that
way.
A
This
hard
drive
is
gonna,
get
full
in
two
days,
so
you
have
enough
time
to
handle
that
situation
and
maybe
look
for
the
culprit,
who
put
that
verbose
flag
on
just
kidding,
we're
right
now
in
the
alert
manager
0.5.
So
this
is
obviously
obsolete,
but
I
want
to
tell
you
that
we
get
with
one
older
version
and
suddenly
we
update
it
and
I
want
to
show
with
this
slide,
that
there
are
a
lot
of
changes
being
done,
good
ones
before
alerts
where
a
condition
based.
A
It
was
a
condition
based
list
right
now,
a
tree,
so
they
get
grouped.
You
get
only
one
if
you
have
from
a
same
project
cluster
five
instances
down.
You
only
get
one
email
saying
these
five
instances.
Instead
of
having
five
different
alerts.
Imagine
you
have
2,000
instances,
yeah,
so
very
cool
stuff.
The
page
review
integration
works
perfectly
and
will
really
trust
on
it.
So
we
don't
do
as
many
manual
operations
as
we
did
on
the
per
beginning
and
that's
the
truth
like
we
really
trust
our
monitoring
system
and
alert
system
and
I.
A
Don't
really
I
know
that
everything
is
working
if
I
don't
receive
any
email
or
I
get
paged,
which
was
not
really
the
case
before.
Is
that
what
we
have
right
now?
Not
really
because
if
you
remember
from
the
very
beginning,
we
had
a
third
party
for
the
dashboard
and
all
that
complex
system
for
the
dash
world,
where
fana
has
an
awesome
integration
with
Prometheus
and
it's
very
straightforward.
So
we
got
rid
of
that
and
it's
literally
a
matter
of
hours
to
set
it
up
even
less
and
set
it
up
with
all
the
charts.
A
I
mean
everything
and
you
get
a
fancy
dashboard
with
everything
you
want.
Another
good,
very
cool
stuff.
Is
that
all
the
charts
you
can
the
data
you
have
in
the
charts.
It
uses
exactly
the
same
prom
ql
language
that
you
use
in
Prometheus,
so
you
don't
have
to
learn
something
different,
just
copy
you
try
and
Prometheus
copy
and
paste,
and
you
get
the
charts
in
Ratana,
very
straightforward.
I
didn't
get
that
sorry
graph,
Hannah
yeah.
A
Anything
else
missing
if
you
remember:
I
talked
about
Pingdom,
so
black
box
monitoring
black
box
monitoring,
we
didn't
have
anything
internal
from
us
and
it's
very
easy
and
straightforward.
There's
a
component
called
black
box
Porter
that
not
only
can
monitor
if
your
URLs
API
is
so.
External
services
are
odd,
but
also
you
can
have
some
metrics
on
the
certificates
expiring
date,
and
you
can
tell
you
like
in
two
months,
there's
one
certificate
that
will
expire.
You
can
set
up
that
metric,
which
simplifies
your
life.
You
have
different
external
URLs
and
different
certificates.
A
This
was
explained
adverse
link
to
a
robust
perception
that
there's
that's
a
company.
They
had
a
blog
with
a
lot
of
cool
information
and
cool
of
tweaks.
You
can
do
on
primitives.
We
got
that
from
there,
so
kudos.
And
finally,
this
is
what
we
have
yeah
and
it
works,
and
it's
perfect
and
I
promise
I
didn't
think
of
anything
political
when
I
use
that
slide.
A
So
sorry
about
that-
and
this
is
my
favorite
part-
like
you-
get
asked
that
yet
you
have
had
an
awesome
company,
a
great
budget
and
some
very
expensive
stuff-
and
that's
not
true.
That's
what
I
told
you
that
it's
very
good
for
your
startup
right
now.
We
are
currently
monitoring
200
instances
where
small
start-up,
not
thousands.
It's
200.
We
only
use
one
primitive
instancing
GCE.
We
should
be
higher
validity.
I
will
come
to
that
one
hard
drive.
We
have
a
default
retention
period
of
15-day,
which
is
unnecessary
because
we
don't
go
back
so
many
days.
A
A
A
Not
in
our
case,
not
at
all
I,
don't
know
by
heart,
but
maybe
you
all
maybe
I
was
thinking
that
maybe
the
we
don't
so
that
we
chose
carefully
our
labels.
So
we
don't
have
a
lot
of
time
serious.
So
we
we
only
have
like
the
what's
called
so
what's
yeah,
so
our
cardinality
is
not
really
extreme.
So
maybe
that's
the
reason,
but
we
didn't
have
I,
don't
know
what's
the
usage
of
that
machine
because
we
never
got
any
problem
with
it
like.
A
Wouldn't
it
always
went
fine,
so
well,
some
of
the
things
that
we
are
working
right
now,
I
told
you.
In
the
beginning
we
have
some
custom
metrics
that
we
are
still
using
the
text
file
panel
Explorer.
We
would
like
to
probably
in
this
case
with
couchdb,
probably
open
source,
a
solution
for
the
cows
to
be
current
exporter,
which
exposes
metrics
about
how
your
cow
chip
is
performing,
but
not
the
data
or
the
replicas
or
whatever
you
have
inside
the
country.
So
just
the
performance
of
database
itself,
maybe
we'll.
A
If
we
have
time
we've
loved
to
to
implement
that
because
we
need
it
but
it'll
come
soon.
We
don't
have
to
meet
using
H
a
we
should
it's
one
of
our
yeah
one
of
the
things
that
we
should
be
doing,
and
it
can
be
done
I'm
not
saying
that
it
cannot.
It's
just
that
we
haven't
got
the
time
to
do
T
to
do
it.
This
is
something
we're
working
in
like.
A
Instead
of
having
a
global,
Prometheus
monitoring
all
of
your
projects,
you
can
have
one
primitives
in
every
project
because
it
actually
simplifies
the
the
alert
system.
If
you
have
super
critical
projects
and
normal
projects,
the
thresholds
you're
using,
are
not
the
same.
So
you
conditional
alerts
like
hey
if
this
come
if
this
metric.
So
if
the,
if
this
comes
from
a
super
critical
project,
then
say
half
an
hour
trigger.
A
A
Excuse
me
with
different
alert
system
alert
systems
with
different
alerts,
setup
for
each
one
of
them.
That's
very
interesting
and
we
don't
have
an
instrumentation.
Our
code
right
now
we
are
good
with
the
operation
metrics,
the
business
metrics
and
the
blacks
focus
monitoring,
that's
something
we
should
do
for
us,
the
development
team,
because
it
would
ease
our
lives
if
there's
something
wrong
exceptions,
whatever
you,
you
guys
know
it,
but
we
will
first
do
the
DHA
and
then
go
to
instrumenting
our
metrics
and
I
hope.
A
B
A
We
actually
have
separated
dashboards
because
we
are
very
client
oriented,
so
we
have
like
huge
clients
with
their
project.
So
we're
really
done
hat
right
now,
even
if
it
comes
from
one
Prometheus
on
one
graph
Anna,
we
have
separated
dashboards
for
each
one
of
them,
so
it
will
be
seamless.
You
have
separated
graph
Anna's
on
top
of
each
Prometheus
or
one
graph,
Anna
connected
to
different
Prometheus
I'm,
not
really
sure,
if
you
can
do
that,
but
yeah
I
hope.
A
No,
it's
everything
from
it
is
like
it's
one,
Prometheus
right
now
and
yeah.
We
only
have
the
the
only
thing
we
have
we
have
to
do.
Is
that
operation
exporter
that
there
so
that
operation
Explorer
is
something
homemade,
because
it
it
is
specific
to
our
case
from
getting
data
and
processing
it
from
that
CouchDB.
A
True
that
that's
a
very
fair
question:
we
actually
would
still
have
the
email,
the
email
reports
we
have
them
in
place
and
we
do
some
periodical
reviews
on
if
everything
is
going,
fine
and
so
on,
and
also
most
of
the
metrics
we
have
right
now.
So
we
have
to
be
honest
sometimes-
and
this
like
you,
have
to
either.
You
have
very
few
make
very
few
alert
or
too
many
you
cannot.
While
you
are
trying
to
focus
on
that
your
target,
we
have
maybe
too
many.