►
A
So
thanks
for
going
through
these
these
items
already
like
it
sounds
like
you
already
have
a
discussion
here.
First
of
all,
I
just
wanted
to
summarize
how
this
will
be
set
up.
I
answered
your
question,
like
I
think
I
think,
between
five
and
ten
shards
total
across
all
environments,
where
each
shard
is
a
full
petroleum
cluster.
So
if
that
helps
with
like
sizing
to
give
you
an
idea.
A
A
It's
going
to
start
out
with
this.
It's
basic.
I
think
the
first
phase
is
probably
going
to
be
a
replica
like
we'll,
create
a
new
database.
That's
going
to
function,
a
new
database
cluster,
that's
going
to
function
as
a
replica
and
then
there'll
be
a
feature
flag
that
will
start
using
the
replica
for
certain
tables,
probably
ci
to
start
and
then
and
then
eventually
we'll
you
know
make
it
so
that
that
cluster
is
no
longer
a
replica
but
functions
on
its
own
independent
of
the
main
cluster.
A
B
A
No,
the
well
there's
going
to
be
a
dedicated
gcp
project
per
environment,
similar
to
what
we
have
now.
I've
created
these
they're
called
like
production
db
staging
db
and
then
there's
some
sandbox
projects
within
each
project.
Then
there'll
be
n
number
of
database
shards
and
each
shard
will
have
like
its
own
network
space
so
that
we
can
and
then
the
current
idea
is
that
we'll
peer,
the
vpc
in
each
environment,
to
the
corresponding
environment,
like
productiondb,
will
get
peered
to
production
and
then
using
firewall
rules
will
give
production
access
to
individual
charts.
A
A
The
forecast
for
where,
like
how
much
runway
we
have
for
the
database,
is
sometime
next
year
before
we
like,
I
think
the
estimate
was
like
april
or
may
of
next
year
before
we
get
to
like
critical,
where
it
becomes
critical
that
we
have
to
do
this,
we're
we're
kind
of
like
doing
doing
the
sharding
now
to
get
ahead
of
that,
because
we
figure
there's
probably
going
to
be
lots
of
issues,
and
we
just
want
to
I
mean-
and
we
don't
know
like
it's
possible-
that
that
timeline
could
change
right.
Okay,
cool!
B
A
B
A
A
A
What
we've
just
described,
yeah,
okay,
so
that's
simple
so
down
to
well,
let's
jump
to
2c.
You
made
a
comment
here
about
indices
yeah.
So
I'm
thinking
I
don't
know
what
I'm
thinking
here.
To
be
honest,
I'm
I
have
no
idea,
but
I
I
I
maybe
better
not
to
use
the
existing
index.
So
we
don't
mess
it
up,
so
we
create
a
new
one
like
pub
sub,
post
resident
staging
db
or
or
if
db,
shards,
yeah
actually
yeah.
I
was
thinking
something
like
yeah.
B
So
there's
a
few
things
to
consider
here.
One
thing
is
that
the
volume
of
logs-
I
don't
think
like
one
of
the
reasons
why
we
were
splitting
indices
in
the
past,
was
that
the
load
volume
was
massive
and
like
elastic,
wasn't
just
able
to
cope
with
that.
So
splitting
it
into
separate
indices,
allowed
it
to
to
use
more
shards,
but.
B
B
It
might
be
perfectly
reasonable
to
to
say
we're
sending
we're
sending
all
postgres
logs
from
production
to
the
single
index,
because
we
don't
want
to
split
them
in
in
separate
indices,
because
there
there
will
be
a
transition
period
where
the
existing
postgres
chart
will
be
still
running
in
the
gitlab
gprd
project.
So
the
question
would
be
like
you
know:
how
do
you
transition?
Do
you
like
sent
to
both
of
them,
etc,
etc?
A
B
A
A
So,
given
that
we
have
n
b
equals
gprd
db
and
n
equals,
you
know
g
stage
db.
Should
I
just
create
new
indexes
using
gprd
db,
gstagedb,
keeping
everything
else
the
same
and
then
and
then
we
just
use
the
project
because
that's
that's
sort
of
like
it's,
not
the
project
name.
Project
name
is
production
db,
but
it's
the
same
kind
of
yeah.
It
follows
the
same
convention.
B
A
B
B
Well,
but
it
would
still
be,
it
would
still
only
be
prosperous
logs,
so
I
wouldn't
expect
there
to
be
and
presumably
deployed
using
the
same
method,
so
it
wouldn't
be
like
in
one
environment
in
one
gcp
project,
it's
running
in
kubernetes
in
the
other,
it's
in
chat,
and
thus
you
know
you've
got
different
fields.
So
I
guess
it's
something
to
to
to
figure
out,
as
as
we
move
along.
B
On
the
other
hand,
on
the
other
hand,
some
like
eoc
might
get
alerted
for
possibilities
performance
they
might
go
to
the
yeah.
That's
true,
yes,
and
you
know
forget
that
there's
multiple
indices
they
need
to
check.
Like
I
don't
know
like
I
think,
yeah
I
I
don't
know,
I
think
I
think
we'll
we'll
need
to
or
you'll
need
to
figure
out
as
we
go
like
make
it
like,
make
all
the
basically
make
a
decision
here.
A
Yeah
I
mean
I
could
it's
it's.
We
can
probably
delay
it
like
we
could
do.
We
could
use
the
existing
index
on
staging
we're
not
going
to
do
this
on
prod
for
a
while.
Yet
so
we
do
use
the
existing
index
on
staging.
We
have
to
come
up
with
a
shard
label
for
the
main
cluster,
so
we'll
call
it
like
main
or
something.
A
B
A
B
Running
kubernetes
would
be
an
overkill
well,
we
would
still
be
running
well
so
and
in
point
2d,
you
mentioned
that
you
wanted
to
get
the
matrix
into
that.
That
means
that
some
components
of
diamonds
will
have
to
be
running
in
that
project
and
prometheus
as
well.
So
we've
got
at
least
one
thanos
component,
I'm
thinking
about
thanos
cipher,
but
there's
at
least
one
thomas
component
and
some
prometheus.
B
Both
of
them
need
to
be
deployed,
configured
maintained
so
at
some
point
we'll
need
to
update
them.
So
my
thinking
is
what's
the
easiest
way
to
update
like
so,
let's
say
we
we
hit
a
bug
in
in
famous,
which
is
something
that
happened
in
the
past.
What
we've
done
like
we,
we
had
to
upgrade
monitoring
components
on
multiple
occasions,
but
on
one
occasion
we
had
to
patch
the
thomas
binary
and
just
having
a
single
source
of
that
binary
was
very.
A
B
A
Well,
right
now
we're
we're
flirting
with
the
idea
of
using
omnibus,
but
you
know
we're
not
ready
to
marry
it
so
we're
going
to,
but
right
now
we're
just
doing
ansible
on
you
know,
bass
ubuntu
with
the
omnibus
installed
and
we're
doing
that
everywhere.
We're
not
using
the
part
of
this
is
not
using
chef
at
all
nice.
A
B
A
My
take
is
like
for
expediency.
I
feel
like
it'd,
be
simple
for
infant.
Like
you
said
like
for
inventory
generation,
it's
really
simple
to
do
this
and
ansible,
because
I
can
just
write
out
the
prometheus
config.
I
don't
think
setting
up
thanos
and
pub
sub
is
gonna
be
tricky
either.
I
would
just
like
create
one
vm
and
install
all
of
these
components
on
it
and
then
be
done
with
it.
That's
my
that's
what
I
was
thinking.
B
So
I
was,
I
was
updating,
alert
manager
the
other
day
and
some
of
the
prometheus
instances
didn't
really
deal
with
that
very
well,
and
I
had
to
go
through
prometheus
conflict
like
basically
go
through
contact
with
all
arbiters
instances
and
there's
the
conflict
is
spread
across
multiple
places
and
like
this
would
be
if
we
were
to
to
run
this
in
with
ansible.
That's
yet
another
way
to
configure
it.
Just
just
just
you
know
like
I'm,
not
I'm
just
I'm
just
like
voicing
voicing
that.
B
I'm
not
saying
that
it's
a
bad
thing
or
a
good
thing,
necessarily
and
my
point
being
since
we
intend
to.
I
will.
I
don't
know
if
that's
still
the
case,
but
up
to
my
best
knowledge,
we
intend
to
move
chef
conflict
to
ansible
conflict.
So
perhaps
this
is
a
good
first
candidate
because
he
would
be
starting
from
scratch,
and
we
once
we
once
we
have
all
the
ansible
playbooks
for
prometheus,
then
moving
the
existing
prometheuses
that
are
managed
with
chef
would
be
much
easier.
A
B
Yeah,
because
is
one
of
the
things
that
you
could
you
could
leverage
if,
if
prometheus
was
running
on
a
vm,
is
the
prometheus
service
discovery
mechanism?
So
it
can
talk
to
gce
to
discover,
scraping
end
points,
and
I
I
don't
think
that's
possible.
Well,
I
don't
I
don't.
I
don't
know
if
that's
possible
in
in
kubernetes
like
it
might
be
possible
that
you
just
tell
a
prometheus
business
running
in
cuba
and
he's
hey.
A
B
A
A
A
A
B
B
A
A
A
A
B
That's
that's
a
good
question.
I
and
the
short
answer
is,
I
don't
know.
I
think
that
then
managed
to
move
all
of
it
to
kubernetes,
but
I
don't
know
I've
definitely
seen
some
vms
named
after
thanos
components,
but
I'm
not
sure
if,
if
they're
still
being
used
like
what's
running
on
them,
I
don't
know
the
short
answer.
Is
I
don't
know
I.
I
think
the
intention
was
to
move
all
thanos
stores
to
run
in
and
they've
been
started.
A
A
Yeah,
okay
yeah:
this
sounds
to
me
like
the
yeah.
The
easiest
thing
would
be
just
to
have
a
kubernetes
cluster
and
you
know
just
have
to
figure
out
how
to
scrape
these
extra
because
we're
not
going
to
be
really
monitoring
nothing.
It's
great
writing,
including
it's
all
going
to
be
outside
like
petroni,
and
I
mean
I
guess
we
could
run
some
exporters,
like
maybe
a
stackdriver
exporter
or
something
in
kubernetes,
but
I
think
most
of
you.