►
Description
Weekly demo issue - https://gitlab.com/gitlab-org/incubation-engineering/apm/apm/-/issues/26
A
Hi
joe
shaw
here
full
stack
engineer
in
incubation
engineering,
I'm
looking
into
application
performance,
managing
monitoring
and
observability
as
a
single
engineer
group
in
incubation,
here's,
my
weekly
update
issue
and
you
can
link
to
the
weekly
issue
here.
If
you
want
to
subscribe
so
we've
been
on
the
road
to
getting
test
infrastructure
set
up
for
this
project
and
there's
a
few
items
that
we've
had
to
take
off
before
we
can
quite
get
there.
So
in
the
last
week
we've
been
working
our
way
through
these,
so
initially
the
refined,
datadog,
metrics
gateway.
A
The
initial
cut
of
the
code
was
quite
rough
around
the
edges
and
not
really
worthy
to
go
anywhere.
So
I
had
to
give
that
a
bit
of
a
clean
up.
Part
of
that
is
adding
linting
into
the
girl
code
to
make
sure
that
errors
are
being
handled.
Things
like
that
and
we're
running
analyzers
like
gosek
and
whatnot
to
make
sure
that
there's
some
level
of
quality,
automation
and
static
analysis
in
the
code
base
there.
So
that's
all
running
as
part
of
that
and
we've
got
a
bit
of
testing
around
ci.
A
So
I'll
just
show
you.
We
did
quite
a
large
merge
request
for
a
lot
of
this
stuff
that
we
got
a
bit
carried
away
with.
This
is
merged
in
now
I'll
quickly
show
you
the
ci
file,
there's
a
lot
of
changes
here,
but
I'll
I'll
show
you
this
one.
So
we
are,
you
know,
doing
standard
stuff
like
building
using
golang
images,
running
unit
tests,
linting
things-
this
is
a
potential
interest
to
people-
is
how
we're
running
the
integration
test.
A
So
we
have
both
tests
that
test
the
migrations
that
we'd
worked
on
previously,
to
make
sure
that
those
migrations
can
go
up
from
scratch,
be
rolled
all
the
way
back
down
and
go
back
up
again.
This
is
good
practice
with
migrations
to
make
sure
that
they
are
reversible.
So
if
we
need
to
roll
back
changes
in
in
a
real
environment,
we
can
kind
of
rely
on
this
test
to
do
that.
A
We
want
to
test
that
as
close
to
that
as
possible
in
integration
tests,
whilst
doing
it
as
cheaply
as
possible
as
well-
and
one
thing
we
do
is
we
have
a
config
file
checked
into
the
project
that
works
locally
as
well.
In
docker
compose
and
also
works
in
this
test,
so
we
just
have
to
do
a
little
bit
of
checking
around
here,
because
the
services
come
up
before
the
git
repo
is
checked
out
in
the
ci
container.
So
first
we
wait
for
that
file
to
exist.
A
We
sleep
while
we
wait
for
it
and
then
we
copy
that
config
into
the
click
house
servers,
config
d
folder
there
and
run
the
entry
point.
So
that
way
it's
for
that
to
come
up-
and
we
also
it
took
me
a
little
while
to
work
out-
was
adding
this
feature
flag
into
the
gitlab
ci
job
here
as
a
variable,
and
you
can
find
these
feature
flags
documented.
A
It
sets
up
a
whole
network
as
if,
like
you
were
using
something
like
docker
compose,
so
these
containers
can
talk
to
each
other.
So
in
this
config
that
we've
got
click
house
is
talking
to
that
zookeeper
instance
and
we
don't
have
to
mess
around
with
ip
addresses
and
things
so
that
actually
works
really
nicely.
That's
a
really
solid
integration
test,
and
I
can
show
you
that
in
you
can
see
the
pipelines
that
we've
got
in
ci
here
builds.
A
You
know
tests
where
we're
doing
linting
golang
lint
there,
for
example,
and
the
integration
test
there
so
starting
the
services
at
the
top.
Here,
click
house
zoom
keeper,
and
it's
going
to
wait
for
that
to
start
up,
and
then
it
runs
our
code.
It
runs
the
migration
test
where
we
migrate
up
down
up
again
and
then,
if
that
works,
then
we
move
on
to
the
integration
test
in
the
database,
where
we're
doing
some
basic
integration
tests
there.
A
So
that's
great
and
the
other
thing
we
worked
on
got
done
in
the
last
week,
was
to
get
the
ci
images
built
and
pushing
to
our
own
registry.
Here
we
use
a
tool
called
scaffold
for
local
dev
in
this
project,
which
allows
us
to
really
iterate
really
quickly,
with
getting
images
built
and
deployed
into
mini
cube.
We
can
use
the
same
tool
to
build
and
push
to
the
gitlab
registry.
A
The
exact
git
commit
that
that
image
is
released
with,
which
is
a
really
good
practice,
and
it
makes
it
nice
and
traceable,
which
is
good.
So
a
few
useful
links
came
up
some
discussions
about
other
metrics
databases
like
victoria
metrics.
There
I've
got
a
link
to
in
particular,
though
I
will
show
you
this.
Someone
suggested
to
me.
A
This
see
loki
implementation,
so
it's
in
grafana
you've
got
loki,
which
is
a
back-end,
both
back-end
and
a
sort
of
query
language
which
will
store
logs
in
s3,
and
this
is
a
clean
room,
implementation
with
click
house
as
a
back
end.
Implementing
the
log
ql
query
language.
So
we
we
could
have
a
look
at
this.
This
looks
really
useful.
We
you
know
either
use
it,
as
is
for
the
logs
or
or
take
a
lot
of
good
ideas
from
it.
A
So
this
looks
very,
very
useful
for
logging
in
another
example
of
how
you
can
use
click
house
for
this
sort
of
sort
of
work.
So
I
will
look
look
more
closely
at
this
at
some
point
soon,
so
I'm
still
getting
the
test
infrastructure
set
up.
That's
what
I
need
to
do
this
week.
I
really
need
to
get
this
done.
A
A
A
And
we
can
embed
those
images
in
there
via
their
shar
versions
as
well.
So
I'll
just
quickly
show
you.
If
you
look
at
the
package
registries
here,
you've
got
your
container
registry,
but
you
also
want
your
package
registry,
so
that
allows
us
to
do
things
like
push
push
down
charts
into
there.
I
haven't
tried
that
yet,
but
that's
that's
what
I'm
going
to
do
next.