►
From YouTube: Linking Events to Achieve Traceability and Data Aggregation in CI/CD Systems - Emil Bäckmark
Description
Linking Events to Achieve Traceability and Data Aggregation in CI/CD Systems - Emil Bäckmark, Ericsson
The usage of immutable events notifying occurrences in CI/CD systems is increasing and we see standards like CloudEvents and event implementations in e.g. Tekton emerging. Events are useful for triggering and propagating information in these systems. To see the bigger picture, by aggregating data from several events, the receiver need to have some kind of context to see which events belong together. One way to create this context is by using event linking, where events reference other events. This will enable data aggregation, complex event triggering and traceability. Traceability is crucial when for example a bug or a security vulnerability is found in a deployment. When traced back to a bug in a library, we also need to trace forward to what deployments that library was part of. This is needed regardless of who developed the products and what tools where used.
For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/
A
A
A
A
A
A
So
what
are
events
then?
According
to
this
definition,
made
by
cloud
events,
which
is
a
standardized
generic
event
protocol
that
has
gained
a
lot
of
interest
lately,
an
event
is
a
data
record
expressing
an
occurrence
and
its
context.
Events
are
routed
from
an
event
producer,
which
is
the
source
to
interested
event.
Consumers
the
routing
can
be
performed
based
on
information
contained
in
the
event,
but
an
event
will
not
identify
a
specific
routing
destination.
A
A
So
what
are
these
events
links,
then,
that
we
talk
about
when,
when
events
are
sent
from
a
ci
series
system,
they
normally
have
natural
relations
to
each
other.
For
example,
a
build
often
follows
after
source
change
that
is
merged
and
a
test
activity
often
follows
after
a
build,
etc,
and
as
I
mentioned
previously,
events
should
have
unique
ids
and
what
we
mean
with
event.
Links
is
essentially
them
a
reference
to
the
id
of
a
previously
sent
event.
A
A
A
We
could
also
add
a
third
event
to
this
picture
and
to
trace
back
from
event
c
to
event
a
you
will
need
to
follow
links,
recursively
backwards.
First
from
event
c,
you
will
go
to
event
b,
the
id
there
and
from
event
b
that
event,
you
will
go
and
find
up
the
event
a
so.
The
current
event
of
nc
in
this
case
does
not
contain
a
link
to
all
previous
events
explicitly.
A
It's
also
so
that
cloudoven
states
that
publishers
should
keep
events
compact
by
avoiding
embedded
and
avoiding
to
embed
large
data
items
into
event,
payloads
and
rather
use
them
to
event
and
rather
use
the
event
payload
to
link
to
such
data
items
and
the
follows
with
from
an
access
control
perspective.
This
approach
also
allows
for
a
broader
distribution
of
events
because
accessing
event
related
details
through
this.
Resolving
links
allows
for
differentiated
access,
control
and
selective
disclosure,
rather
than
having
sensitive
details
embedded
in
the
event
directly.
A
A
A
And
these
types
could
then
explain
what
kind
of
relationship
fnc
has
with
us
to
previous
events.
So,
for
example,
event
c
could
be
an
event
describing
that
a
component
has
been
built.
Then
event
a
could
describe
the
source
change
in
a
library
involved
in
the
build
and
event
b
could
describe
another
source
change,
making
this
component
c
here
or
the
reference
the
component
referenced
by
event
c,
to
become
an
integration
of
two
sub
components.
A
Similarly,
multiple
events
could
also
link
to
the
same
previous
events
where
each
link
could
have
different
types
as
well
and
in
this
example,
event
b
and
c
could
represent
two
different
deployments,
for
example,
of
the
same
component
where
the
component
is
represented
by
an
artifactory
image
built
and
modified,
built.
Sorry
and
notified
through
event.
A.
A
A
Now
why
should
you
then
link
events
in
this
way?
Let's
go
into
the
main
topics,
the
topics
of
traceability
and
data
aggregation,
and
we
start
with
traceability.
So,
with
the
graph
of
linked
events
from
your
cser
system,
you
can
trace
events
end
to
end
to
find
what
has
happened
in
complex
production
flow
from
the
earliest
source
code
commit
which
we
exemplify
to
the
left
of
the
picture
here
of
a
low
level
library
all
the
way
to
a
customer
deployment
and
beyond.
If
you
like,.
A
You
can
also
trace
backwards
or
upstream,
as
we
call
it
to
find
out,
for
example,
what
source
changes
were
involved
in
a
certain
artifact
version
denoted
by
this
green
bubble.
Here,
even
if
those
source
changes
were
made
by
different
organizations
and
in
different
sem
systems,
even
also
trace
forwards
or
downstream,
to
find
out,
for
example,
where
a
certain
application
has
been
deployed,
even
if
those
deployments
are
made
in
different
target
environments
and
in
different
customer
networks.
A
Even
so,
traceability
is
crucial
when,
for
example,
a
bug
or
a
security
vulnerability
is
found
in
a
deployment
when
traced
back
to
a
bag
in
the
library
away
to
the
left.
Here
we
also
need
to
trace
it
forward
to
see
what
deployments
that
library
were.
Part
of
this
is
needed,
regardless
of
who
developed
the
products
and
what
tools
were
used
for
it.
A
A
Then
we
have
two
different
parallel
test
activities.
They
are
not
directly
related
to
each
other.
They
are
in
parallel,
one
of
them
references,
the
linux
field
and
one
other
references
to
windows
build,
and
then
we
have
the
release
activity
in
the
end,
which
then
references
both
tests
and
test
activities
from
linux
and
windows.
A
A
Then
it
could
be
so
that
our
pipeline
engine
has
has
possibilities
to
rerun
things
when
they
fail
for
different
reasons,
of
course.
So
then
there
will
be
a
new
unit
test
event
sent
out
after
that
activity
is
done,
and
if
that
succeeds,
then
it
will
try
to
continue.
But
how
should
we
then
visualize
this
well
since
the
unit?
The
second
unit
test
activity
here
has
a
reference
to
the
original
build
activity
directly.
A
A
In
order
to
perform
complex,
triggering
based
on
data
from
multiple
events,
the
events
could
strongly
benefit
from
being
connected
through
semantic
links
it
could
trigger.
We
can
trigger
certain
upstream
builds
or
releases
based
on
linked
information,
such
as
triggering
each
release
of
a
certain
upstream
library,
so
this
is
triggering
on
a
release
of
an
oxygen
library
here
in
which
this
other
library
is
also
updated.
So
there
could
be
another
library
here
which
is
used
in
this
field.
A
So
when
that
happens,
when
there's
a
build
here,
which
then
has
a
reference
to
this
library
back
here,
we
will
then
trigger
the
next
activity.
Here,
an
example
could
be
that
we
should
trigger
on
any
build
from
a
certain
upstream
library,
but
we
should
wait
for
a
certain
confidence
level
to
be
reached
for
that
build
before
we
actually
run
it
in
our
production
environment.
A
A
A
A
A
A
You
could
also
have
a
local
store
in
the
consumer
itself,
as
all
events
are
typed
and
there
are
link.
The
links
are
semantic
between
these
events.
It
should
be
possible
to
store
just
the
events
that
you
will
be
interested
in
keeping
and
with
the
combination
of
these
stores,
we
should
be
able
to
deal
with
connected
events
sent
from
a
ci
series
system
and
glenn
spanning
over
multiple
source
code
communities,
as
well
as
multiple
end
user
environments.
A
So
now
it's
time
for
reality
check
them
all.
Those
links
haven't
used
in
practice
anywhere.
Yes,
they
are
so
my
company,
where
I
work
ericsson,
is
a
telecom
networks
provider
focusing
on
mobile
networks
and
services.
We
are
about
a
hundred
thousand
employees
worldwide
and
our
r
d
organization
holds
about
25
000
employees.
A
A
It
was
open
source
to
the
public
a
few
years
back
and
is
now
maintained
on
github
by
a
community
with
representation
from
diverse
businesses
such
as
security,
business
or
surveillance,
and
the
track
building
business
in
excel.
We
have
developed,
plugins
and
adapters
for
managing
these
events,
to
several
of
the
tools
that
we
use
in
our
in
rci
city
systems,
and
we
send
millions
of
these
events
daily
in
our
software
production
pipelines.
A
A
A
A
Soon
there
we
have
linkedin
community,
where
we
discussed
the
ecosystem
around
that
asia
protocol
and
we
have
the
sig
special
interest
group
for
events
in
cdf
and
in
that
we,
as
I
mentioned
in
the
talk
I
will,
we
will
soon
have
a
birth
of
a
feather
session
there
with
a
small
panel,
including
me
and
andrea
fritelli,
and
also
eric
sternerson,
three
of
us.
We
are
all
in
that
sig.
So
we
are
happy
to
talk
to
you.
A
A
A
We
send
multiple
events,
so
actually
millions
of
events
per
day
in
the
in
our
cicd
pipelines
in
ericsson,
and
we
use
the
events
for
telling
that
now
we
have
made
a
source
change,
and
now
we
have
created
a
baseline
or
composition
that
we
as
we
call
it
out
of
those
number
of
source
changes
and
then
those
then
composition,
events
linked
to
each
individual
source
change
event
and
on
a
composition.
We
then
build
some
kind
of
artifact
or
multiple
artifacts
and
they
then
link
to
that
composition,
and
then
we
test
those
artifacts.
A
And
we
have
something
that
we
call
a
confidence
level
or
maturity
level.
If
you
will
that
when
a
certain
number
of
tests
have
passed,
we
can
set
the
confidence
level
on
the
artifact
or
the
composition
itself,
stating
that
now
we
have
reached
a
certain
maturity
level
in
our
system,
in
our
inner
source
code
and
in
our
deliveries.
A
So
I
guess-
and
all
these
then
are
using
links.
So
between
all
these
events,
there
are
multiple
links
going
on,
so
we
frequently
do
this
so
so
called
upstream
and
downstream
searches
on
our
events
to
trace
what
was
included
in
a
certain
build
in
certain
artifacts
or
what
tests
have
been
performed
on
a
certain
source
change
and
those
kind
of
things,
and
we
also
calculate
kpis
and
keep
track
of
our
ci
series
system
performance
in
that
way.
A
A
A
A
A
We
currently
store
our
events
in
a
mongodb
database
as
documents,
then,
so
they
are
stored
as
plain
documents
and
then
we
have
implemented
in
that
mongodb
instance.
We
have
implemented
this
upstream
downstream
searches,
as
as
entry
points
to
the
to
the
database.
We
can
easily
get
the
the
full
trace
of
the
event
so
to
say
so.
Well
we
store
them,
of
course,
with
their
timestamps
intact,
so
they
are
just
json
documents
and
then
we
also
aggregate
them
in
other
systems.