►
Description
On March 24, 2021 the Knative community hosted a meetup featuring a demo, "Release Automation for Knative Apps with Iter8" presented by Srinivasan Parthasarathy, Research Scientist at IBM. Iter8 is an open source AIOps platform for cloud native release automation. Iter8 enables developers, data scientists, and SREs maximize business value and guarantee SLOs by automating metrics-driven experiments, progressively delivery, validation, and promotion/rollback of new versions of apps and ML models.
A collection of demos will showcase how Knative app developers can use Iter8 to maximize their release velocity with confidence while protecting end-user experience. The demos highlight a few key benefits of Iter8 during Knative app releases, namely, conformance and canary testing, progressive, fixed-split and dark-launched deployments, mirroring/shadowing and request routing, integration with Helm and Kustomize, and the ability to use metrics from any REST API during experiments.
A
My
name
is
srinivasan,
I'm
a
recent
scientist
at
ibm,
research,
new
york
and
I'm
here
to
introduce
the
iterate
project
to
the
key
native
community.
Today,
the
iterate
project
is
made
possible
by
contributions
by
a
number
of
wonderful
people.
I've
listed
their
names
here
and
this
is
a
growing
list
and
hopefully
will
continue
to
grow.
A
Okay.
So
what
is
iterate
iterate
is
an
open
source
platform
for
cloud
native
release,
automation
and
experimentation
and
as
developers
of
kubernetes,
apps
or
machine
learning
models,
we
have
a
often
we
have
a
number
of
goals.
Maybe
we
want
to
in
a
canary
test.
We
want
to
validate
our
canary
release.
Canadian
reversion
is
satisfying
service
level
objectives
or
maybe
you're
doing
a
b
test
or
an
abn
experiment,
and
you
want
to
identify
the
best
version.
You
have
a
number
of
versions
of
your
application.
A
You
want
to
compare
them
and
maybe
pick
the
one
that
maximizes
the
business
objective
of
interest
to
you:
maximizes
user
engagement,
maximizes
revenue,
minimizes
cost,
whatever
pick
the
best
version,
and
in
doing
these
types
of
releases
and
doing
these
types
of
experiments,
we
also
want
to
protect
the
end
user
experience.
So
if
a
version
is
satisfying
slos,
maybe
we
want
to
shift
traffic
to
it
gradually.
A
A
A
In
particular,
you
can
use
this
experiment
to
say
that
you
want
to
do
a
conformance
test.
This
is
a
different
conformance
test
than
what
we
heard
about
before.
This
is
really
about
validating
slos
using
metrics,
and
you
can
do
a
canary
test
where
you
have
two
versions
and
you're
comparing
two
versions,
you
can
progressively
shift
traffic
to
a
particular
version
of
your
application
and
based
on
how
it
performs
with
respect
to
metrics,
you
can
specify
slos
or
slis
based
on
metrics,
and
you
can
also
bring
in
more.
A
You
know
integrate
more
advanced
features
like
dark
launches
traffic,
mirroring
and
segmentation,
which
we
will
see
today,
and
you
can
also
integrate
experiments
with
app
configuration
tools
like
help
and
customize.
So
this
is.
These
are
some
features
at
a
glance,
and
these
are
things
that
you
can
start
using
today,
all
right,
so,
let's
dive
into
our
first
demo.
This
is
a
conformance
testing
demo
for
a
key
native
app
and
once
again,
by
conformance
testing.
What
we
mean
is
that
your
app
your
this
version
of
your
app
satisfies
slos
that
are
specified
within
the
experiment.
A
This
is
the
scenario
that
we
are
going
to
demo.
You
have
a
single
version
of
your
k
native
application,
so
we
will
see
that
this
is
actually
a
canadian
revision
in
k.
Native
versions
correspond
to
revisions,
so
you
have
a
canadian
revision
and
you
know
the
picture
says
that
it's
really
a
production
version,
so
it's
receiving
user
traffic
and
it's
serving
requests
from
users
so
in
in
reality,
you
can
do
this
type
of
conformance
test,
either
with
a
production
version
or
in
a
staging
environment.
A
It's
really
up
to
you
how
view
which
version
you
pick
to
do
this
experiment
on
essentially
as
part
of
a
conformance
test,
you're
going
to
continually
validate
the
version
and
check
if
it
is
going
to
satisfy
sls
that
are
that
you
care
about
in
this
experiment.
We
will
specify
latency
sellers
and
we
will
specify
error
data
sellers,
but
the
sls
can
be
based
on
any
metrics
that
are
available
to
you,
and
this
is
how
the
demo
is
about
to
unfold.
A
So
manually,
I'm
going
to
launch
a
k-native
service
with
a
single
revision,
and
then
I
will
launch
the
iterate
conformance
experiment
and
iterate
is
going
to
automate
several
steps
for
me,
so
it
is
going
to
first
of
all,
it's
going
to
do
some
basic
checking
in
the
beginning.
It's
going
to
verify
that
you
know
the
canada
version
actually
exists.
It's
not
a
it's,
not
a
non-existent
revision
and
all
the
metrics
that
I
specify
within
the
experiment.
They
actually
exist
and
then
it's
going
to
periodically
query
the
metrics
back
in,
in
this
case
prometheus.
A
A
A
A
Oh,
so
I
must
say
that
I
already
set
up
a
few
things
here
right,
so
obviously
I
have
a
local
kubernetes
cluster
and
it's
running
canada
and
it's
also
has
got
ins
iterate
installed.
So
some
of
the
setup
has
been
taken
care
of
in
the
background
already,
so
that
I
can
get
started
with
the
demo.
So
I've
gone
ahead
and
launched
this
keynative
service.
It's
a
very
simple
service
taken
from
key
native
tutorials.
A
A
Of
course,
in
a
real
environment
in
a
production
environment,
your
service
is
probably
receiving
user
requests,
real
user
requests,
but
you
know
in
this
demo
I'm
just
going
to
simulate
user
requests
by
generating
a
few
requests
myself
using
a
using
this
tool
called
fodio
all
right.
So
I'm
about
to
create
the
iterate
experiment,
but
let's
take
a
quick
look
at
what
this
experiment
is
going
to
do
so.
A
First
of
all,
I
specified
that
the
target
of
my
experiment
is
this
particular
key
native
service,
so
sample
app
server
is
in
the
default
namespace,
and
if
we
dive
a
little
bit
deeper
and
specifically
saying
within
that
service,
there
is
a
revision
called
sample
app.
We
want
pick
that
up.
That
is
what
we
are
going
to
do
or
experiment
with,
and
we
want
to
validate
this
version
by
verifying
it
satisfies
these
objectives.
A
So
these
are
the
objectives
that
I
would
like
my
version
to
satisfy
its
mean:
latency
needs
to
be
within
50
milliseconds,
its
state
latency
needs
to
be
within
hundred
milliseconds,
and
its
error
rate
needs
to
be
within
one
percent.
If
I
see
these
things
happening,
then
I
would
say:
yes,
your
version
is
great.
Otherwise,
well
not
me,
but
iterate
is
going
to
say.
Your
question
is
great.
Otherwise,
literate
is
going
to
declare
that
your
version
is
not
satisfying
objectives
and
there
are
other
things.
There
is
a
initialization
action
that
I'm
specifying
here.
A
A
A
The
experiment
looks
like
it's
just
getting
started,
so
there
is
no
data
coming
back,
we'll
look
into
what
is
coming
back,
but
you
know:
experiment
itself
is
a
community's
resource,
so
we
can
just
cuba
city
will
get
it.
We
are
seeing
that
there
are
three
iterations
of
the
experiment
that
are
completed
going
back
to
the
experiment.
I
missed
one
part,
so
the
duration
of
the
experiment
says
that
there
are
10
iterations
of
the
experiment
and
in
between
each
iteration
there
is
a
10
second
gap,
so
periodically
iterate.
A
As
doing
this,
metrics
fetch,
followed
by
evaluation
of
the
worship,
looks
like
it's
now
beginning
to
get
some
metrics
from
the
metrics
back
end
or
there
are
a
few
requests
that
have
gone
to
your
version
and
it
is
reporting
back
the
mean
latency,
the
tail
latency
error
rate
and
your
version
is
doing
great.
All
the
objectives
are
satisfied
and
looks
like
it's
it's
doing
well.
A
So,
as
I
said,
this
is
a
very,
very
simple
conformance
experiment,
but
it
already
demonstrates
how
iterate
validates
your
application
and
your
application
version
in
this
in
this
simple
setting
all
right.
So
I'm
going
to
go
ahead
and
stop
this
experiment
and
move
on
to.
Hopefully,
what
is
a
bit
more
interesting
experiment,
so
in
between
these
demos,
I
need
to
do
a
little
cleanup
so
that
my
demo
environment
is
fit
for
the
next
step,
all
right.
So
let's
go
back
and
look
at
our
next
demo.
A
A
Once
again,
we
are
going
to
do
validation
of
these
versions.
We
are
going
to
check
if
they
satisfy
the
slos
that
you
specify
in
the
experiment.
At
this
time
we
are
going
to
progressively
shift
traffic
to
the
candidate
if
it
satisfies
the
sls
and
at
the
end
of
the
experiment.
Assuming
the
candidate
is
doing
well,
and
it's
found
to
be
a
good
one,
not
a
problem.
It
satisfies
everything
that
you
want
to
satisfy.
We
will
promote
it.
In
other
words,
it
will,
it
will
become
the
new
baseline.
A
A
It's
going
to
use
the
traffic
spec
as
part
of
the
service
resource
to
do
this
traffic
shifting
and,
as
I
said
in
the
end,
it's
going
to
automatically
promote
your
candidate
version
because
it
would
emerge
as
the
winner
in
this
experiment.
It
would
have
satisfied
all
the
slos
and
it
will
end
up
being
the
one
and
only
revision,
the
other
baseline
will
be
garbage
collected
by
canada
all
right.
So
once
again,
this
is
how
the
experiment
is
going
to
unfold.
A
A
A
So
this
is
once
again
the
simple,
a
sample
application
with
a
blue
version
and
then
upgrading
it
to
a
green
version
and
in
the
beginning
the
green
version
is
not
going
to
receive
any
traffic.
It's
just
you
know
deployed,
but
not
receiving
traffic.
At
the
moment,
that's
how
the
server
specification
is
specified.
A
A
A
But
this
time
I
have
more
things
more
details
specified
for
the
version,
so
I
have
first
of
all,
not
one
but
two.
I
have
a
baseline
version
and
a
candidate
version,
a
baseline,
a
sample
of
v1
these
lines
just
a
way
for
me
to
see
if
anything
goes
wrong,
not
able
to
do
the
experiment
properly,
even
or
canada
is
not
working
out,
just
go
back
to
the
baseline.
So
it's
a
it's
a
fallback
mechanism,
so
baseline
is
my
sample
of
v1
and
revision.
The
candidate
revision
example
up
widget.
A
Oh
sorry,
about
the
background
noise,
maybe
I'll
head
over
to
different
space.
Hopefully
the
background
noise
will
subside.
So
that's
about
the
versions
themselves.
Once
again,
I'm
using
the
same
criteria,
I'm
using
the
same
objectives
mean
latency,
pay,
latency
and
error
rate
metrics,
and
here
is
the
action
that
I'm
specifying
as
part
of
the
experiment
for
doing
the
version
promotion.
A
So
when
the
experiment
finishes,
I
can
simply
use
a
cube.
Ctl
apply
command.
It's
not
me
who's
using
it.
It's
literally
experiment
which
is
automating
this
step.
For
me,
it's
going
to
use
this
cube
serial,
apply
command
and
launch
the
appropriate
version.
If
the
version
that
needs
to
be
promoted
is
the
baseline,
it
will
launch
the
baseline.
It
will
cube.
Cd
will
apply
the
baseline
manifest,
otherwise
it
will
keep
cd
reply
the
candidate
manifest.
A
A
A
A
So
what
we
see
here
in
the
status
is
that
the
traffic
shift
is
held
at
95.5
right,
so
you're
not
actually
seeing
traffic
shifting
from
baseline
to
revision
to
the
candidate,
because
you
know
we're
not
getting
any
metrics
for
the
baseline
or
the
candidates,
so
traffic
is
actually
held
at
95
for
the
baseline
and
candidate
at
five
percent.
I
wonder:
what's
going
on
unavailable,
unavailable
and
available.
A
Thing
all
right,
so
there
was
a
downtime
with
create
araya,
which
is
actually
I'm
using
a
red
hat
operator
for
launching
prometheus
pods
and
up
until
last
night.
This
the
container
registry
was
down
at
way
and
I'm
wondering
if
it
has
something
to
do
with
it,
so
the
experiment
actually
is
finishing,
but
it's
not
finishing
the
way.
I
want
it
to
actually
finish.
A
I
was
hoping
to
see
some
metrics
here
and
traffic
shifting
to
the
candidate,
but
that
it
didn't
quite
unfold,
as
I
was
hoping
it
would,
but
you
can
see
that
if
the
candidate
is
not
validated,
the
other
rollback
mechanism
actually
kicked
it.
The
experiment
is
over
and
iterate
has
actually
gone
ahead
and
rolled
back.
The
candidate
100
of
the
traffic
is
now
going
back
to
the
baseline
because
it
could
not
confidently
declare
the
candidate
as
being
the
winner
in
this
experiment.
A
B
A
A
A
So
once
again,
there
is
going
to
be
a
canary
release
experiment
but
and
then,
once
again,
hopefully
we
will
see
some
progressive
traffic
shifting
in
this
particular
instance,
but
we
are
going
to
segment
the
traffic
and
use
only
a
portion
of
our
user
base
to
do
the
experiment,
and
this
is
how
the
the
demo
scenario
looks
like.
So
you
have
a
once
again.
You
have
a
baseline
version
and
a
candidate
version,
but
we
are
only
going
to
use
traffic
from
a
specific
country
called
wakanda
in
our
experiment,
all
the
users
from
wakanda.
A
They
will
be
split
between
the
baseline
and
the
candidate
during
the
experiment.
But
if
you
are
not
from
wakanda,
then
you
are
going
to
end
up
seeing
only
the
base.
There
is
the
all
the
other.
The
rest
of
the
world
is
not
going
to.
Essentially
it's
not
going
to
participate
in
this
experiment.
It
has
a
clean
path
to
the
baseline,
but
if
you're
from
wakanda,
then
we
have
an
experiment
for
you
and
you
will
either
go
to
baseline
or
the
candidate
based
on
the
traffic
split
percentage.
We
will
split
the
traffic
from
a
condo.
A
Once
again,
we
have
the
same
slos
that
we
want
to
verify
within
the
for
the
baseline
and
candidate
versions
and
once
again,
hopefully
we
will
progressively
roll
out
the
candidate.
But
this
time
the
progressive
rollout
will
happen
only
for
users
within
wakanda.
As
I
said,
all
the
other
users,
they
are
unaffected
by
this
experiment.
Even
if
the
camera
is
doing
very
well
the
rest
of
the
users,
we
don't
want
the
rest
of
the
users
to
see
the
candidate
version,
because
we
want
to
minimize
the
exposure.
That's
the
idea.
A
So
this
time
I'm
going
to
go
ahead,
launch
two
different
key
native
services.
So
I'm
going
to
use
some
histo
magic
under
the
powers
to
do
this
experiment
right,
because
the
traffic
segmentation
is
a
capability
that
I
actually
get
from
istio
virtual
services.
So
I
can
overlay
that,
on
top
of
the
canadian
resources
that
I'm
creating
to
achieve
this
experiment
here,
so
I'm
going
to
go
ahead
and
create
two
k
native
services.
A
So
my
two
versions
here
are
canadian
services,
not
revisions,
but
creative
services,
and
I
will
split
traffic
between
them,
based
on
headers
I'll
check.
If
a
request
is
from
wakanda
and
based
on
that,
I
will
decide
to
use
it
or
not
within
the
experiment,
and
that
policy
is
specified
as
part
of
these
two
virtual
cells
and
then
I'll
launch.
The
experiment
and
the
experiment
will
once
again
do
the
validation
and
traffic
shifting
for
me.
A
A
Service,
what
is
this
virtual
server
saying?
So
if
you
look
at
the
virtual
service,
I
have
a
custom
host.
If
you're
you
know
doing
this
in
production,
you
probably
have
your
own
host
for
which
you're
routing
traffic,
I
have
a
custom
host,
and
I'm
saying
if
the
request
for
this
house
comes
from
a
conda,
I
can
split
it
between
the
baseline
and
the
candidate
version.
A
But
if
the
request
for
this
host
is
coming
from
elsewhere
other
than
wakanda,
we
just
go
to
the
baseline
version.
It's
not
going
to
participate
in
this
experiment.
That's
what
the
virtual
service
is
letting
me
do,
let's
go
ahead
and
generate
some
traffic,
and
this
time
the
traffic
generation
is
a
little
bit
more
involved.
So
I
have
traffic
coming
from
wakanda
to
my
services,
and
I
have
traffic
coming
from
ghandar
to
my
services
under
is
just
rest
of
the
world.
A
Once
again,
these
are
just
placeholders
right.
You
can
segment
the
traffic
in
any.
Whichever
way
that
issue
actually
allows
you
to
segment
the
traffic
it
can
be
based
on
headers,
it
can
be
based
on
country,
headers,
user
headers.
It
can
be
based
on
other
parts
of
the
request.
Also,
okay,
so
let's
go
ahead
and
launch
the
iterate
experiment.
A
The
virtual
service,
in
terms
of
what
is
doing
with
the
traffic
wow
okay,
this
time,
I'm
getting
some
metrics,
looking
helpful
now,
okay,
so
I
am
getting
metrics
for
my
baseline
version
and
yes,
the
baseline
version
is
satisfying
all
the
sll's.
True
true
true,
because
its
mean
latency
is
within
50
milliseconds.
Its
state
latency
is
within
100
milliseconds
and
its
error
rate
is
zero
and
looks
like
I'm
beginning
to
get
some
metrics
for
my
candidate
versions.
A
Also,
candidate
version
is
also
satisfying
all
the
slls
that
are
specified
as
part
of
the
experiment
here
and
experiment
is
proceeding
and
traffic
is
beginning
to
shift.
Initially
it
was
a
595
split
between
the
baseline
and
the
candidate
or
between
the
canada
and
the
baseline.
But
now
you
see
the
candidate
taking
on
45
of
the
traffic
and
the
baseline
at
55.
A
C
A
Perfectly
perfect,
so
I'm
done
with
all
the
demos,
so
you
can
head
over
to
iterate.journals,
because
there
are
more
demos
out
there
and,
for
example,
you
can
do
a
dark
launch,
fuk
data
application
and
you
can
use
traffic
mirroring
another
obvious
use,
capabilities
to
route
traffic
and
you
can
do
other
things
like
you
know.
I
showed
you
is
to
your
networking
layer,
but
you
know
key
native
supports
other
networking
layers
and
you
can
experiment
with
iterate
with
canada,
with
other
networking
layers.
A
Oh,
we
do
have
a
kubecon
and
cloud
native
con
europe
321
talk
and
the
iterate
team
will
be
talking
about
best
practices
in
kubernetes
experimentation.
So
if
you're
attending
kubecon
be
sure
to
check
us
out
there
all
right,
so
you
can
join
us
on
italy.tools
on
our
slack
channel
workspace,
and
you
can
also
check
out
our
repos
and
raise
questions
or
comments
there.
Thanks.
C
Thank
you
so
much
sirini
as
you
were
mentioning
you,
have
a
presentation
right
at
qcon,
and
you
mentioned
to
me
that
you
were
interested
in
getting
some
feedback
from
the
canadian
community.
So
I
just
wanted
to
share
that
with
everyone
here.
There
were
a
ton
of
comments
and
some
questions
in
the
chat.
D
E
Well,
this
this
is
max,
I
I
don't
have
a
question,
but
I
have
a
comment,
which
is
that
I
I
love
it,
and
this
is
very
timely.
I
think
we
need
to.
We
need
to
figure
out
how
to
how
to
use
this
more,
but
we'll
be
following
up.
I
guess
this
is
all
open
source
just
to
be
clear.
It's
open
source
under
I'm,
looking
at
the
repos
right
now
the
license
is.
Is
it's
compatible
right
with
what
we
have
right
now
in
canada,.
A
A
A
A
So
if
you
have
more
questions
or
comments,
you
see
part
of
the
reason
why
we
are
doing
this
presentation
is
to
understand
the
needs
of
the
canada
community
right.
So
what
are
the
types
of
release?
Automation,
pain,
points
and
what
are
the
types
of
experimentation
needs
and
pain
points
that
are
worthwhile
attacking?
So
this
is
an
initial
presentation,
and
this
is
definitely
based
on
our
interaction
with
the
istio
community.
It's
based
on
our
interaction
with
the
kids
serving
community,
but
for
the
canadian
community.
A
F
A
Way,
that's
a
good
question
so
internally
when
I
was
demoing
it
in
fact
to
doug
who's
on
the
call
here
his
suggestion
was
that
hey?
Why
not
do
the
traffic
segmentation
using
canada
service
instead
of
using
a
istio
virtual
service,
and
my
answer
was
that
canada
service
doesn't
support,
features
like
that?
Probably
because
you
know
it's
your
dust
and
canada
is
meant
to
work
on
top
of
multiple
network
relations,
so
I
don't
know
the
answer
but
anyway,
so
some
of
these
features
that
makes
experimentation
interesting.
That
makes
experimentation
useful.
A
F
Yeah
I've.
I've
definitely
heard
that
before
for
this
work,
that
our
current
kind
of
traffic
stuff
is
hard
to
use
outside
of
a
demo
like
of
k
native
like
it's
hard
to
actually
use
it
for
any
anything
other
than
the
simple
stuff.
That's
that's
really
interesting
yeah.
I
I.
I
definitely
encourage
like
any
issues
that
you
could
open.
That
kind
of.
Like
I
mean
I,
I
don't
think
it
is
the
case
that
we
think
you
should.
F
I
mean
I
I'm
not
speaking
for
the
project
but
like
I
think
we
have
a
traffic
section
such
that
you
shouldn't
have
to
go
down
into
istio
to
do
at
least
most
of
the
stuff
that
you
were
doing
there
so
it'd
be
interesting
to
kind
of
have
a
like
if
you
had
a
sketch
of
like
what
would
the
api
and
k
native
look
like,
so
they
could
be
orchestrated.
I
think
that'll
be
pretty
cool.
A
A
E
This
is
max,
I'm
also
from
ibm,
but
we'll
we'll
also
follow
up.
You
know
with
the
community.
Obviously,
with
all
this
one
question,
for
you
is,
I
mean
this
is
this
seems
to
be
a
little
bit
mature
work.
I
was
in
research,
so
I'm
assuming
that
this
is
based
on
previous
work
that
you
had
before
or
or
this
is
all
4k
native-
that
you
did.
A
Now
so
this
is
definitely
based
on
previous
work
before
we
have
worked
with
the
istio
community,
in
particular
the
piali
section
of
istio
to
support
experimentation,
and
we've
also
worked
with
the
kf
serving
community
which
is
about
you
know,
surveillance
inferencing,
on
top
of
clean
native
right
so,
and
we
have
some
sense
of
what
the
community
wants
in
terms
of
experimentation
and
release.
Automation
tooling,
so
this
is
based
off
of
that.
We
are
coming
off
of
that.
G
A
It's
mainly
open
source,
however,
we
do
have
users
who
want
to
use
this
introduction
right,
so
in
particular,
if
you're
engaging
with
them
in
the
open.
So
we
are
talking
to
mlaps
communities.
We
are
talking
to
trip
advisor
and
selden,
who
are
interested
in
using
iterate
features
in
production,
and
the
top
comment
that
we
have
right
now
is
that
if
there's
a
way
to
make
this
kidopsy
inject,
you
know
experiments
and
get
ups
pipelines.
They
would
be
using
this
today.
A
So
that's
the
kind
of
feedback
that
we
are
getting
and
that's
definitely
direction
that
we
are
planning
to
head
forward
to
pretty
soon.
But
this
is
what
we
have
heard
so
far
from
users
who
want
to
use
this
introduction.
We
have
more,
you
know,
usage
in,
I
would
say,
non-production
environments,
but
for
production
users.
This
is
what
we
are
here.
G
Yeah,
so
I
that
that
way,
my
my
feedback,
I
put
it
in
the
chat
but
like
these
things,
are
also
in
youtube,
and
everything
in
the
chat
is
lost
by
the
way.
So
so
it's
recorded.
That
was
my
feedback
on
on
git
ups,
and
I
think
scott
also
had
some
comments
in
terms
of
product
production
concerns
and
security.
So
my
recommendation
is
in
the
webpage
or
things
like
that
point
out
in
documentation
or
implementations
of
doing
githubs
of
separation
concerns.
G
I
saw
like
the
the
program
is
doing
keep
ctl
commands
against
itself
or
using
this
in
a
in
a
cluster
that
you
call
the
management
cluster,
and
then
you
have
the
you
have
a
cluster
that
is
the
dev
cluster
and
the
cluster.
Is
production
cluster
like
how
is
that
visibility
of
the
things
that
you're
changing
reflected
back
into
git,
so
an
operations
team
can
can
monitor
that.
G
So
those
type
of
operational
concerns
in
security
and
transparency
are
the
things
that
the
next
customers
ask,
I'm
consulting
with
ibm
the
customers
that
we're
going
to
put
it.
I
work
in
engagements.
That's
the
first
thing
that
I
want
to
look
at.
So
if
it's
not
real
guidance
there
and
I
have
to
like
dig
through
the
documentation
to
really
find
it
yeah,
let's,
let's
adoption-
so
that's
fine.
A
Yeah
appointee
can
I
appreciate
that
comment.
The
experiment
that
I
showed
you
was
definitely
not
a
github
c
experiment,
so
just
to
be
very
clear-
and
you
would
you
wouldn't
do
stuff
like
keeps
you
deal
apply
as
part
of
a
githubs
experiment.
So
that's
not
how
the
githubs
integration
would
look
like.
So,
yes,.
A
C
And
thank
you
so
much
srini
for
presenting
as
well.
I
think
we
have
come
to
the
close
of
our
media.
Today
we
are
going
to
upload
this
recording
to
the
youtube
channel,
so
look
for
it
there
and
if
you
have
an
implementation
or
an
app
that
is
running
on
creative,
please
do
consider
submitting
a
demo
as
well.
We
have
a
form
that
I
can
share
on
the
slack
face
later
and
thank
you
everybody
for
coming
and
if
you
haven't
filled
out
the
survey
that
was
launched
recently.