►
From YouTube: Building a Portable Kubernetes Deployment Pipeline with Argo Workflo... Thomas Meadows & Ollie Young
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Building a Portable Kubernetes Deployment Pipeline with Argo Workflows and Events - Thomas Meadows, Jetstack & Ollie Young, Improbable
In this session, Jetstack and Improbable will uncover the key learnings from building a Kubernetes based platform, driven by Argo workflows and events. Featuring discussion on the platform's strong data aggregation and the ability to deploy on any estate (including air-gapped facilities), this session will investigate the problems and benefits of using Argo and Kubernetes as the key ingredients towards the goal of a holistic approach to portable platforms. The technical content of the presentation will cover: - A general overview of why Argo Workflows and Events were selected - Best practice when building a cloud native platform to work across multi-cloud environments - Rationale for rejecting the typical CI/CD model in favour of a custom solution - Techniques for isolating workflows in multi-tenanted environments - Running Terraform in Argo - Ephemeral environments and managing highly scalable cluster configurations - Platform portability and being infrastructure agnostic
A
B
A
This
is
a
multifaceted
story
which
includes
many
complexities
and
design
challenges
that
we
had
to
overcome
in
order
to
be
able
to
achieve
the
ultimate
goal
of
providing
improbable
with
a
platform
that
could
be
deployed
anywhere
under
any
circumstances,
whether
that
be
on
the
edge
on-prem
or
in
public
cloud.
So
without
further
ado,
we
want
to
kick
off
by
talking
to
you
about
the
problem
at
hand.
We
needed
to
start
out
by
determining
what
is
the
platform
and
what
are
the
ultimate
goals
of
what
we
need
for
it
to
be
delivered
so
ollie.
B
Improv
we've
built
a
synthetic
environment
platform
and
a
synthetic
environment
is
essentially
just
a
highly
scalable
simulation
or
grouping
of
simulations
that
can
be
used
to
represent
various
movements
across
multiple
domains.
So
when
I
say
a
domain,
I
actually
mean
in
the
defense
sense
of
the
word,
so
we're
talking
land
sea
air
space
cyber
and
we
can
simulate
vehicle
movements,
civilian
movements,
and
we
can
package
all
of
that
together
and
enable
our
customers
to
build
a
virtual
world
containing
all
of
that
information.
B
So
that's
what
our
platform
does.
It
enables
you
to
build
a
fully
capable
synthetic
environment
at
scale,
so
our
platform
is
essentially
made
up
of
multiple
components
built
by
multiple
people
or
multiple
teams.
When
we
deploy
it,
we
cannot
really
assume
that
we're
going
to
run
on
any
specific
flavor
of
kubernetes,
it
could
be.
A
So
I
guess
ollie
you
mentioned
that
the
end
goal
of
the
platform
was
to
be
able
to
deploy
on
any
kubernetes
service.
It
needs
to
be
completely
unbiased
in
that
sense,
but
also
it
can't
rely
on
an
external
internet
connection,
these
sort
of
problems
you
can
think
of
the
top
of
your
head.
If
you've
got
any
experience
with
building
platforms
like
this
as
to
what
the
sort
of
challenges
we
face
were
to
be,
but
what
sort
of
challenges
does
that
pose?.
B
Well,
there's
things
like
you
know
not
being
able
to
pull
containers
down
is
is
a
is
a
huge
thing.
You
know
we're
having
to
think
about
how
we
deploy
to
somewhere.
Where
you've
not
got
you
can't
just
pull
source
containers
down,
so
we
we
need
a
way
to
actually
deploy
it
as
some
other
way,
whether
that's
by
like
us,
put
getting
the
yaml
together
and
putting
it
on
a
usb
stick.
It
goes
through
some
some
vetting
process
and
then
it's
plugged
into
a
customer's
site.
B
A
Absolutely
so
moving
on,
I
can
imagine
that
you
know
a
platform
that
allows
you
to
build.
Virtual
worlds
can
be
very
complex,
and
I
can
imagine
that
there
are
lots
of
different
components
moving
around
that
being
developed
by
a
lot
of
different
people
in
a
lot
of
different
teams
with
lots
of
different
methods.
So
what
sort
of
problems
did
you
face
from?
From
that
perspective?.
B
You
may
need
collaboration
from
third
parties,
or
certainly
even
just
other
teams
within
our
own
business
and
getting
them
all
working
together
and
then
once
you've
got
all
these
components
sort
of
talking
to
each
other
and
working
together.
You
need
to
test
that,
and
so
that
presents
challenges
in
itself
as
well.
You
know
you,
you
may
have
a
different
use
case
for
one
customer
and
then
another
use
case
for
another
customer.
You
can't
use
the
same
approach
to
testing
across
across
both
those
use
cases,
because
you've
got
the
different
components
so
yeah.
B
We
need
a
way
to
easily
do
integration,
testing
of
components
on
a
per
customer
basis,
tracking
changes
and
versioning.
There's
there
wasn't
really
any
of
that.
It
was
a
bit
of
a
mishmash,
especially
when
you've
got
third
parties
involved
as
well.
That's
when
it
really
gets
complicated,
so
it
would
often
be
a
case
of
just
getting
some
automated
tests
running
in
a
container
standing
up.
You
know
a
temporary
cluster
or
something
mini
cube,
or
something
like
that.
B
Even
you
know
just
to
just
to
get
some
end-to-end
tests
to
run
problem
was
those
tests
might
work
on
one
set
of
components
to
one
customer
you'd
have
to
write
a
whole
other
set
in
a
separate
pipeline
and
everything
for
another
customer.
We
really
needed
an
easier
way
to
do
that,
because
it
was
starting
to
get
a
bit
messy
and
a
bit
uncontrolled
and
really
difficult
for
auditing.
That
sort
of
thing.
A
I
want
to
ask
you
about:
you
know
the
the
end
deployment
to
the
end
customer,
because
we've
spoken
a
bit
about
how
we
need
to
be
able
to
deploy
to
any
any
customer
infrastructure.
So
if
you
want
to
tell
us
a
little
bit
about
about
the
you
know
just
to
reiterate
over
some
of
the
problems
that
were
posed,
there.
B
Yeah,
we
have
to
essentially
figure
out
how
to
package
up
our
product,
which
is
like,
I
said,
a
bunch
of
different
components
which
is
could
be
concerned
in
some
cases,
a
bunch
of
different
micro
service
or
data
sources
package
that
up
and
get
all
of
that
information
that
deployment
yaml
and
the
containers
and
figure
out
how
we're
going
to
put
that
essentially
anywhere
so
yeah
bare
metal
is
obviously
another
another
popular
one
with
you
know,
because
military
customers
are
not
going
to
be.
You
know
running
on
aws,
for
example,.
A
You
can
imagine
the
manual
task
of
having
to
sift
through
those
manifests
to
find
the
specific
images
having
to
having
to
troll
through
lots
of
manifest
to
find
annotations.
That
could
be
a
real
challenge
and
I
guess
the
the
end
goal
is
trying
to
find
a
situation
where
you
can
develop
a
platform
that
requires
as
little
extra
combing
and
refactoring
after
being
built,
so
it
can
run
on
the
likes
of
these
air
gap
facilities
or
any
facility
for
that
matter,
no
matter
where
you're
going.
A
I
guess
this
is
where
it
get
got
to
the
point
of
the
project
where
we
had
to
come
up
with
a
solution.
I'll
start
off
by
you
know,
walking
you
through
what
we
call
the
orchestrator.
We
decided.
We
want
to
have
a
centralized
approach
of
the
development
platform,
so
we
call
it
the
orchestrator,
and
what
this
is
is
a
single
source
of
truth
for
tracking
component
changes
on
the
improbable
defense
platform.
A
This
orchestrator
is
also
our
single
back-end
data
store.
So
this
this
will
be
relied
upon
to
store
all
of
the
data
required
for
tracking
those
changes
and
all
the
management
that
we
want
to
be
taking
place.
It's
also
responsible
for
orchestrating
infrastructure.
In
an
ideal
world,
we
want
the
improbable
developers
to
be
able
to
request
an
environment
based
off
specific
version
and
just
get
to
work
from
there.
They
don't
want
the
hassle
of
deploying
and
configuring
before
they
can
test
their
changes.
B
Just
on
those
demo
environments
I
mean
that's
even
just
between
you
know,
developers
and
business
people
within
our
company.
It's
it's
really
useful
to
be
able
to
say
and
look
what
we've
created
and
we
can
do
a
demo
and
that
we
can
quickly.
You
know,
spin
up
a
cluster
and
just
ping
a
link
to
somebody
and
can
show
off
the
work
we've
done.
So
that's
a
really
useful
capability
for
both
developer
and
business.
A
A
We
want
this
orchestrator
system
to
be
able
to
handle
data
and
object
separation
between
each
project,
but
what
we
also
wanted
was
dedicated
versioning
for
those
projects,
as
well
as
automated
testing
and
finally
dedicated
developer
environments
for
those
projects.
There
are
external
developers
that
are
collaborating
on
these
projects
to
make
the
synthetic
environments
come
to
life,
and
we
need
to
make
sure
from
from
the
probable
defense
standpoint,
that
access
is
limited
to
all
all
the
access
that
they
need
and
no
more.
B
Yeah
absolutely
and
yeah
it.
I
guess
like
in
a
sort
of
how
you
how
you,
how
you
apply,
that
it
can
be
achieved
through
for
our
back
and
various
other
means
as
we'll
talk
about
in
a
minute
the
sort
of
name
spacing
of
various
jobs,
and
things
like
that.
Yeah
lends
itself
nicely
to
this
sort
of
thing.
That's.
A
A
Open
shift-
that's
nothing!
I
found
the
time
to
do
no.
Okay,
so
the
ground
reveal
what
were
our
technology
choices?
I'm
sure
you're
not
all
shocked,
to
find
out
that
our
decision
for
the
orchestrator
cluster
was
to
use
kubernetes
and
google
kubernetes
engine.
As
I
mentioned,
it's
our
nucleus
for
all
operations.
It
is
our
main
back-end
store.
We
mentioned
that
we
wanted
to
keep
track
inside
of
our
orchestrator
cluster
with
dedicated
versioning.
We
achieved
that
by
using
kubernetes
crds.
B
Yeah
there's
some
very
easy
answer
to
this
question
I
mean
just
the
overhead
of
managing
a
database
to
store
all
of
this
information
is,
is
not
an
attractive
proposition
to
us.
You
know
we.
We
talked
about
ways
of
storing
this
versioning
data
of
all
the
all
the
components
and
how
we
manage
that
kubernetes
out
of
the
box
can
do
this.
You've
got
crds
and
you've
got
fcd,
so
lcd
is
database
for
free
right.
B
A
We
also
have
namespace
separation
with
rbac
for
our
customer
projects,
so
this
was
a
very
powerful
tool
that
we
were
able
to
leverage
using
kubernetes
and
gives
exactly
the
sandbox
that
we
needed
from
project
to
project
to
store
confidential
secrets,
config
maps
of
data,
but
also
the
generation
of
google
cloud
objects
inside
of
those
sandbox
namespaces.
For
each
project.
A
The
orchestrator
cluster
on
gke
also
houses
some
other
cloud
native
toolings,
such
as
prometheus
and
grafana,
for
monitoring
and
observability
valero
for
disaster
recovery,
which
is
particularly
important
as
we're
using
kubernetes
as
our
main
back-end
store
pamerium,
which
is
an
identity
aware
proxy,
which
allows
us
fine
grain
access,
controls
on
public
facing
web
applications
and
external
dns
for
being
able
to
generate
the
the
web
addresses.
We
need
to
be
able
to
provide
these
tools
to
anyone.
We
want
on
top
of
the
orchestrator
cluster
on
gke.
We
also
built
a
dedicated
rest
api.
A
We
also
wanted
the
automated
workflows
that
are
executing
these
tasks
to
be
abstracted
as
well,
so
that
the
orchestrator
is
something
you
communicate
with,
but
you
do
not
need
to
control,
so
I
guess
ollie.
This
was
a
really
important
design
decision
when
we
were
going
about
trying
to
give
developers
a
more
efficient
experience.
B
Well,
that's
exactly
right,
so
our
certainly
my
team's
goal
is
is
to
build
tooling
and
increase
developers
velocity.
So
we
want
to
developers
developing
fast
and
failing
fast
and
fixing
fast
and
succeeding
fast.
So
we
don't
want
them
messing
around
with
having
to
build
their
own
sort
of
yaml
for
deploying
services,
and
this
sort
of
thing
we
don't
want
them
to.
You
know,
go
into
the
google
cloud
console
and
do
gk.
You
know
spin
up
a
gk,
because
we
or
you
know
we
don't
want
to
be
writing
terraform
or
anything
like
that
themselves.
B
So
we
just
decided
to
abstract
all
of
this
away
and
what
it
means
is
they
just
have
we
have
we
put
a
cli
in
front
of
this
rest
api,
so
they've
got
one
place
where
they
can
do
all
the
things
we
talked
about,
so
they
can
deploy
a
development
environment.
You
can
query
it
and
get
its
information
and
then
should
you
need
to
to
log
into
it.
B
You
can
get
that
information
very
quickly,
employ
pretty
much
anything
you
want
on
it,
but
as
part
of
it,
what
it
will
do
automatically
for
them
is
deploy
our
platform
and
the
particular
project
components
that
they're
working
on,
and
so
that's
all
there
in
one
place,
and
they
can
also
use
this
tooling
and
this
api
to
query,
test
statuses
and
results
and
version
information.
So
they
know
what's
in
each
version,
we
wanted
like
a
one-stop
shop
to
do
it.
We
didn't
want
people
worrying
about,
like
you
know,
workflows
or
the
orchestrator,
or
anything
like
that.
B
A
And
I
think
in
terms
of
like
providing
it
through
like
a
really
simple
cli
tool,
it
really
empowers
the
developer
to
be
able
to
like
stay
away
from
the
stuff
that
they
don't
want
to
be
doing.
Maybe
they
do
want
to
be
using
kubernetes.
That
might
be
the
case
sometimes,
and
in
that
case
they're
more
than
welcome
to,
but
this
gives
a
mechanism
for
them
to
be
able
to
do
the
work
that
they
need
to
do
rather
than
spending
lots
of
time
doing
stuff,
they
probably
shouldn't
how's.
B
Team,
I
think
so
far
it's
had
very
good
uptake
from
the
teams.
We've
had
lots
of
positive
feedback
so
far
one
good
thing
about
it
is:
we
were
able
to
sort
of
develop
fast
against
it
as
well,
because
you
know
developers
are
very.
You
know,
we've
got
a
lot
of
clever
people
working
at
improvable
and
you
know
they've
got
a
lot
of
ideas
about
how
things
like
this
should
look,
so
yeah,
really
good
to
sort
of
yeah
get
their
feedback,
and
yes,
so
far,
it's
been
great.
B
It's
it's
been
a
huge
improvement
over
sort
of
manually
having
to,
I
don't
know,
deal
with
like
mini
cube
or
something
or
even
deploying
the
product
on
some
vms
in
somewhere.
You
know
this.
This
is
just
unsustainable
and
and
no
good,
so
yeah
we
had
to
give
them
a
form
of
just
quickly,
spinning
up
ephemeral
environments
and
so
far
it's
working
really
well
for
us.
A
A
We
used
a
combination
of
argo,
workflows
and
argo
events
to
do
all
of
the
automation
that
goes
on
inside
of
this
orchestrator
cluster
on
gke.
The
reason
why
we
chose
argo
is
because
it
fits
right
in
with
our
design
to
make
this
whole
orchestrator
platform
kubernetes
native.
How
is
that
the
case?
Well,
its
use
of
kubernetes
pods
makes
it
truly
language
agnostic
again
in
yaml,
applied
as
crds
you're
able
to
define
workflow
templates
inside
of
those
workflow
templates
are
manifests,
which
explain
the
steps
that
you
want
to
take
place.
A
A
We
can
slide
from
one
to
the
other,
and
it
also
has
integrations
such
as
artifacts
and
parameters,
if
I
say,
do
a
git
clone
on
an
alpine
image
in
one
step,
I
can
take
that
into
the
next
step
and
start
performing
terraform
actions
in
it
by
passing
that
repo
in
an
artifact
which
is
backed
up
on
a
data
store
like
gcs.
The
final
area,
where
I
think
that
argo
was
a
truly
great
choice,
was
its
integration
with
argo
events.
A
So,
through
the
development
cycle
of
this
platform,
there
are
many
different
components
that
are
being
developed
by
many
different
teams.
Well,
luckily,
what
we,
what
they
all
have
in
common,
is
they're
all
being
developed
using
some
git
provider.
In
our
case,
it's
mostly
github
and
argo
events
allows
us
to
have
dedicated
eventing
that
is
instantly
pluggable
into
argo,
workflows
to
automate
the
tasks
that
are
triggered
by
a
github
web
hook.
It
also
has
other
really
strong
integrations,
most
importantly,
the
ability
to
trigger
off
of
kubernetes
resource
changes.
A
All
of
our
versions
are
declared
in
kubernetes
crds,
but
also
same
goes
with
our
environments
that
are
vended
into
gke.
So
it
means
that
when
we
vend
an
environment
with
terraform,
we
can
update
the
status
on
that
environment
crd
to
ensure
that
the
developer
accessing
that
crd,
on
the
other
end
abstracted
away,
can
see
its
status.
It's
being
built
up.
B
Yeah
we
sort
of
spoke
about
this
earlier,
like
I,
I
tend
to
think
argo
events
in
argo.
Workflows
is
one
in
the
same
thing
like
the
orchestra
is
kind
of
turned
into
this
one-stop
shop
for
everything
and
because
they
work
so
seamlessly
together.
It's
not
like
you're
complicating
things
by
adding
multiple
tools.
You
know
these
these
two
things
work
so
well
together.
I.
A
Think
that's
a
very
important
point
and
I
think
I
think
I
experienced
that
as
well,
and
it
really
shows
the
strength
of
of
what
kubernetes
can
do
alongside
tours.
Like
argo
events,
so
continuing
on
with
this
theme
of
cyclical
automation
and
autonomy,
a
component
change
is
made
in
github
by
an
improbable
engineer.
A
This
triggers
a
web
hook
that
is
sent
from
github
to
the
event
source
pod
sitting
on
the
orchestrator
cluster.
Once
the
event
source
is
notified,
it
uses
the
event
bus
to
send
a
signal
to
the
sensor
that
has
been
configured
to
execute
a
workflow
based
off
of
that
event.
From
there
it
checks
the
state
of
each
component's
github
repo,
and
if
it
does
find
a
change,
it
will
know,
because
when
we
generate
a
version,
we
give
it
a
unique
cache
based
off
of
the
components
that
are
configured
for
the
platform.
A
It
then
checks
out
the
specific
commit
of
each
of
those
branches
on
the
github
repo
and
then
generates
a
unique
hash
that
which
is
then
applied
to
the
cluster.
The
hash
is
unique
and
hasn't
been
applied
to
the
cluster
at
this
point
and
is
successfully
applied
to
the
cluster.
This
can
then
be
consumed
by
the
improbable
developer
that
made
that
change
in
github,
and
they
can
start
spinning
up
environments
and
checking
out
the
change
and
how
that
was
received
inside
of
the
wider
platform.
B
A
B
Yes,
so
we
want
to
do
integrated
tests
for
each
versions
and
we've
introduced
the
idea
of
of
the
streams.
So
you
know
you
create
a
new
version
that
gets
that
this
is
where
essentially,
the
qa
journey
for
that
version
of
the
product
starts,
so
it'll
get
promoted
to
the
alpha
stream.
At
that
point,
some
tests
will
run
and
they
can
be
configured
per
version.
B
You
can
add
to
the
version
as
many
sort
of
test
repos
as
you
want
each
container
in
their
own
test
suite,
and
then
those
tests
pass
a
slightly
longer
set
of
tests
which
can
run
called,
which
tend
to
be
like
a
nightly
testing
sort
of
situation,
so
they
run
each
night
on
that
particular
version.
If
they
pass,
then
you
know
we
can
promote
further
and
eventually
we
can
get
to
a
release
which,
at
the
moment
for
us,
is
like
a
manual
process.
B
A
Think
that
I
think
that's
truly
awesome
because,
from
a
developer's
perspective,
the
fact
that
they
can
apply
the
change
in
git
that
they
made,
but
what
this
orchestrator
cluster
goes
and
does.
Is
it
takes
the
change
from
that
specific
developer?
It
doesn't
matter
if
they've
gone
off
for
a
coffee.
It
really
opens
up
the
doors
of
what
the
development
cycle
of
a
platform
like
this
can
look
like
yeah.
B
So
I
mean
one
idea,
and
you
know:
we've
got
developers,
spinning
up
clusters
left
right
and
center
to
do
development
work
on
we
we
need
to
manage
cost
and-
and
things
like
that,
the
the
idea
of
of
debate
being
able
to
deploy
a
version
is
that
it's
going
to
be
the
same
every
time.
So
these
environments
don't
need
to
be
up
all
the
time
they
can
just
spin
up
a
new
one
every
morning.
So
what
what
we've
done?
B
Isn't
you
know
we
put
like
a
time
to
live
on
each
environment
and
that's
based
on
an
extra
field
in
the
crds,
and
we've
got
just
another
argo
workflow,
that's
just
looking
at
each
of
these
each
of
these
timings
for
each
environment
and
figures
out
how
long
they've
been
up
and
if
they've
been
up
for
a
certain
amount
of
time
it
would
just
destroy
them.
So
it'll
run
the
terraform
and
cleanly,
which
is
very
important
cleanly.
Do
it
do
a
terraform,
destroy,
bring
everything
down
and
remove
all
the
resources
in
the
cloud.
B
A
Think
that's
a
really
exciting
part
of
our
implementation
and
I
think
it
even
backs
on
further
into
into
our
choice
for
kubernetes,
as
our
data
store,
because
that
time
to
live
is
is
described
in
a
kubernetes
annotation.
It's
you
know
a
very
subtle
piece
of
metadata,
that's
attached
to
the
crd,
but
what
it
allows
us
to
do
is
like
it
is
without
doubt
with
or
authority,
decide
a
date
and
time
in
which
this
environment
should
be
yeah
should
be
decommissioned.
A
If
there's
one
thing,
that's
that
that
project
managers
like
more
than
a
successful
project,
it's
being
able
to
keep
it
as
a
lower
cost
as
possible.
I
know
it's
been
very
well
received
that
we're
able
to
keep
these
costs
low.
One
other
thing
as
well
is
persistent
disks.
Of
course
we
spin
up
our
environments,
our
gk
environments
with
terraform.
A
We
spin
them
up
and
we
spin
them
down,
but
gke
keeps
hold
of
the
persistent
disks
inside
of
the
google
cloud
projects,
but
we
were
able
to
avoid
that
by
just
adding
a
simple
step
in
the
auger
workflow.
That's
not
necessarily
something
that
we
can
easily
integrate
inside
of
that
terraform.
Destroy
step,
why
not
just
add
an
extra
workflow
step,
so
they
were
the
areas
where
we
thought
the
argo
workflows
and
events
were
a
great
pairing
for
our
work
on
the
delivery
service
for
the
improbable
defense
platform.
A
Through
this
presentation,
we
have
taken
you
through
the
story
of
how
we
take.
We
took
a
selection
of
problems
that
improbable
defense
were
experiencing
through
the
development
of
their
platform
and
to
combat
this,
how
we
built
a
secure,
kubernetes
native
delivery
service,
which
automates
the
test
and
release
cycle,
giving
improbable
the
flexibility
to
expand
further
to
separate
iterations,
while
enabling
easy
collaboration
with
external
entities.
A
I'd
like
to
thank
you
all
very
much
for
coming,
I
hope,
you're
having
a
really
good
cubecon
and
if
you
have
any
questions
whatsoever
we'll
be
in
the
q
a
afterwards
so
feel
free
to
ask
any
questions
there,
but
also,
if
you
have
any
questions
about
jet
stack,
if
you're
interested
in
the
sort
of
work
that
we've
discussed
in
this
presentation
feel
free
to
get
in
contact
with
me
in
fact
feel
free
to
get
in
contact
with
me
about
anything.
I'm
happy
to
discuss
anything
kubernetes
or
not.
A
Kubernetes
related,
so
thank
you
very
much
for
coming
to
the
talk.
Thank
you
kubecon
and
thank
you
cncf
for
for
inviting
me
on
to
speak.
Also,
thank
you
for
the
entire
improbable
defense
team
that
we've
worked
with
throughout
this
project.
You've
all
been
fantastic
to
work
with,
and
it's
been
a
joy
delivering
this
service
for
you
guys
and
well
done
to
the
jet
stack
team
on
my
side,
who
have
helped
me
along
the
way,
with
all
the
all
the
challenges
that
I
faced
throughout
so
olly.
B
Yeah,
thank
you
very
much
tom.
I
mean
yeah
been
fantastic
to
work
with
on
this
project
and
yeah
to
everyone
out
there.
If
you
are
again
just
mirroring
what
tom
says,
but
for
ourselves,
if
you
are
interested
in
you
know
what
we're
doing
with
synthetic
environments,
you
know
the
the
problems
we're
trying
to
solve.
You
know
simulating
it's.
You
know
thousands
or
millions
of
entities
at
scale.
You
know
kubernetes
based
platform
and
and
delivery
yeah
get
in
touch
with
us
as
well.